In the domain of search technology, the precision of matching user queries with the most relevant products is paramount. This session ventures into the forefront of relevance engineering, spotlighting a technique that marries unclear signals from user-traffic logs with programmatically obtained relevance labels from large language models (LLMs). We focus on improving the labels used for learning to rank models, central for mature search systems. We'll dissect the process of integrating user-generated data with synthetic relevance labels, underscoring the synergy between real-world behaviors and artificial intelligence insights. This blend not only addresses the inherent ambiguity in user queries but also significantly amplifies the performance and accuracy of search results. The presentation will guide you through the intricacies of collecting, processing, and integrating these diverse data sources. We'll navigate through the challenges this novel approach presents, proposing robust solutions and best practices for implementing this strategy effectively.