We describe the system we built to support ML in search at Mercari, Japan’s largest C2C e-commerce platform. We start by describing the journey to enable the use of ML in a “traditional” term-based search infrastructure with high throughput and strict latency requirements. We also discuss the mixed blessing of rushing a successful proof of concept into production and the technical challenges this posed on the infrastructure side. Next, we discuss the nuts and bolts of data engineering, ETLs, training pipelines, and serving/monitoring our ML model in production. We also discuss some of the weaknesses of our initial homegrown system, including A/B testing and model monitoring. Finally, we discuss our efforts to evolve our homegrown system into a more modern MLOps infrastructure using an A/B testing framework and Seldon for traffic routing and model serving.