Giskard is an open source project that implements automatic tests for Machine Learning models. In this conference talk, we'll explain (1) why testing ML models is an open problem, (2) why testing ML models requires different tools compared with traditional software systems. We'll then explain, using concrete examples, how our Giskard project helps ML Engineers deploy their AI systems into production safely by (1) designing fairness & robustness tests and (2) integrating them in a CI/CD pipeline.