Despite best intentions, Open Source releases with regression errors are published every day. In the best case scenario, a downstream user detects it early thanks to good tests, files an issue, and the maintainer can fix it before too many people have upgraded. Other scenarios involve various degrees of brokenness and games of "is it broken for everyone or just me?". Renovate Bot is an open source dependency automation tool but which also is run as a free app on github.com, where it is installed into almost 200,000 repositories. A feature called "Merge Confidence" helps downstream users know if a release is likely good or not based on automatically sourced crowd data (tests, deployments, rollbacks). Now we are planning to turn the focus upstream to help open source maintainers get an early indication of accidentally breaking releases and even provide a mechanism for downstream users to opt into silent pre-release testing so that major features can be smoke tested downstream before release.