The attack surface of USB is quite large and while disabling USB altogether solves the problem, it creates many other. As do existing protection mechanisms. They suffer from poor usability and missing integration into the operating system. We present our approach to defending against rogue USB devices for a GNOME-based operating system. USB is arguably to most exposed interface of a user's machine. It allows an attacker to interact with pretty much any driver, many of which are of questionable quality. We try to be smart about when to allow new USB devices. First, we try to detect when the user is present, arguing that if the user is not then new USB devices should not work. But it is not that simple, because you might very well want to attach a new keyboard in case yours breaks. Keyboards, however, pose another risk as several attacks have shown. But not all keyboards are equally bad and we propose to promote a harmless keyboard to become dangerous after getting the user's consent. It is not entirely clear yet how to that best, so your feedback and a discussion is certainly welcome!