Anyway, I work for Red Hat in the desktop team working on hardware enablement and I've been spending the last, I don't know, one and a half, two years working on Thunderbolt enablement. Okay, so what is this Thunderbolt anyway?

And Intel says it's the USB-C that does it all. And so I think C in USB-C is a very good name because for me it's the first indication that it's a very confusing thing because Intel on their marketing slide put a connector there,

A lot of people keep getting it wrong and I've been, since I've worked on this project, I've been trying to educate people about the differences between USB-C, Thunderbolt and everything in between because USB-C is the connector type and not the technology. Anyway, so a brief history about Thunderbolt.

It was first introduced in 2009. It was back then an Apple thing, it showed up mostly on MacBook Pros and stuff. It could do 2x10 gigabits of data transfer, had display port 1.1a and the connector it used was a mini-display port. It was not very widely used, I think.

Then there came version 2 which used a different controller chip, default controller. It was also basically mostly shipping on MacBook Pros, so not very widely expected. They opted to the spec a bit, mostly the display port. And then in 2016, I think Intel made a big push to drive this to more general adoption,

not only in MacBook Pros but more in the general market. And this is, I think to some extent, successful. I think all new modern laptops that you buy, like the Lenovo's from two years ago and all the Dells and whatever, they all nowadays ship with a Thunderbolt controller.

And the most commonly used nowadays is the Alpine-rich controller introduced in 2016. And then in 2017, Intel made another push to widespread the adoption, or make the adoption more widespread, and that declared it's going to be a royalty-free standard.

So other manufacturers could also adopt it. And I think there are some AMD motherboards these days where you can actually have a Thunderbolt controller. And they updated the specs quite drastically. So they updated it to double the speed, to 40 gigabits per second, to PCI Express 3.0. And they used a new USB-C connector.

And very recently, actually, I think a few weeks ago, USB 4.0 was announced. And one of the features of it is that it's Thunderbolt 3.0 compatible. So the new USB 4.0, which also is going to use the USB Type-C connector, is going to be even more confusing because it can optionally have Thunderbolt 3.0.

This is going to be a lot of fun. Yeah, so it's the USB Type-C connector, it has to be that thing. So you can have USB, it features native USB 3.1. So you can have a USB 3.1 connector with the old port.

But if you have a USB Type-C connector, it can also only just be USB 3.1. But if it has a little symbol, this Thunderbolt symbol, then it's a Thunderbolt connector. But every Thunderbolt 3 port will have a USB Type-C connector. And if you're already confused, that's the main point, it's confusing.

And people get it wrong all the time. But the main feature that it has, and that's why it's also quick, is it has four PCI Express 3.0 lanes. It can also, over the Thunderbolt fabric, it can channel eight DisplayPort lanes,

DisplayPort 1.2, and the newest chip, Titan Rich 1.4. Of course, you cannot tell from the outside that you have a Titan Rich controller or an Alpine Rich controller, because it's the same port, it's the same symbol. So you have it or you don't, and you don't know it. With the USB Type-C connector also came USB power delivery.

So you can actually charge your laptop via the same port that you connect your peripherals with, which is very cool. You can do up to 100 watts for charging, which is not enough for all laptops. So some of the laptops have a USB Type-C connector and then another little connector next to it that you have to still plug in, because 100 watts is a lot, but not for 15-inch laptops

with an external graphic card or discrete graphic card. The newest chip, the Titan Rich controller, also can act as a USB sync mode, which is mostly interesting for docks. So you have a Thunderbolt dock with the new chip, and you plug it into your USB Type-C port that is not a Thunderbolt port,

and it will also work with limited functionality, adding more to the confusion. Because now you can mix and match ports and you get a result, but you get different results depending on which ports you use. One big feature that they added was security modes, and I'll talk about this in a second.

And the main idea is that you drive external storage and graphics cards and docks, mainly docks. The internal architecture of the Thunderbolt chip of all the third-gen chips is basically the same. You connect PCI Express lanes to the Thunderbolt chip, you get the DisplayPort lanes into the Thunderbolt chip.

Internally, there's a PCI Express switch, a USB controller, and then the Thunderbolt switch that will wrap all these packages, the USB packages or the PCI Express packages, into the Thunderbolt packages and send them on the Thunderbolt wire. You'll notice that there's also arrows not going to the Thunderbolt switch,

but they go directly to this Muxer area there, and then to the chip, which is because if you plug in a USB-C device or a DisplayPort device, then they will bypass the Thunderbolt and the port will go into what is called the alternate mode,

and will directly access the USB or the DisplayPort. It would actually not have anything to do with the Thunderbolt port at this point. And then for external peripherals, if the packages go into the Thunderbolt controller, get wrapped into the Thunderbolt packages,

and then at the end, which is basically the same chip that you have in the host, but in an endpoint mode, they will unwrap the packages and then there's going to be another PCI Express switch or USB hub in there, or the DisplayPort signal, and they will unwrap this and then transport it to the devices.

And here's a very good example of a UX failure of these USB Type-C ports and Thunderbolt ports. I think this is the T for ATS or something that is currently in production from Lenovo. The very left USB Type-C port, you can charge your laptop, you can do USB 3.1, but you cannot do Thunderbolt.

The Thunderbolt port is the one on the right next to it, the one that does not really look like a USB port because it's together with the proprietary Lenovo network thingy. So I get, I guess, like once a month at least,

I get a bug report, the Thunderbolt is broken, and then people realize they plugged the thing in the rock port. And it's very embarrassing for everyone involved. So the connection mode is another thing that you can do in the BIOS. You can change how the Thunderbolt port actually works. You can put it into USB-only mode and it will only do USB.

You can put it into DisplayPort mode and it will only do DisplayPort. You can do it in the DisplayPort and USB mode and can do both. And then there's the normal operation mode of Thunderbolt port, which is Thunderbolt 3 where it does actually support all the above. And Thunderbolt.

And because Thunderbolt is basically PCI Express, and with PCI Express you can do direct memory access, means your peripheral can basically write and read into the main memory as they please, bypassing CPU control, because that's the whole idea of very fast I.O.

People came up with numerous attacks to the Thunderbolt port. I mean, it started out with FireWire from Apple actually, and then with Thunderbolt 1 and up to Thunderbolt 3. I think this year there was a paper released, and they had a cute new name for it, because that's a trend that you have cute new names for security issues, called Thunderclap.

And it was actually a very good paper. They had an FPGA board simulating an external network card over Thunderbolt. And not only did the very trivial attacks, so reading out in memory via normal DMA, but even for systems which had already IOMMU enabled,

which I'm going to talk about later, they found attacks and basically bypassed all of the IOMMU protection from Mac OS and Windows. Linux by then didn't have IOMMU protection, so there was nothing to bypass. We were unsecure anyway. Yeah, so what Intel thought of as a way around

is they added security modes to a Thunderbolt controller. That's the new thing in Thunderbolt 3. So by default, there's no security at all, which is really a bit stupid, but that's the default how most letters are shipped, until Thunderbolt 3.

And then in Thunderbolt 3, they defaulted to the user mode, which means that before the Thunderbolt tunnels are actually created, you have to, from user space, authorize the device. So you plug it in, you will see that there's a device connected to it, into the port, but the Thunderbolt connections will not be built, so there's no PCI Express packages going over this thing,

so there's no DMA. And there's other modes, so you can also set this in the BIOS, you can set it to Display Port Only mode, and USB Only mode is basically like the alternate modes we saw before. And there's also SecQ mode, where you can actually challenge the device with a key

that you first imprint on the first connect, and then on Subsequent Connects, you can verify that this is the same device that you authorized some time ago. But sadly enough, most laptops actually, even nowadays, ship in user mode. So you authorize the device, but the only way you identify devices is via some UUID,

and you can obviously fake that UUID if you know it. So how do we do this thing on Linux? The main interface to the kernel is SysFS, so the kernel has device drivers for Thunderbolt, and a lot of stuff is in PCI Express, in the PCI Express subsystem,

and it's all exposed in SysFS. And then there's a new system daemon called BoltD, which talks exclusively to the SysFS interface, and exposes the DBus API, and then there's a command line utility called BoltControl, a BoltCuddle,

and the Gnome shell talks to it, Gnome settings talks to it, the firmware update daemon nowadays talks to it, and the idea was that it's also open for other desktop integrations, I'm not sure if anybody actually has done that. How does the kernel interface look like? Well, it's basically a device tree. So there's SysFS Thunderbolt,

and there's a devices directory, and in this devices directory, there's device nodes for all the devices that you have connected. There's two special ones, there's the domain controller exposed, and then there's also another device for the host itself, for the computer itself. And then there's a bunch of files that you can read and write from, read from and write to control how the Thunderbolt devices are behaving.

The first version, the first report for Thunderbolt was in the kernel 4.13, and that was in September 2017, so it's relatively new actually.

The Bolt first release was also in December 2017. And then there were a number of features added by Intel and by me to the Bolt daemon over the last two years. The most prominent one is probably that we got the pre-bolt as access control lists in 4.17 and Bolt 06,

and then we finally got an IOMU in 5.0, but that was like before the Thunderclap paper came actually out. It turned out that a bunch of the stuff that was vulnerable in Mac OS was also vulnerable on the first implementation that we did in Linux.

So we did some more fixes to the IOMU's implementation in 5.1, and actually very, very recent, so this landed like two weeks ago or something. What will be in 5.4 is going to be the bounce buffers, and only with the not yet released kernel, we will be completely safe until the next security leak comes up

for the attacks that were mentioned in the Thunderclap paper. And you need new hardware for this. So for the IOMU support in Thunderbolt, you cannot do firmware upgrade or anything, you actually need a system that is certified for Windows 1803 or something,

so you need a very recent system. I haven't had my hands on any of those, so we are all still unsafe. So how does the security levels look like? You set the security level in the BIOS, that's why you change it, you cannot change it at runtime,

you set it once by boot and that's why it is. It will be communicated to us via the security SysFS attribute of the domain controller, and then every device has an authorized field and a unique ID. The unique ID we use to identify the device, and the authorized field you can read where the device is authorized, and if you are a system administrator,

you can also echo into it and write to it, and with this leak you actually also authorize it. Exactly. Then there's also, if you have security level 2 enabled, then there's also a key attribute there, and with this key attribute you can do the device challenge. So the first time you actually authorize a device, you write in a unique key into that field,

and then a subsequence connects. You specify the key in there, and then in the device they will do the matching, and if the key doesn't work, it will refuse to connect. There's a command line tool that you can use to authorize a device,

and then in Bolt there is a policy file per device, so you can say, authorize it just once or authorize it, and the next time it's connected, automatically authorize it as soon as you see it. And in GNOME, the shell will do this for you. You don't have to do it manually,

but if you run in GNOME, then the GNOME shell will automatically authorize devices as soon as you connect it, if you're logged in and you are system administrator. There's also a GNOME settings panel which you can use to connect the devices and manage devices. There's also a global flag, which maybe is a good idea if you go to conferences, there's a global switch where you can turn off authorization.

So for example, if you go somewhere and you plug in a USB type C cable from an unknown source, and you have your laptop unlocked, then we would normally authorize it, and it would grant this device access. So before a conference, you flip the switch, you plug it in, and we don't authorize it. So the display port will always work,

because display port is just passed through, but the PCI lanes will connect it, and you will be safe, hopefully. There's also two different modes that change, and we got support for this in 4.18 and 4.19, but I'm just going to skip it, because this is not super interesting.

And if you want to use this, your Thunderbolt device is also during boot, or during pre-boot, like to enter your looks password or something, then there's a new feature called pre-boot access controllers. You have to enable it in the BIOS, because it will basically disable the secure mode,

because there's no key for it, it's just UUID based. And both will basically do the right thing for you automatically. And last but not least, in 5.0, as I said, we get IOMU support, so if you have the right hardware, and if you have the newest kernel and the newest bolt, then we basically bypass the security modes.

We will always authorize the device if the kernel communicates to us that the support for it is there, which is communicated via this attribute, IOMU DMA protection. And as I mentioned, there's a number of fix-ups that we had to do in the kernel, so if you really want to be safe, you have to have the latest hardware,

the latest kernel, 5.4, which is not released yet, because only there it contains all the fixes that were mentioned in the Thunder Club paper. And if you're interested more about this, you can talk to me, or there's a very nice LWN article about the bounce buffers and the timing attacks and all the other stuff.

Anyway, USB4 is going to be basically Thunderbolt plus USB, and that's it.

Hi, so you mentioned that you need to have the latest hardware to take advantage of some of the mitigations. Is that the later controller you mentioned, or is it a firmware thing? It's a firmware thing. So on Intel hardware, it's using the Intel VTD IOMU thing,

and it's a combination of firmware and hardware, and it's so recent that I haven't got, I'm begging for some test hardware, because I released Bolt with support for this without ever actually testing myself. It's been tested by Dell and Intel that it works, but I actually have never verified that my code actually works.

Is there particular hardware you have in mind that someone would be able to furnish that would help you get past that?

What you mean, get past that? Is there a specific hardware that you need that you might be able to, someone might be able to get your hands on to fix it? Yeah, well, I mean, there is hardware in Red Hat, but it was scarce. It's like you have only five machines with the newest chip from Intel that has those things,

and one is in Westford and one is in Brno, and it's not where I am. And they didn't want to give it away because they also needed for enabling the graphics chips and something.

So are the pull requests already in Linux 4.5 master branch now? So they will be in RC1. Exactly, they are in there, yeah. I mean, the first, like these bounce buffers were a bit of a, it took a long time to get, I think the first version of this was posted in March.

It just took a while until, because there was some overlap with the software-based IOMU, and then there was some idea to like unify this, and it didn't actually work out, so it just took a while to get in. And do you have an idea why yesterday there were certain problems with the adapters here? I don't know.

I get those questions a lot, but I don't know. Casey. So you mentioned that there are all these great things

that the device manufacturers have to do, and as we all know from all the hardware standards, they don't. So what sort of things have you seen in the real world that Thunderbolt 3 devices, like just the docs and things like that have done wrong? Oh my God. Good question. Well, I mean, what I've seen actually

is that some of these logos, which are required by the specification to identify the ports, right, there's a whole USB specification and the whole Thunderbolt specification which require you to put the logos on there, and some of the logos are actually not correct. So like they're missing, or you know. And then the obvious problem is that there's two different types of cables,

Thunderbolt cables, there's active fonts and passive fonts, and you have no way to tell from the outside. So like, I don't know how this is really bad UX, and I mean, this laptop also, these two ports here are Thunderbolt ports, this one is not.

And I mean, sure, if you look at the logos, you notice, right, but I still think it's very bad UI. I think on this one you can even just boot from the live USB stick on one side. On this side, yeah. Unless you enable the BIOS support for Thunderbolt,

then it will authorize the, it's a bit silly. And where can you, do you have this FPGA stuff to test this, or do you know how expensive it is, or where to get this? I know that we internally wanted to get our hands on it, and we couldn't, because QA was eager to play around with it.

Since we're asking you all of our Thunderbolt tech support questions, what dock do you recommend? Oh, God. God, can I lose off the record? No, so I personally use the Lenovo Thunderbolt dock,

but, and I'm okay with this, but I know a lot of people in the office that have had problems with this. And I also, like the TB16 dock from Dell, there were some USB issues in older versions of the Linux kernel with supposedly a fix,

but I, okay, they're not. All right, so yeah. If you just use the dock for power and display, it's fine if you don't use the internal network card, because that sometimes then craps itself because it's connected via USB. Which I don't, this is so that I don't have the Thunderbolt dock from Lenovo.

But I mean, it might also be a kernel regression, because this one slide I didn't show, there's different modes for the Thunderbolt controller, and we only recently, very recently, gained support for full runtime management in the Thunderbolt controller, and then new laptops do suspend to idle, and like runtime PN and suspend to idle

apparently leads to some of the Thunderbolt controllers and some hardware not to wake up anymore. Thank you, Christian.