Uncover, Understand, Own - Regaining Control Over Your AMD CPU
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 254 | |
| Author | ||
| License | CC Attribution 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/53126 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
ArmBefehlsprozessorStudent's t-testLevel (video gaming)MicroprocessorLogic gateGroup actionCovering spaceInformation securitySystem callFirmwareComputer animationLecture/Conference
01:22
VirtualizationInformation securityFirmwareStudent's t-testEncryptionBefehlsprozessorHypothesisVisualization (computer graphics)Right anglePresentation of a groupComputing platformPlastikkarteComputer animationLecture/Conference
02:17
ArmBootingComputer hardwareInternet service providerData managementServer (computing)FirmwareVirtual realityPhysical systemRead-only memoryKernel (computing)Patch (Unix)Digital rights managementSystem programmingControl engineeringCodeMotherboardSource codeBefehlsprozessorFlash memoryTuring testComputer-generated imageryUniform resource locatorBootingHard disk driveServer (computing)Point cloudForm (programming)Level (video gaming)BefehlsprozessorFunctional (mathematics)Flash memoryCodeData storage deviceOpen sourceMiniDiscOperating systemInheritance (object-oriented programming)Physical systemKey (cryptography)Content (media)Kernel (computing)System administratorConnectivity (graph theory)BootingMotherboardPhysicalismVirtualizationDevice driverComputer hardwareProcedural programmingInformation securityCartesian coordinate systemElectric generatorGoodness of fitSemiconductor memoryFile formatLatent heatMedical imagingData managementFirmwareArmDigital rights managementBlack boxData flow diagramSimilarity (geometry)Data centerRight angleComputing platformVirtual machinePresentation of a groupInternet service providerIntegrated development environmentSound effectService (economics)DreizehnCASE <Informatik>Slide ruleShooting methodMultilaterationComputer animation
07:23
VolumeFunction (mathematics)ArmDigital rights managementIntelMIDIDirectory serviceCountingComputer fileTable (information)FirmwareLatent heatComputer-generated imagerySource codePointer (computer programming)Revision controlData typeInformationAddress spaceLine (geometry)Interface (computing)Key (cryptography)Finite element methodControl engineeringCodeComputer programSequencePoint (geometry)Latent heatAnalytic continuationDirectory serviceMedical imagingTouchscreenTable (information)Field (computer science)Source codeLine (geometry)Element (mathematics)NumberAddress spaceFunction (mathematics)Web 2.0CASE <Informatik>Programmer (hardware)Inheritance (object-oriented programming)Speech synthesisRight angleUtility softwareMultiplication signOpen sourceInformation securityFirmwareImplementationProjective planeMachine codeComputer fileLogicInformationFile systemRevision controlBitText editor2 (number)CodeMikroblogComputing platformMathematical analysisComputer forensicsStandard deviationVolume (thermodynamics)Computer hardwareString (computer science)Process (computing)Data structureFlash memoryGraphics tabletBootingBlock (periodic table)WhiteboardBefehlsprozessorBootingEmailCentralizer and normalizerHexagonElectronic signatureType theoryOrder (biology)ArmParsingBinary codePointer (computer programming)Core dumpProgram flowchart
12:30
Computer programLogikanalysatorFunction (mathematics)CASE <Informatik>Line (geometry)LogicOffice suiteInheritance (object-oriented programming)BootingFlash memoryUniform resource locatorReading (process)Procedural programmingRow (database)WhiteboardAddress spaceState of matterFitness functionLecture/Conference
13:38
ParsingInformationPhysical systemData typeRange (statistics)Table (information)FirmwareComputer hardwareComputer fileBootingFlash memoryHash functionEmailStructural loadDirectory serviceFormal verificationBitMultiplication signOrder (biology)Function (mathematics)Speech synthesisCAPTCHAPetri netRead-only memoryBootingPublic-key cryptographyHash functionFirmwareBootingBefehlsprozessorData structureFlash memoryKey (cryptography)Lenovo GroupEmailDirectory serviceTable (information)Cartesian coordinate systemEnumerated typeMotion captureTracing (software)Computer fileComputer hardwareModule (mathematics)Procedural programmingReading (process)File systemRootField (computer science)Electronic signatureCryptographyRepository (publishing)Connectivity (graph theory)Physical systemProjective planePosition operator2 (number)Hydraulic jumpWebsiteSemiconductor memoryNetwork topologyFunktionalanalysisLecture/Conference
17:12
Core dumpComplex (psychology)Thread (computing)Read-only memoryBefehlsprozessorTable (information)BootingService (economics)Web pageCodeSupport vector machineAsynchronous Transfer ModeMereologyEmailTuring testStructural loadLatent heatPhysical systemRun time (program lifecycle phase)Operations researchTelecommunicationData managementComputer networkControl engineeringAddress spaceComputer musicSynchronizationString (computer science)Parameter (computer programming)Point (geometry)FirmwareFlash memoryHydraulic jumpDigital filterBroadcasting (networking)Information securityState of matterLocal ringPrinciple of maximum entropyRaster graphicsThread (computing)Data structureCore dumpOrder (biology)FunktionalanalysisComputer hardwareModule (mathematics)BefehlsprozessorFirmwareComplex (psychology)Endliche ModelltheorieContent (media)Group actionBlock (periodic table)BootingSlide ruleWebsiteFunctional (mathematics)InformationSystem callLevel (video gaming)SoftwarePhysical systemReading (process)Right anglePhase transitionBootingOperator (mathematics)Flash memoryInformation securityTraffic reportingSound effectExecution unitBitSemiconductor memoryQuicksortFile formatCodeModal logicAreaMultiplication signBuffer solutionProcess (computing)DialectLetterpress printingBus (computing)Cycle (graph theory)System administratorStructural loadReverse engineeringPublic-key cryptographyNetwork socketLink (knot theory)MultiplicationSingle-precision floating-point formatLatent heat32-bitString (computer science)Socket-SchnittstelleNumberOcean currentStatic random-access memoryCryptographySpacetimeAddress spaceCartesian coordinate systemAsynchronous Transfer ModeMobile appArmBootstrap aggregatingBinary codeSierpinski triangleMaxima and minimaFlow separationSupport vector machineEncryptionFormal verificationTotal S.A.SpeicherschutzVirtualizationVirtual memorySeitentabelleRun time (program lifecycle phase)Kernel (computing)Web pageService (economics)WritingFunction (mathematics)Computing platformTelecommunicationPhysicalismWorkstation <Musikinstrument>TetraederData managementPeer-to-peerState of matterCausalityExpressionView (database)Identity managementPhysical lawRow (database)Extension (kinesiology)WhiteboardBlogForcing (mathematics)Error messageLecture/ConferenceProgram flowchart
27:11
Core dumpRing (mathematics)Read-only memoryWritingAddress spaceSystem callCodeProcess capability indexPoint (geometry)Digital object identifierMIDIDivision (mathematics)Computing platformFunction (mathematics)Server (computing)EmulatorRepository (publishing)Interface (computing)FirmwareElectronic visual displayComputer-generated imageryComputer hardwareKernel (computing)Proxy serverLibrary (computing)Core dumpPhysical systemNumberLimit (category theory)Library (computing)Virtual machineCodeAddress spaceGame theoryData managementRoboticsSoftwareInterface (computing)Mobile appSystem administratorProxy serverFunctional (mathematics)Statistical hypothesis testingPoint (geometry)Computer hardwareComputing platformKey (cryptography)State of matterShared memoryDifferent (Kate Ryan album)Keyboard shortcutRepetitionReading (process)System callPrimitive (album)Device driverSemiconductor memoryPublic key certificateRepository (publishing)WritingAdditionSoftware developerParameter (computer programming)Tracing (software)ResultantMultiplication signPeer-to-peerSpeicherschutzLipschitz-StetigkeitSpacetimeBootingEmulatorInformation securityReal numberDataflowFlow separationEncryptionServer (computing)FirmwareWrapper (data mining)Gastropod shellEmailInterrupt <Informatik>PCI ExpressLecture/ConferenceComputer animationProgram flowchart
30:57
Directory serviceEmailComputer fileMathematicsService (economics)Web pageBootingKey (cryptography)Control engineeringAsynchronous Transfer ModeParsingHyperbolic functionKernel (computing)Read-only memoryMemory managementCodeFlash memoryAddress spaceHash functionMereologyFlash memoryFirmwareDirectory serviceBootingWeb pageService (economics)Public-key cryptographyBootingPhysical systemParameter (computer programming)Right angleContent (media)SpacetimeFunktionalanalysisNumberElectronic signatureCartesian coordinate systemHeegaard splittingConnectivity (graph theory)Maxima and minimaData structureSemiconductor memoryOperator (mathematics)CodeBefehlsprozessorParsingBitSheaf (mathematics)Inheritance (object-oriented programming)Computer fileKernel (computing)Address spaceAsynchronous Transfer ModeEmailFehlererkennungRun time (program lifecycle phase)Virtual memoryType theoryFile systemSource codeInformationWebsiteNetwork topologySpeech synthesisWordAuthenticationPersonal area networkOrder (biology)Forcing (mathematics)Group actionProcess (computing)ChecklistMultiplication signLecture/ConferenceComputer animationProgram flowchart
37:53
Flash memoryEmailRead-only memoryDirectory serviceHash functionCodeAddress spaceTable (information)Web pageKernel (computing)Asynchronous Transfer ModeControl engineeringParsingStructural equation modelingBootingFirmwareImplementationSystem programmingIntegrated development environmentEncryptionVirtualizationComputerStructural loadBefehlsprozessorJames Waddell Alexander IIRepository (publishing)Reverse engineeringPatch (Unix)Reading (process)Software bugBefehlsprozessorBootingDirectory serviceParsingPatch (Unix)Semiconductor memoryOperating systemSheaf (mathematics)SpacetimeServer (computing)CodeEncryptionIntegrated development environmentKey (cryptography)Electric generatorLatent heatSeries (mathematics)LaptopBitCartesian coordinate systemInformation securityRollback (data management)Asynchronous Transfer ModeKernel (computing)Ring (mathematics)CausalityBootingTheorySeitentabellePhysical systemReal numberDigital electronicsVisualization (computer graphics)INTEGRALRight angleVirtual memoryDisk read-and-write headProcess (computing)RippingGame theoryPresentation of a groupDigital photographyGroup actionPlanningSystem callView (database)Interrupt <Informatik>Computer animation
43:11
Video gameSeitentabelleNumberMultiplication signFlash memoryEmulatorLecture/Conference
43:59
BootingCodeSemiconductor memoryPower (physics)Physical systemDirectory serviceLecture/Conference
45:14
Power (physics)PlanningFirmwarePhysical systemSemiconductor memoryLevel (video gaming)Dynamic random-access memoryCodeData managementBitBootingFunctional (mathematics)Projective planeComputer fileLecture/Conference
47:07
Graphics processing unitCodePCI ExpressMixed realityData managementThomas BayesForm (programming)InternetworkingLecture/Conference
47:56
Public-key cryptographyBefehlsprozessorElectric generatorKey (cryptography)InternetworkingBitMultilaterationVirtualizationFirmwareRevision controlInformation securityComponent-based software engineeringBootingRollback (data management)Structural loadMobile WebFunctional (mathematics)EncryptionFlow separationRoboticsLecture/Conference
49:39
Exploit (computer security)Core dumpBootingPhysical systemCodeAsynchronous Transfer ModeInterface (computing)SpacetimeSemiconductor memoryBootingSystem administratorCuboidAreaFigurate numberData managementLecture/Conference
50:56
Open sourceFirmwareBootingCodeSource codeFunctional (mathematics)BootingCore dumpTheoryHidden Markov modelMultiplication signInternetworkingPhysical systemLecture/Conference
51:55
Physical systemFlagBootingBlock (periodic table)RoboticsLimit (category theory)Interface (computing)QuicksortMatching (graph theory)Electronic signatureBitMobile appFlash memoryLecture/Conference
52:41
Internetworking1 (number)String (computer science)Information securityLine (geometry)Connected spaceCartesian coordinate systemInheritance (object-oriented programming)Computer fileComputer hardwareAsynchronous Transfer ModeMobile appLecture/Conference
54:33
EmulatorProcess (computing)Multiplication signConnectivity (graph theory)Physical systemInformationFirmwareBootingFile systemTracing (software)HypothesisLecture/Conference
55:34
Touch typingInternetworkingLecture/ConferenceComputer animation
56:20
Computer animation
Transcript: English(auto-generated)
00:23
understand and own regaining control over your AMD CPU and I must say it's the days where a your homebrew PC would have been like one CPU plus a lot of discrete logic those days are long long gone and now every single device
00:41
probably even this microphone is full of microprocessors it's pretty crazy Robert Alexander and Christian discovered that the arm sorry that discovered an actual arm processor on an AMD CPU which I find quite mind-boggling and and it actually includes its own firmware and to talk about that I like
01:05
to welcome them onto the stage and and I'm really looking forward to hearing all about this discovery and what it has for consequences for us so thank you very much
01:22
alright thanks so before we dive into the topic quick introduction so this is Christian and this is Alex and I'm Robert and the reason why there's three of us today is I'm a PhD student at the technician was it hit in Berlin and beginning of 2018 I was looking into the secure encrypted virtualization
01:43
technology from AMD and this technology requires a firmware running on the secure processor of AMD and that's where Christian came into play because he was looking for a master thesis now Christian is done with this thesis and Alex here kind of took over his work but today we're going to explain to
02:04
you what the embassy could process is doing and what we have uncovered so with that I'm going to hand over to question so let's dive right into our first part of the presentation which is about reverse engineering a completely
02:23
unknown subsystem and when we started our research we had to find out what the AMD secure processor formerly called platform security processor and this talk PSP actually is and it's a dedicated security subsystem that is integrated into your AMD CPU both on server and desktop CPUs it's an arm
02:47
cortex a5 inside your x86 CPU and it's there since around 2013 it runs a so-called secure OS and a kernel and it's actually undocumented and
03:03
proprietary it has access to some secure off-chip storage for the firmware and some some data and it mainly provides crypto functionality to the main CPU as well as key generation and key management functionality it's
03:23
required for the early boot in fact it's required for secure boot and it's it acts as a trust anchor in in your system so the PSP is a security subsystem so it adds security to our system and that's that's good right you
03:44
might notice that this has some similarities with the Intel management engine which on this very stage we heard a lot about three hours ago so let's look into the applications of this yeah on of this piece of hardware
04:02
for that we need to talk about trust the one form of trust AMD tackles in what they call secure encrypted virtualization so you as a cloud customer can be sure that your virtual machine can even run in an untrusted physical location for example in a data center the PSP that is running inside
04:25
that server CPU acts as a remote trusted entity for you as a customer and it promises you to protect your memory so your data from the hypervisor and even from physical access for example through an data center administrator
04:47
the other form of trust that the PSP tries to to establish him is now arriving in the Linux kernel and that's an API to a trust execution environment what that actually is is that the PSP acts as a black box inside
05:07
your system that is trusted by an external entity for example a content provider like Netflix this would enable for example digital right management on an untrusted system that is your system like Linux so to
05:26
sum this all up the PSP runs code that you don't know and that you don't control and first of all let's talk about the knowing what you see here is a super micro motherboard a server motherboard from the top and I
05:46
highlighted three components here which are required or essential for for boot up of course that is the CPU the the disk and so-called SPI flash the SPI flash is a simple storage that is available during early boots so if
06:04
you look at the boot procedure in a simplified manner then the CPU will first load the BIOS from this SPI flash and only at a later stage of booting when the necessary drivers are at hand it will be able to access the hard disk to load the operating system now as we saw from from AMD's
06:27
marketing slides this there's the PSP now and the PSP is actually part of the CPU it even boots before the CPU boots and will only after successful initialization of the system release the x86 CPU so the PSP firmware is
06:46
loaded loaded first and after that the boot is proceeding as we know it with the BIOS and the operating system so where's this PSP firmware coming from well the BIOS is stored on the just just mentioned SPI flash memory and it
07:07
contains all the data and code that is used of course during during boot up and it is arranged according to the UEFI image specification so it's a standardized format that's that's good so maybe we should have a look into a
07:21
super micro UEFI update you see a screenshot from the open source tool UEFI tool which is able to parse the UEFI image specification you see information for example like the full size this is 16 megabytes that's
07:42
the traditional that's the size of a traditional SPI flash and you see several volumes which contain BIOS code and data what you can also spot are two so-called paddings non empty paddings and these are paddings because or these
08:00
are called padding by the tool because they are not part of the UEFI standard and we're not able to parse them with the standardized information available so let's use another tool probably many of you know binwalk command-line tool for extracting firmware from from images and for
08:23
forensics in general and let's look at the machine instructions we can find in that UEFI update for the super micro board so the second block you see our Intel x86 instructions this is what we expect right it's it's a BIOS update for an x86 CPU so that's that's that's not surprising what is more
08:45
surprising is are the ARM instructions so we might be very close to the PSP firmware and what we found out by staring at bytes and hex editor a lot
09:02
is what we call the firmware file system of the platform security processor and the central data structure in it is the directory and directories starts with a magic string in this case dollar PSP and it will have a checksum it will have a number of elements that it will list and a field
09:25
we don't know and then with each line in the screenshot you will have an entry in this directory and each entry has a type and a size and an address where it is located inside that UEFI image so the last entry of this
09:45
directory is a special entry it points to a secondary directory or that's how we call it it's a continuation of this directory and each entry points to something like a file file definitely has a buddy and it might
10:03
have an header and a signature but I'm gonna go into detail about this in just a second so now we just need a reliable entry point to pass this whole firmware file system and this is the firmware entry table the firmware entry table begins with a specific byte sequence that's how you can find it
10:21
and it lists pointers to firmware blobs such as those directories inside the UEFI image earlier versions of the firmware entry table are documented in source code of the core boot project and open source BIOS implementation and that was very helpful in the beginning of our research so to make
10:42
use of all that knowledge and all that staring at bytes here we develop PSP tool the command line utility that is able to pass any any AMD firmware from UEFI updates such as the super micro update and in the output you will see
11:03
something like a directory header here you will find entries like something called PSP firmware bootloader you will see that it has a version and PSP tool will even try to find out whether it's compressed signed will try to verify the signature and so on and just as a as a as a recap here you
11:25
can see that the last entry of this directory actually points to another directory which PSP tool parses for you as well so in order to enable you to look into the code that is running on your AMD CPU right now PSP tool is available on github and you can check it out today so the PSP runs code we
11:48
don't know well now it's a matter of binary analysis to actually find out what it does let's talk about the control are we able to alter the firmware to run
12:00
our own code for that we had to play around with hardware and more specifically we used an SPI programmer to flash any arbitrary uefa image onto the SPI flash after for example taking the original uefa image and
12:24
tinkering around with one byte or one bit we would then try to boot the system and in most cases cases it just wouldn't boot this was this was insufficient because we only had binary and binary output from these
12:41
experiments so we also used the logic analyzer that you can see on the top of this of this picture a logic analyzer is just an electronic instrument that can capture the data that runs through the logic lines in this case between the SPI flash and the super micro board so looking into a recording of one
13:08
of our boot procedures we would now be able to make sense of these of these data so for example we can see that the chipset here issues a read command
13:21
that's defined by the byte 3 it would try to read the address e2 0 0 0 0 and then the SPI flash would gladly respond with data at that location now you might argue the data is not that interesting because that's what we control that's what we can program that's what we can look into with
13:41
the PSP tool so what we were more curious about is the order and timing of the actual accesses and to make that a bit more visual we wrote PSP trace so PSP trace takes such a SPI capture and correlates it to the output from PSP
14:03
tool and we will get a enumeration of the specific components of the PSP during boot and I'll get into detail about this also in just a second PSP trace is available as part of the PSP tool repository if you are more
14:24
interested about our hardware in our hardware setup you can check out our talk from the CC camp earlier this year where we actually had a ryzen pro a CPU at hand and yeah just use the Lenovo ThinkPad so that might be more
14:43
suitable for your homework so I want to share two more insights that we gained through our experiments in the beginning first of all cryptographic protections on files files are protected by signature and a field in
15:03
the header and determines the according public key that can be used to verify that signature and and that's what the PSP does so there are several keys actually inside the firmware file system and then all these keys are signed by the AMD root public key which does not have a trailing signature
15:24
but as we found out it after loading after after it is loaded from flash it will be compared to a hash in read-only memory of the PSP so we were not able to alter it like that the second insight is how the early boot
15:43
procedure of the PSP works we have an on-chip bootloader that is burned into the chip into the PSP we have an off-chip bootloader that is loaded from flash and then we have several applications that are loaded subsequently so now let's look a bit more closely at the output of PSP trace
16:04
the first few read accesses are to the firmware entry table the global data structure and then the the on-chip bootloader will load the PSP directory it will load the AMD public key and verify it as I just told you by
16:21
comparing it to a to a hash in and read-only memory it will load the PSP firmware bootloader that's what we call the off-chip bootloader and this one will be verified with the AMD public key then in the boot trace of PSP trace we see a delay that's due to some initialization work the PSP does and
16:45
then it will load more directories and will load and verify some applications eventually and with this rough overview of the boot procedure I'm gonna hand you over to Alex okay so now that we uncovered the basic modules of
17:10
the firmware we obviously wanted to gain deeper knowledge about what these individual modules do how the firmware functions of the PSP is constructed what hardware it provides and how we can interface it so in order to do that
17:23
we need to do a quick recap about how AMD structures its CPU itself so what you see here is a little x86 core being able to execute two threads using simultaneous multi-threading and AMD groups four of those cores into what they call a core complex it contains up to four cores based on your
17:43
exact model and two of those complexes are put onto a CCD or core complex die that is what AMD also calls a chip letter so it's a single silicon chip on your CPU and you have multiple of those chips on your on your CPU among the two CC axis it contains to the memory controller for
18:03
the DDR for memory PCI Express lanes communication links for to communicate with other CPUs in a system and much more so in our setup you saw earlier already we had the two socket system with two CPUs and each of these CPUs had four CCDs and now we have just not just one PSP in this whole
18:25
system but up to eight so each of these CPUs or if each of these little PSPs is actually executing code even before the x86 cores have executed anything so AMD calls the one on CCD 0 the master PSP and all the others are
18:44
referred to as slaves the master coordinates the initial bring up of the platform so for the whole initial initialization for the memory controllers and so on and the slaves respond to requests made by the master PSP so each of these PSPs is identical in the system because they are 32-bit arm
19:05
course they have a 32-bit address space layout the first 256 K of this layout are backed by actual on-chip SRAM those the first the on-chip boot loader will load the off-chip bootloader so the PSP wfv bootloader and place it into memory where it will be executed among the actually firmware
19:26
bootloader you will also have the page tables for the MMU yes the PSP also has a MMU and virtual memory enabled and the code is separated into a supervisor or kernel mode and the user mode part so the last page you see here is the
19:42
so-called boot ROM service page it contains information about the PSP the code is currently executing on like number of sockets in the system the current CCD ID where it's executed it contains some some other things like a number of sockets and so on and it will become important later on then the
20:02
off-chip bootloader will call the applications in they are executed in user mode they contain the code and data to bring up the actual system and they also contain the stack memory and this is done on during the initial boot up process by using a fixed order and later on when the host OS runs it
20:23
will be loaded the application for example for the SEV functionality will be loaded on demand so the rest of the space there we have to fill is taken up by MMIO so this PSP has its own cryptographic code processor which is not shared with the x86 you have the hardware registers to access is x86
20:41
memory to access the system management network what this is we will come to in a bit and much more we don't know about now right now so the boot process in detail so Christian already to have gave you a rough overview how the boot process is done and now we will take a deeper look into this so first of course you have the on-chip bootloader it loads the off-chip
21:02
bootloader from flash and executes it the off-chip bootloader will execute and initialize the PSP to a bare minimum and then call the apps the first one we have here debug unlock and security gasket we have no idea what they actually for but we named them after some strings we found in the binaries itself so the big chunk you see here is the actual bootstrapping
21:22
phase AMD calls it a geezer bootloader and it's not just a single binary but it hosts a binary which loads binaries from the flash furthermore and then execute it in a specific order so you see a bl2 one two three four and six a bl5 is used for something like a warm resume from
21:41
suspend to run for example so later on if the SEV app is for example loaded if the week OS requests a specific SEV functionality and not before that because we have the separation between supervisor and user mode we obviously need a way that the app can communicate with the option of chip
22:00
bootloader and that is done using the ARM instructions supervisor call or SVC so we identified 76 this calls in total we have mostly reverse engineer 30 by now we can access the x86 memory we can communicate with other PSPs in a system we can load entries from flash and so on 28 are partly reverse
22:21
engineer there's a mostly CCP operations for airs up public key verification AES encryption and so on and there are also more elaborate functions to communicate with other PSPs which are required during the a geezer bootloader stage and then we have 18 left and these we don't know about yet because they are not cold at all or they have exactly one call site and a
22:42
non-trivial to reverse engineer so system management network I already saw on the slide already that there was excess SMN if you google for system management network or SMN you will don't you won't find much information about it by AMD or otherwise the only reference you may find is code in the Linux corner to read out the thermal sensors on the CPU so the
23:05
system management network actually is a hidden control network inside your CPU each and every hardware block which is in there is connected to it and is used for the PSP to control and initialize the hardware blocks during the boot up phase so it is a dedicated address space with the PSP
23:21
can't directly access it using mmm I owe instructions and we have the PSP there we have identified the memory controller the system management unit for which there was a talk about I think two years ago on this very Congress these 86 course are there as well and a lot of other things we didn't reverse engineer so far one of the thing okay so to access the system
23:44
management network the PSP has to map a certain region of the system management network address space into its own address space and then can access the register right read and so on and has to unmap it again and one of the functions we identified is what we call memory protection slots so the
24:00
PSP has the possibility to in for two to configure the memory controller to revoke access to certain regions of the DDR4 memory from the x86 course this is done by using free registers we have a start register with a physical start address an end register to denote the physical end address of the region you want to protect and the controller just that have you only know yet so
24:22
far the enable bit to flip it on off and what it does is if the protection is flipped on the x86 will only read all the debt when it tries to access this particular region and rights will have no effect report to this region as well and this is for example used for the system management mode you if I code and for certain functionality for the secure encrypted
24:43
virtualization feature of AMD so the next thing we did was running strings over all modules obviously and what we found there were a lot of interesting debug strings and even a lot of format strings and we wanted to know what the values were during the runtime so when we disassembled the firmware and analyze
25:04
that we saw that most of these strings were reference right before a special call called SVC 6 so this must be some sort of debug debug print for the PSP the problem is SVC 6 is not implemented in the release firmware so we had to find another way to gain access to these debug strings
25:21
and this is what I will talk about now so the problem here is first we need to know where we want to store this debug strings and we don't have any x86 memory available at this time in the process so we need to find another device or buffer where we can store it for later use but the only
25:42
device we didn't know about at this time was the SPI flash luckily for us writing to this SPI flash area from the PSP generated the necessary bus cycles on the SPI bus without altering the flash then we need a code execution on the PSP to inject our own SVC handler and how we gained execute
26:02
execution will Robert will talk about in the third part of this talk but for now we assume that we have code execution on the PSP already can inject our own SVC 6 handler and then leave let it run so the app will call SVC 6 it will be forwarded onto the SPI bus where we can collect it with our already existing setup use a tool to filter the debug strings from the rest
26:25
of the traffic on the SPI bus we don't want to have we don't want to have in the debug output and then hopefully get a raw PSP lock and we had success with that so what you see here is the initial boot up was the very first
26:41
stage of the boot up stated the locks are several magic megabytes long and we didn't have the chance to go through all of them so there's a lot of interesting stuff hiding there already so so the next step was to explore
27:05
what is hidden inside the system management network and we didn't want to always refresh the whole system all the time and write code for a debug it because that is error prone and tedious so we created our own setup where we could dynamically use the x86 cores on the system to write and read
27:23
from the system management network for that we replaced the SEV app with the stop and the stop provides three primitives that you can read write a system management network address we can execute an arbitrary this call from the off-chip bootloader and we can read write general PSP memory and because the PSP is exposed as a separate PCIe device to the x86 we use the
27:46
existing Linux kernel driver and modified it to expose these requests to user land where we created a user space library wrapper and some Python bindings and with that we were able to use a Python shell to dynamically read write registers headers spurious reboot in between with if you did the wrong
28:02
thing and but could start over very quickly so what you see here and the code snippet is what we did to discover what these memory protection slots were about you can see that we call in a space and a syscall handler that we write some system management network address and so on and we do it for all the different PSPs in the system so the masterpiece P can also
28:22
forward these requests to all of the other piece piece in the in the whole system next thing you wanted to also analyze the SEV app further and see what the code or how the code is executed and how the data flows in this SEV app but because we already had our PSP stop running there and
28:41
couldn't share it on the PSP we had to find another method and we created a PSP emulator for that and using our lip PSP proxy to forward requests onto the PSP so the current state can run the SEV app up to a certain point and we are still actively developing that so that started a few
29:01
weeks ago and that's this will continue in the development so what it does is what you see here is the AMD SEV to manage the host and configure all the keys and certificates on the system and we modified the Linux corner driver to reroute these requests out to our own PSP emulator running in user space which is based on the unicorn engine
29:22
any hardware access because we don't know much about the hardware yet is forwarded to the real PSP results are collected and when the SEV app finishes it will return the result back to the AMD SEV tool and with that we are able to execute some of the requests the SEV app implements successfully so far
29:43
yeah what you see here is a small snippet from one of the traces you can see a Syscom being made it's a CCP request we don't know exactly how the arguments are used by now that's why there's a lot of unknown stuff but this will aid us in development and furthermore in addition to allowing it tracing code execution observe the data flow we later on may be able to
30:04
provide functionality which is currently only available on the epic server platform from AMD like secure encrypted virtual machine the problem here is we don't know yet if all the hardware is there which is supported and whether it's only as firmware limitation by AMD if you're interested the code is
30:23
here on the repository it will be made available in the next few days we have a number of repositories available you already saw PSP tools we have some repository we collect documentation about hardware interfaces this calls and so on we have our PSP emulator there and also the PSP apps repository if you
30:41
want to dive into writing your own apps for the PSP and with that I will hand over to Robert who will talk about how we gain code execution on the PSP itself okay so for everything that Alex talked about we need code
31:03
execution on the mic better all right so this part of owning the PSPs again split into two parts no Christian already talked about the firmware and
31:24
the SPI flash so this is something we can control because we have physical access to the device we can flash everything we want so what can we do with that so on the SPI flash we have these directories which have a header and entries and an entry is actually compromised of an ID and
31:43
address and the size we've talked about files so an entry could be a reference to a file and we also talked about these secondary directories so an entry could refer to another directory now if you look at the files you see that they have a signature usually so we cannot manipulate those files
32:03
directly if we touch them this will be noticed and they won't be loaded and the system will immediately reboot now what we can manipulate is the directories themselves because they are not protected at all so specifically what we can do is we can for example add additional entries these entries
32:23
might point to the same files that doesn't matter we can add entries what we also can do is we can remove some of those entries or we can change entries so for example this reference to the secondary directory this has the size parameter right and this size refers to the size of that directory
32:42
and actually what we can do is we can change that size so we can make the directory appear to be smaller without removing any of those entries now during boot this PSP directory that Christian already talked about is parsed so this PSP directory contains among other things the reference to the AMD
33:04
public key which is used to authenticate all the applications which are loaded now this directory also has a secondary directory the content is not really relevant here so the on-chip bootloader that executes first will set up this bootrom service page that Alex talked about and this bootrom
33:23
service page contains a copy of those directory entries just for the first directory and also the on-chip bootloader will copy the AMD public key itself to the bootrom service page so it only copies the AMD public key if it's been verified before okay so now this bootrom service page
33:45
contains this AMD public key and this public key in memory is from then on used to authenticate applications so the off-chip bootloader which executes later will use that bootrom service page and will extend it specifically it
34:02
will copy the entries of the secondary directory to that bootrom service page so I guess you can already see where this is going so what could possibly go wrong here well we have space for 64 entries here and if we write more
34:21
entries to that page we'll hit the AMD public key so the off-chip bootloader should better check that we only copy at most 64 entries there it is there's a check let's say this is the function that appends entries and it says okay if the number of entries exceeds 64 we return an error code and
34:42
do not copy sounds good thing is that number refers to the number of entries in the secondary directory so this has a maximum size of 64 but there is already space there entries there on this bootrom service page so actually what we enforce with this check is whatever we append can have at most 64
35:05
entries and within that 64 entries well there's the AMD public key super convenient so what we do now we place our own public key inside the directory structures of the firmware file system the off-chip bootloader copies the
35:21
entries and copies the AMD public key so what does it mean for us now all this parsing happens before the first application is loaded so that means we
35:41
control the very first application and can replace the content and from there on we control the user land part of the secure processor so now coming to the next part so the natural next target is of course I mean we have user lens code execution we want to have the rest kernel mode so how we can we
36:03
take over the kernel mode now let's have a look at how this distinction between kernel and user mode happens so if we look at the virtual memory layout we'll see that there's a user mode part and a fixed split with the kernel mode where our off-chip bootloader resides so our application
36:23
which we already control can try to access that memory of course but that won't work right the MMU will prevent any access to privileged memory ok so let's see how this works at runtime so this bootloader component if we specify the privileged memory a little bit more we have code and data
36:44
there and at runtime another type of directory is parsed and this is called the BIOS directory I mean it's a similar structure as the directory before we have entries and the reference to a secondary directory the entries here again of no relevance so during boot the off-chip bootloader
37:03
will copy those entries into its data section ok so for the copy operation we need some some information so let's say this is the copy operation kind of looks like memcpy what we need is destination where to copy we need source this is the secondary directory this is the thing we want to copy which is
37:24
already under our control so convenient we control whatever is whatever data is copied and we need the size value so where do we get that size oh yeah this this entry here has a size value super it's ours also right we control the directory structures we can manipulate the size so to sum up we
37:45
have a copy operation into privileged memory with attacker control data and attacker controlled size this isn't very old meme and I think it's appropriate because this this bug is so easy to prevent actually but for us it's good
38:02
because now we control everything in red here ok so we control that control that part the thing is as you can see code is not part of what we control so what might be here what is of interest for us to overwrite thing is
38:23
it's the page tables right the page tables are part of that data section within the privileged part of the virtual memory space so again what we do we place our own page tables here the data is copied and replaces the page tables in memory of the secure processor so now if we look at that
38:42
virtual memory overview again well our page tables will define the virtual memory a bit different we make it everything user writeable so we control the application our application now can touch the privileged memory and just overwrite everything there if we want to write for that we need to
39:02
have to reimplement everything but we can patch now the secure operating system if you want so that means this parsing of the directory also happens
39:21
before the first application so we control the first application that takes over the bootloader if you want and from there on we have everything all those issues I presented were fixed were even fixed before we discovered them right so we might not be the first one that discovered them there some of you remembered that there was some website called AMD flaws they
39:45
did not present too many technical details maybe what they discovered was something I present here I don't know thing is it does not really matter for us because the secure processor does not implement any rollback prevention so we can always go back and reflash a vulnerable firmware and from that use
40:04
whatever code you want to place there so what what we did is we used all this on an epic Naples based server system and you cannot just use that issue on every AMD system because the bootloader we are using was signed
40:24
with an key specific for the epic Naples CPU series however we believe we have not tested it fully yet but we believe the same kind of issues exist in bootloaders which are signed with an rise in first generation key
40:41
and for the rest we don't know yet so maybe for Threadripper or epic Rome there are similar issues maybe not we don't know so the question is is this really an security issue I mean of course it's a security issue but for whom so everything we did requires physical access to the device so if it
41:05
were my laptop personally I wouldn't be concerned too much however there are some things where this is a real issue for example if you rely on secure boot because the circuit processor is the first part of what's up and if
41:21
that is broken everything later on is also broken so a Christian already told you that AMD plans to use the secure processor trust execution environment if your application relies on that you better not have any security issues in that secure processor and for the last part the secure encrypted
41:41
visualization technology from AMD is dependent on the integrity of the secure processor if that is broken this technology is also broken so Christian and I published a paper of that about that if you're interested you can read it up but for us here this is actually more of an opportunity right because we
42:02
can gain more insight into this PSP with code execution we can do a lot of cool things with that so it allows to do further research and other subsystems which are present in the AMD CPUs for example the PSP is responsible to load the SMU firmware the PSP allows access to the SMM mode so this is a
42:24
ring minus two mode on the x86 CPUs so higher privileged and your kernel and there is proprietary code running in that mode with the PSP you have access to that code and could replace us analyze it whatever and the PSP is
42:40
responsible to kick off the x86 cause at all so everything that comes later is in theory now under our control thank you that's it yes thank
43:01
you very much Robert Alexander and Christian that was fantastic Wow I have a lot of questions I guess in my head going on but do we have any questions from the audience and if you have any questions we have microphones lined up here a question is just so that you know what we're talking about with questions is a sentence with a question mark behind it and not your
43:22
life story and I think I saw number one first so let's start with number one hey is there a reason why the page table is located at the end of the data segment I don't think so I mean just because you have to place it somewhere
43:43
should be in the why not in the beginning I don't have no idea that's what I meant with a lot of weird questions here from the signal angel we had one question this question goes to the first lecturer didn't you have access to an SBI flash emulator to end and time of use versus time of check
44:04
attack so we had access to different tools but the talk to attack that you mentioned was not even was not even necessary to to mount the attacks we talked about and actually so far we don't see any possibility to mount talk
44:25
to attack okay so I think I saw microphone five next up is there so many microphone yes yes so as can wondering if you consider looking at the boot ROM for issues yes of course the thing is we cannot find its code in
44:48
the memory anymore after we mounted our attacks so I believe the boot ROM code is not there which would make it more not there anymore which would make it much easier to analyze we tried simple things like increasing directory sizes
45:04
which are passed by the boot ROM itself we haven't found any suspicious thing there yet microphone 2 thanks for your research you have really nice big power over the system right now they have plans to make an PSP firmware
45:27
which is minimal and which makes your system work but without some strange untrusted code I wouldn't call it plans yet of course there are ideas to
45:44
do that the thing is some of the functionality which is implemented from AMD is really required so this the stages that Alex talked about they configure your D or train your DRAM so without those stages you don't have access to memory your x86 course wouldn't not work and to re-implement
46:04
that without having access to any manuals is really really hard work so I'm not too confident that this will be possible in the near future I just refer to management engine cleaner there is such a project which makes your
46:21
management engine firmware slim so the AMD firmware is already kind of slim the only thing that is not strictly required on the systems we have been looking at would be the SEV firmware which is loaded on request and you can like disable that by just flipping a bit inside that file that would the
46:44
system would still boot but when it tries to initialize the SEV technology the corner would say okay this does not work the system will still work after that thanks and last little question does PSP work with microcode somehow we
47:01
didn't find anything related to any microcode there so far thanks so let's move on to microphone three thank you first for the great talk I have one question do you have maybe found something evil or potentially evil in the code that it does no so far we didn't find anything which could be
47:25
used as a foreign attack for example so what the PSP might be able to do is access PCIe devices we found some code related to that but we are we don't we are not we are not sure yet whether it's actually used because also the PSP is
47:42
executed was existing on graphics cards made by AMD so that might be also ready to that we couldn't find anything there yet but so far the PSP looks rather clean compared to the inter management engine thank you so we have a question from the internet is the AMD public key an RSA one only 576
48:05
bit it's an RSA key yes but it's 2048 bits for the first generation epic CPUs and I think of 4069 for later generations microphone to for me it
48:23
seems like preventing to flash old vulnerable firmware is really important for a scenario like secure encrypted virtualization can you comment on how difficult it is for AMD to at this retrospectively okay so technically
48:44
rollback prevention is there for I guess mobile devices for example you have that should be possible for adding this functionality afterwards I don't think that's really possible because the on-chip bootloader is the thing that loads the off-chip bootloader and verifies it and that software
49:03
component has to like stop loading if the firmware version does not match for example and you have to change that and that functionality is not there and you cannot update the on-chip bootrom so in that sense I don't think that's that's possible to change and if you look at our paper you will see that
49:23
the firmware issues are kind of devastating for the SUV technology because there are some keys which are now accessible which can be used for attacking a severe protected guests thanks microphone three please one
49:43
question did you analyze the API to the x86 core and did you find anything that could be exploited without flashing anything so that you could directly go from x86 to PSP exploitation yeah we try to find the
50:04
necessary code to interface with the x86 we think we found one place where the x86 cores are released after the PSP initialized the whole system but obviously we can't do much with it except preventing the x86 to boot at all and otherwise we couldn't find anything there yet so we focused on on a lot of
50:24
other like the memory controller and didn't had a deeper look at the into the x86 interface so what there is there is the BIOS can interface with a PSP using a special mailbox register which is mapped in mmio space and x86 for
50:40
our request so if it can for example the UAFI when it boots it will say to the PSP hey this is my system management mode code region please protect that for me and it will execute this request but apart from that we couldn't find anything so far thank you so microphone for hi so is it correct that your work enables a hundred percent open source firmware for this kind of
51:07
processes and if so have you already contacted core boot team to make that actually happen so well hundred percent open source as for the PSP there is
51:21
on-chip boot rom which we can't replace right so this will be closed source then there is code of the off chip bootloader until the first exploit which runs which is not open source in theory you could from now on take over the PSP write your own coach but as I said before you had have
51:41
to reimplement reimplement a lot of functionality without having any documentation right so technically it's possible I guess to do something like that practically hmm I'm not too sure so we're gonna go to the internet for another question is it possible to block PSP from within Linux or BSD for
52:02
the system's runtime by using certain boot flags sorry to block what to block the PSP from the Linux or BSD so what you can do is like a robot mentioned already you can flip the bit in the SPI flash and then the PSP once
52:23
it initialize the whole system it will it won't run the SUV app for example because the signatures won't match anymore and there's no other sort of interface where the PSP is actually triggered we are we couldn't find it so far microphone three okay all right right
52:48
enable any superpowers from PSP like J tag or special tricks with voltage or something else I mean the first application that is loaded has some strings in it like debug unlock sounds interesting but then again jtech
53:07
where would you access to jtech of the PSP you need to have some some connection to the lines right until supports USB debugging yeah I know with special devices right no even where cable okay so anyhow I have the
53:24
suspicion that this debug unlock app is responsible to to allow some debug mode which then I assume with special hardware you can do have jtech but we have not touched it yet now microphone 3 so I miss far from a liar
53:43
lawyer as possible but could AMD in any way file a cease-and-desist for anything you do probably not I guess but just curious I have no idea thank
54:01
you and as I said before we're not the ones that initially discover probably not the ones that initially discovered these issues and it's not really about these issues I mean for me personally these issues are a nice way to get more insight into the PSP and it's not about having the super new
54:20
security issue whatever so if AMD wants to file something I guess they would have also filed other people that did similar research before maybe they did I don't know so we had another question from the internet how long did it take you to reverse engineer and develop all the stuff so I
54:46
think beginning of 2018 Christian was starting with his master thesis and we spent a lot of time on figuring out how this firmware file system works and the boot process and writing this PSP trace and PSP tool to better
55:04
understand the components of the firmware and Alex joined in May Mayish this year and well we're still working on it right so the emulator once we figured out a lot of information of the PSP I think the
55:22
emulator was easy to to develop so in the sense that it didn't take too much time but of course there was a lot of work going into it before that so I do not see always I do see another question from the internet question all
55:41
right did you try to glitch the PSP by manipulating the voltage of the sock uses why I think our approach is easier but no seriously we did not try so with that I don't see any further questions and I would like you to help
56:03
me thank Robert Alexander and Christian for this fantastic talk