A debugger from scratch
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 50 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/43135 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
All Systems Go! 20184 / 50
1
4
11
12
13
14
16
17
19
23
24
25
29
30
32
34
35
39
40
41
43
44
50
00:00
SpacetimeSystem programmingDebuggerPhysical systemProcess (computing)Control flowRead-only memoryError messageComputer configurationSource codeAddressing modeBefehlsprozessorMachine codeFunction (mathematics)outputComputer virusAddress spaceEmailEntire functionLink (knot theory)InformationFlagSheaf (mathematics)Price indexString (computer science)Data typeConvex hullAreaInformation securityCompilerInformationData structureDebuggerComputer programmingSymbol tableFunctional (mathematics)Address spaceTracing (software)Semiconductor memoryBitBefehlsprozessorMachine codeComputer filePower (physics)MappingEnterprise architectureFile formatDirectory serviceRight angleGoodness of fitSheaf (mathematics)Table (information)EmailPoint (geometry)Correspondence (mathematics)NumberSource codeHexagonLine (geometry)Formal languageWeb pageSingle-precision floating-point formatCommodore VIC-20Structural loadProcess (computing)Set (mathematics)Machine codeSystem callLibrary (computing)Programmer (hardware)Direction (geometry)Letterpress printingVirtual machineComputer virusLink (knot theory)Variable (mathematics)Error messageAxiom of choiceComputer configurationSoftware bugDressing (medical)Domain nameHecke operatorCommutatorPhysical systemLie groupMoment (mathematics)Speech synthesisNetwork topologyOntologyPetri netLevel (video gaming)WebsiteMultiplication signComputer animation
09:52
System programmingDebuggerComputer virusComputer programmingSemiconductor memorySource codeAddress spaceUniform resource locatorFunction (mathematics)Tracing (software)OctahedronControl flowPoint (geometry)Process (computing)Functional (mathematics)Multiplication signLine (geometry)Virtual machineData structureInformation privacyWebsiteStudent's t-testAxiom of choiceAreaHecke operatorWordMoment (mathematics)TrailQuicksortAnalytic continuationAttribute grammarBytecodeHexagonPOKEDebuggerMappingError messageMachine codeRight angleSource codeComputer animation
15:17
System programmingDebuggerComputer virusExecution unitDrum memoryFunction (mathematics)Stack (abstract data type)BefehlsprozessorSystem callParameter (computer programming)Computer programPointer (computer programming)Address spaceLocal ringVariable (mathematics)Table (information)Dean numberDew pointWebsiteComputer programmingConstructor (object-oriented programming)Ocean currentBitState of matterMultilaterationPoint (geometry)Frame problemArithmetic meanComputer clusterRight angleControl flowMereologyAddress spaceFunction (mathematics)Functional (mathematics)Factory (trading post)Table (information)CASE <Informatik>Pointer (computer programming)Ferry CorstenContent (media)Interrupt <Informatik>Source codeInformationProcess (computing)SpacetimeChainParameter (computer programming)QuicksortHierarchyVariable (mathematics)Computer-assisted translationDampingSemiconductor memory1 (number)Web pageData structure3 (number)Virtual machineForm (programming)BefehlsprozessorLine (geometry)DebuggerStack (abstract data type)Graphics tabletComputer fileNumberRippingSymbol tableComputer animation
20:42
System programmingFunction (mathematics)Network topologyCNNInterior (topology)Right angleMultiplication signNumberQuicksortLine (geometry)Stack (abstract data type)Variable (mathematics)NP-hardMachine codeFunctional (mathematics)Computer programmingComputer clusterPoint (geometry)Source code
22:30
System programmingExecution unitDebuggerHexagonDifferent (Kate Ryan album)Line (geometry)Right angleNumberProcess (computing)Point (geometry)BitCASE <Informatik>Computer programmingSemiconductor memoryLocal ringVariable (mathematics)Complete metric space7 (number)LeakFamilySatelliteHecke operatorProgram slicingControl flowWebsiteComputer animation
25:45
Process (computing)Physical systemControl flowRead-only memorySource codeAddressing modeMachine codeSystem programmingDew pointFunction (mathematics)Electronic meeting systemRange (statistics)Point (geometry)Line (geometry)Functional (mathematics)HexagonSingle-precision floating-point formatHypothesisMachine codeSemiconductor memory7 (number)Letterpress printingBitVirtual machineSource codeXML
28:07
System programmingDew pointMaxima and minimaDebuggerComplete metric spaceInformation securityWriting7 (number)Inheritance (object-oriented programming)Line (geometry)MathematicsFunction (mathematics)Process (computing)Power (physics)Set (mathematics)Right angleSource codeComputer animation
29:49
Source codeAddressing modeMachine codeSystem programmingDebuggerBlogProfil (magazine)Compilation albumBitRight angleMachine codeDebuggerInformation securityType theoryDampingMultiplication signDefault (computer science)Link (knot theory)Set (mathematics)Scaling (geometry)Computer animation
Transcript: English(auto-generated)
00:06
All right, good morning. Everybody can see everything, and you can hear me okay? Good. Right. So, my name's Liz Rice. I work for a company called Aqua Security. We help enterprises secure their containerized deployments. So,
00:23
you might think it's quite a long leap from thinking about container security to thinking about debuggers. So, this all came about because I did a talk about how to build containers, and you use system calls for that. And I felt that
00:42
I needed to understand a bit more about system calls. So, I did a talk about what system calls are and how you can trace them using ptrace. And I came across this little paragraph from the man page and it says, ptrace is this very powerful system call that you can use to observe and control the
01:04
execution of another process. And I used it in this other talk for system call tracing. But you can also use it for breakpoint debugging. And I thought, yeah, I should try that. I should figure out how to use that to build a debugger. So, that is what we're going to do this
01:20
morning. We're going to build a debugger. So, before we can do that, let's just talk a little bit about what's happening when we run an executable. Oh, I should talk a little bit about ptrace in the Go code first. So, I'm a Go programmer. Anybody else writing Go here? Not that many
01:41
hands. It's okay. You people will be my peer reviewers when things go wrong. For everybody else, I'm sure you'll be able to kind of follow along. Ptrace, the syscall library in Go gives us a whole set of functions. They map down to this single ptrace system call. But actually, there's a
02:02
whole load of subcommands. And this gives us a pretty good idea of the kind of subcommand you get from ptrace. So, for example, we can see things like getting registers. We can see setting registers. We can see things like single step,
02:22
step from one instruction to the next. And if you're old enough like me to remember things like Commodore 64, I remember peeking and poking data. I know that that is to do with writing data and reading it from memory. So that gives me a pretty good idea of the kind of things I can do with
02:42
ptrace. So, before we start looking at writing a debugger, let's just talk a little bit about how kind of executables work. I'm sure lots of you know this. But make sure we're all on the same page. So, when we compile our source code,
03:00
go is one of those languages that is compiled. Every line in the source code maps to one or more machine code instructions. And when we run the program, we have a CPU register called the program counter that's pointing at the next instruction we're going to execute. So, as we go through
03:22
the code, the program counter gets incremented to step through the code. Now, suppose we want to break point in our debugger. How do we do that? We do that by overwriting where the instruction is where we want to stop. We write this
03:43
code, hex CC. And that tells the CPU, as it's stepping through the code, if it hits that byte code, trigger an interrupt and stop execution. Okay. I think that gives us enough to start building our debugger. Oh, one last thing,
04:04
though. We need to know where in memory to set that break point. Because as humans, we look at the source code, we want to be able to stop at a particular line in the source code and we need to know where that maps to in the machine code in memory. So, we need a way of mapping
04:21
between addresses in machine code and their corresponding source file and line number. Right. So, this is not the best Go code you're ever going to see written. I have a lot of global variables just for convenience. So, forgive me for
04:41
that. This is going to be my debugger and I also have a little executable called hello, which we'll look at in a minute. So, that's going to be my target that I'm going to debug. Now, Go gives me a way of extracting information about that mapping between machine
05:05
code, instructions and the lines in a symbol table. How we actually go about extracting that is not terribly interesting. So, I have written myself a little convenience function to do that. The code for this is on GitHub and I'll
05:22
give you the link later so you can see how I did that. It's not super interesting, the detail of it. But we could just go into my let's check where it is. Yeah. So, my hello target is in a directory called hello. And I
05:43
could use a little tool. I hope I've got read elf on here. Yes, I do. So, read elf. Elf stands for something like executable format. Let's look at. So, this is telling us
06:00
something about that executable file. We can see that it is an executable file. We get some program and section headers. And the one I'm particularly interested in is this one here called go PC line table. So, that's written
06:21
by the go compiler into my executable. And that's what my little program, my little function get symbol table reads out. We don't need to go into the details of what that is. But that information is built into the executable. Okay. So, having got that symbol table, I can do
06:40
interesting things with it. Like, I can look up a function. Now, all go executables have a main.main. So, I'm pretty confident I can find one of those. And I'm going to get back a function structure describing that function. And
07:02
I can print out some information about that. So, the function name starts at a particular address. So, we get the function name and we can get entry which is the address in memory of the first machine code instruction in that
07:23
function. We can also do mapping between program counter. Remember that CPU register is called program counter? We can get the corresponding line in the source
07:42
code from this symbol table. So, if I start with that first address of that function that we just extracted, I can get back a file, a line and a function structure. And I could print that out. So, function whatever. At line,
08:08
whatever. In file, whatever. So, that's going to be function name at the line at the file.
08:23
And finally, we also have the opportunity to go in the other direction. So, we can go from line to program counter. So, we can take a file and a line number. And we get back a program counter. What else do we get? Let's check.
08:43
Yeah. We get back a program counter, a function and an error that I'm going to ignore. And we'll print that out as well. Now, let's choose a line that we're going to look up. Here's my source code. I'm going to
09:04
pick line 22. So, let's go back to my debugger and say, okay, let's find out what function we're at if we were at line 22 in that source code file. So, let's try that.
09:20
We'll run this. So, we were able to see that the machine code address of the first line of the first instruction in main.main is whatever. But then we could map from that to see that main starts at line 5. So, let's just check that that's true. There it is. There's
09:43
line 5 and main starts there. Great. And we also saw that if we were at line 22, we would be inside a function called F3. And that is also true. Here's our function F3. So, we've got that mapping between the machine
10:01
code instruction and the location in source. So, now, I'm going to get my debugger to run this executable. So, I can do that with
10:22
we build up a structure. I need to say that I want to map stood in, stood out and stood out from the OS stood in, stood out, stood out, because otherwise we can't see what's going on. So, this is where I get to do
10:42
multi-cursor, stood out. Okay. And then I can start that executable running. So, I'm
11:03
setting up a structure that describes the thing that I want to run. And then when I call this start function, that actually creates a process for running my target executable. And then I'm going to wait for something to happen. And if that gives me an error of any sort, I
11:25
will print out what it gives me. So, wait returned something. Okay. And I'm also just going to put in a little line here to say that the debugger finished
11:40
when it completes. Right. I'm going to just comment out the output we put in before. Because it might just be kind of getting in the way. So, now, by running my debugger, it should run my target executable. And
12:01
it does. So, my little target executable just outputs a line, returns a value. And then we see that output line telling us the debugger has finished. Right. Now, I'm going to enable P tracing on this target executable. Which I do by setting a
12:21
attribute. And I say, I would like some P trace, please. So, this is saying, when you fork this process for the target executable, I want to attach P trace to it as well. And when we do that, we
12:46
see that wait returned, I mean, it's come back as error is maybe a strong word. We've got a trace breakpoint trap. That got output. And then the debugger
13:01
completed. When the debugger finished, there was nothing holding that breakpoint up anymore. And our target executable was allowed to continue. So, when we get this stop signal, the target executable is just sort of held waiting for waiting to be
13:20
told what to do. Because P trace has stopped it with that. Well, at the moment, just right at the start. So, now I want to let that executable continue. But we're going to set a breakpoint at a particular line. We want it to continue up to that breakpoint.
13:41
So, what do we need to do? We need to set, we're going to use P trace, oops, poke data. And we're going to write into the memory for this particular process. And we need a process ID. We're
14:03
going to stop at the program counter address that corresponds to a line we want to stop at. Why don't we use this one we had earlier? So, we'll stop at line 22. We've got the program counter that corresponds to that. And there's something else I need to do here. Oh, I need
14:21
to actually write in the hex CC value. So, I need to there we go. So, that writes that byte code CC at the correct address in memory. I need that process ID. We get
14:41
that from here. Okay. Having put the breakpoint in, I now want to let the program continue. So, I can do that with P trace. So, allow that particular process to continue
15:06
until any signal is received. Then we have to wait for something interesting to happen on that process. Now, hopefully what's going to happen there is we're
15:23
going to get a another interrupt, another breakpoint trap. And at that point, we should find out something about what's happening. Let's get the state of the registers. P trace, get registers. So, that's the
15:41
process ID. And we'll read them into some registers. And we could look at the program counter. Now, confusingly, the program counter is also called the instruction pointer. Those are kind of the same thing. And I could print out the value of the instruction
16:04
pointer. I'm calling it instruction pointer because in this structure, it's called register RIP. So, that's going to tell us the address where we stopped. And we should also convert that from a program counter to a line
16:24
number. So that we can see in machine readable form where we are. So, I am going to convert the current instruction pointer to a file. I feel like the sound is
16:41
cutting in and out. Okay. Right. Let's see what happens. Okay. So, we stopped and we got this output that told us that we actually stopped in, well, exactly where we wanted to at line 22. Which was
17:01
inside that function F3. We also got the output that told us what the address in memory was. Kind of what's taken us to the point of hitting a breakpoint in a debugger. But we would like to see some interesting
17:21
information about the state of the target executable at this point. Just knowing where we stopped to tell us anything new. So, what I want to do is output the stack frame. Or the stack trace, rather. What's the stack trace? Let's have a look. So, we need to talk about a couple more registers in the CPU for this.
17:45
There is a stack pointer that points to some part of memory and a base pointer. And between those two addresses is the current stack frame. And that's like a kind of scratch pad for whatever function is currently
18:03
operating right now. So, we might have things like parameters, space for return values, any local variables. They all get allocated space in this stack frame. Then when we go from, well, when we call another
18:21
function, interesting things happen. The current program counter gets pushed onto the stack. And that tells us where we're going to come back to when we return from the function we're about to call. The base pointer moves to where the current stack pointer is pointing to.
18:42
And then the stack pointer, we allocate a new frame on the stack. So, the stack pointer is that's a new frame for the new function we're about to call. But because we put the old stack pointer on the top of the stack, we
19:01
can chain through pointing back to the previous stack frame. And we can use that to look at the sort of chain of functions, the hierarchy of functions that we've called to get to this point. And we can also look at the contents of that stack frame to look at things like the address that we are about to return to when
19:24
we exit a particular function. Now, writing all this stuff out would take me a little bit too long to do here and now in this talk. So, I have cheated a little bit. And I've got a thing called output stack. Again, you can see the source code for
19:42
this later. And this takes my symbol table, my process ID. Oh, I've spelled that wrong. And a few registers. So, we need the instruction pointer, the stack pointer, and the break point. Base pointer.
20:06
All right. So, now, we not only stop at line 22, but we get this output of what's on the stack. And I've looked at things like what the return address, you know,
20:22
how we saw the current program counter address gets pushed onto the stack. So, I've used that to see how that kind of function hierarchy got put together. And we can also see some interesting things like all the threes, all the ones, all the twos. If I look
20:43
at my target executable, you can kind of see where that's come from. I've sort of deliberately put things that we can easily identify. So, those are the local variables and the return values. We can see those kind of on the stack. So, that's kind of cool. But
21:05
when you're in the debugger, you can step through, right? You don't just want to stop in one place. You want to be able to allow the program to carry on and look at the trace, you know, at another point in time. So, we can do that pretty easily. Let's just now rather
21:24
than hard coding this, I've got a little function that will let me enter whatever line number I want to stop at. And we'll put this into a loop. Now, let's just have a quick look at that. So, let's
21:46
say I want to stop at line 22. We get that stack trace. And now I can say what line I want it to stop at next. Maybe I'll say, well, it came from line 16. So, let's go to line 17. And that's where before we
22:03
were inside function F3. I started late. So, yeah. Where are we? Right. We are we move to the next
22:22
function. So, we're inside F2 where we were inside F3. So, we can stop at any line we like. But it would be really nice if we could single step. So, we're going to, based on the line number, if I do 0, if
22:43
I enter line number 0, I want to just kind of run to completion. And we can do that by just calling continue. And then I want to break out of this loop.
23:01
And just because of the way go is, I actually need to put a label in this particular case. Ask me about that afterwards. If I don't enter anything at all, let's do a single step. So, we'll step through an individual instruction. And that is pretty easy.
23:26
So, we're going to do a single step. I need to say process ID and 0. And any other line number, we will do what we did before.
23:45
If you were really eagle eyed, you might have noticed that we over wrote something. We stamped hex CC in memory here. So, it might be nice to replace what was there before. So, we can do that. What we're going to do is read out the previous value.
24:05
And we need something to write that into. Which is a make one bite slice. All right. So, that's just somewhere we can write the original value. Oh, thank you.
24:26
And we're going to poke that back in when we have reached the break point. Okay. That shouldn't make any great difference than it's restored our program. It should have contained. Okay. What have I missed?
24:45
This looks reasonably, reasonably okay. I'm just going to cheat and check whether I've missed anything. I need to wait for. Yeah. So, I do need to wait. Okay. So, now I can either enter a line
25:02
number or I can just hit return and it should single step. So, let's go to line 22. And now just single step. And you can see the values of the local variables and things changing as we go
25:26
through the program. We can see that hex 7, 7, 7 that we finally get kind of propagating through the program. So, if I run to completion, we see that all the 7s. So, one last thing.
25:40
I know I'm a bit late, but I'm still, I've only been going for 25 minutes. Right. The last thing we can do with ptrace that's really kind of fun is we can change the tracee, the targets, memory and registers. So, that seems like a cool thing to do.
26:01
Now, this was the machine code that we looked at before and it corresponded to the function F3. And if we go a little bit, if we look at this, we can see, you don't need to understand a lot of machine code to see, oh, here's this hex all the 4s
26:21
and that gets added into, well, ax I happen to know is the name of a register. So, seems pretty plausible that ax is going to contain a value that is kind of important in the, you know, maybe it's our return
26:41
value that we want to, or that we could change. So, let's just confirm my hypothesis by printing out that ax value. Registers are ax. So, if I go to line 22,
27:08
ax currently contains all the 3s. And if I just single step through, eventually we will get to the point where, I hope, eventually, why doesn't it get
27:28
to 23? I'm just going to do that. Oh, yes. It needs to be there, doesn't it? What have I done?
27:47
There we go. I think that's it. Okay. So, let's go to, I'm going to go straight to line 23. Ax contains all the 3s. And if we step through,
28:01
eventually, we see it's getting updated to all the 7s. So, I'm going to say, if I happen to see, if registers are ax is all the 7s, 1, 2, 3, 4, 1, 2, 3, 4.
28:24
And if we are on that line 23, let's set registers. What are we going to set them to? We are going to set our ax to, say, all the 8s, 1, 2, 3, 4, 1, 2, 3, 4.
28:48
Set the registers for that process ID to whatever we currently got in the registers. And just so we can see it's happening, let's go over writing. Very
29:01
important. Okay. Right. So, let's go to line 23. No, it's line 22. And we will single step
29:20
through and we'll see that ax register change. Well, it's all the 7s. We're just saying we're over writing it. Now it's all the 8s. We've been able to modify the value of this return value. And if I let it run to completion, we've changed the output value. So, ptrace is super powerful.
29:42
We can manipulate what's happening inside our target executable. Which ties us neatly back to container security. There is really very, very little reason for any containers running in security to have the
30:00
capability to use ptrace. Really just by default if you're using Docker, it will be off. Hands up if you're using Kubernetes. Right. You may or may not know this. Kubernetes does not by default set the same set comp profile as Docker does. So, you need to
30:22
explicitly choose to use that set comp profile and thereby disable things like ptrace. Ask me about that afterwards. So, I promised you a link to the code so you can check out all the bits that I didn't have time to type in this morning. It's there on GitHub on debugger from
30:40
scratch. I wouldn't have been able to do this talk without some blog posts by Mikhail Levitsky and Phil Pearl. Hopefully that's told you a little bit about how a debugger works. And I will be around. So, ask me any questions after. Thank you.