PACKET HACKING VILLAGE - Mallet A Proxy for Arbitrary Traffic
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 322 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/39931 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
DEF CON 2651 / 322
18
27
28
40
130
134
164
173
177
178
184
190
192
202
203
218
219
224
231
233
234
235
237
249
252
255
268
274
287
289
290
295
297
298
299
302
306
309
312
315
316
00:00
Goodness of fitOffice suiteLevel (video gaming)Information securityServer (computing)Proxy serverQuicksortInterprozesskommunikationTheory of relativitySoftware frameworkVariable (mathematics)Client (computing)Serial portCartesian coordinate systemRevision controlLink (knot theory)Codierung <Programmierung>Communications protocolMechanism designCASE <Informatik>Form (programming)Web 2.0Transportation theory (mathematics)Formal languageField (computer science)BitSocket-SchnittstelleInformation technology consultingPresentation of a groupDisk read-and-write headOrder (biology)Set (mathematics)Dependent and independent variablesRule of inferenceProcess (computing)Hidden Markov modelWave packetControl flowSlide ruleMathematical analysisMereologyLecture/Conference
05:33
Connected spaceServer (computing)Virtual machineComplete metric spaceObject (grammar)NumberDependent and independent variablesQuicksortCommunications protocolClient (computing)Intercept theoremRight angleProxy serverResultantStress (mechanics)Web browserRule of inferenceComplex (psychology)MereologyRevision controlSoftware frameworkMessage passingOperator (mathematics)Type theoryReal numberLibrary (computing)Table (information)Multiplication signComputer-assisted translationWrapper (data mining)CASE <Informatik>User interfaceLine (geometry)Software testingTerm (mathematics)Projective planeOperational amplifier1 (number)Lecture/Conference
11:00
Communications protocolStreaming mediaCASE <Informatik>Software frameworkComplex (psychology)Proxy serverObject (grammar)Traffic reportingVolume (thermodynamics)QuicksortPoint (geometry)Right angleProjective planeElectronic data processingVariety (linguistics)Software maintenanceComputer architectureLecture/Conference
13:01
UDP <Protokoll>Buffer solutionReading (process)Transportation theory (mathematics)Connected spaceMessage passingoutputNetwork socketComputer architectureProxy serverClient (computing)Spherical capRule of inferenceIntercept theoremLibrary (computing)SoftwareNumberMechanism designCommunications protocolWindowMetropolitan area networkDifferent (Kate Ryan album)Computer fontResultantVariety (linguistics)Direction (geometry)Web browserCartesian coordinate systemVirtual machineIndependence (probability theory)DampingSerial portInterprozesskommunikationTable (information)Food energyVideo gameDefault (computer science)Graph (mathematics)Lecture/Conference
17:20
Message passingRow (database)Point (geometry)WindowMotion captureSoftwareConnected spaceComplete metric spaceArrow of timeResultantLine (geometry)Directed graphState of matterBlock (periodic table)Event horizonNetwork socketGraph (mathematics)Revision controlMultiplication signDefault (computer science)RectangleUser interfaceMechanism designBitDemo (music)Group actionDifferent (Kate Ryan album)Level (video gaming)Object (grammar)System on a chipMusical ensembleComplex (psychology)Web 2.0Intercept theoremLecture/Conference
21:20
CASE <Informatik>Connected spaceMessage passingSocial classDifferent (Kate Ryan album)InformationCalculationArray data structurePoint (geometry)Codierung <Programmierung>Event horizonComputer programServer (computing)1 (number)Line (geometry)Right angleMultiplicationString (computer science)Single-precision floating-point formatInheritance (object-oriented programming)NumberJava appletObject (grammar)AdditionCodecModule (mathematics)Constructor (object-oriented programming)Field (computer science)Parameter (computer programming)CountingDuplex (telecommunications)2 (number)Video gameSource codeClient (computing)Graph (mathematics)QuicksortAbstractionMereologyDrag (physics)Context awarenessBitInsertion lossInstance (computer science)LogicProcess (computing)Default (computer science)CodeDirection (geometry)Communications protocolOrder (biology)State of matterSoftwareIP addressIntercept theoremWindowLecture/Conference
31:17
MereologyObject (grammar)ResultantMessage passingComplex (psychology)CASE <Informatik>Representation (politics)Streaming mediaCodeText editorCategory of beingClient (computing)Java appletGraph (mathematics)Right angleSingle-precision floating-point formatLibrary (computing)Communications protocolScripting languageString (computer science)MathematicsoutputMultiplication signUser interfaceBitMechanism designProcess (computing)Software frameworkServer (computing)ParsingRevision controlSequenceInformationReflection (mathematics)SummierbarkeitDefault (computer science)Serial portLecture/Conference
39:06
Message passingGraph (mathematics)Point (geometry)Object (grammar)Computer programBitSoftwareRight angleCategory of beingNumbering schemeField (computer science)Special unitary groupLecture/Conference
40:42
Direct numerical simulationDependent and independent variablesObject (grammar)MathematicsDifferent (Kate Ryan album)INTEGRALType theoryZoom lensDefault (computer science)CASE <Informatik>Content (media)Client (computing)Server (computing)Process (computing)Order (biology)Variety (linguistics)Graph (mathematics)Scripting languageConnected spaceBranch (computer science)Communications protocolPoint (geometry)WindowPublic key certificateGoogolData storage deviceLibrary (computing)EmailMessage passingComa BerenicesStreaming mediaCodecCodierung <Programmierung>Computer fileSemiconductor memorySoftwareBlock (periodic table)Connectivity (graph theory)Presentation of a groupComplex (psychology)Graph (mathematics)Greatest elementWeb 2.0CodeBitComplete metric spaceRegulärer Ausdruck <Textverarbeitung>AdditionMaxima and minimaLevel (video gaming)QuicksortDemo (music)Matching (graph theory)GodNetwork socketParsingJava appletRight angleSocket-SchnittstelleLecture/Conference
49:59
Connected spaceAttribute grammarPresentation of a groupState of matterScripting languageTwitterProxy serverCASE <Informatik>File Transfer ProtocolSoftware repositoryClient (computing)Communications protocolGame controllerWeightLecture/Conference
Transcript: English(auto-generated)
00:00
So without much ado, it is my pleasure to introduce to you Rogan Dawes. Good morning everyone. Can you hear me fine? Awesome. Thank you Ming for that introduction
00:21
and thanks to all of you for making this your first talk of DEFCON or at least of today. I was a little bit nervous when I saw I had the 10 o'clock slot. It's like oh people are still going to be sleeping after partying last night. So I appreciate all of you who've come out and chosen to come here. So what is this talk about? It's going to be a presentation
00:42
and demonstration of a tool called Mallet that I've been working on for the past year or so. It's a framework for creating intercepting proxies for protocols other than HTTP and HTTPS. It doesn't operate at the sort of ARP level. So somebody said arbitrary
01:01
proxies. Can it do ICMP and ARP? No. It's not aimed at that level. That would be a bit too low level. But TCP and UDP proxies absolutely capable of doing serial. So any sort of serial kind of links and possibly Bluetooth. So it's got a mechanism for arbitrary transports.
01:25
So those things can be built in. So it's written in Java based on the Netty framework and it is scriptable in any JSR 223 compatible language. So we can move on then. Okay so this is the obligatory corporate sponsorship slide. They paid for me to come
01:42
here. So I got to tell you about us. I work for a company called Sensepost. We are a fairly small information security consultancy based out of South Africa. We have our head offices in Pretoria. We have offices as well in the UK, in London and in Cape Town.
02:03
So we do security analysis, assessments, we build, we break and we do training as well. We've been training at Black Hat for the last 18 years or so. So we were founded in
02:21
2001 on the 14th of February, Valentine's Day. What else are nerds going to be doing, right? Okay and if you want to get hold of me that's who I am. So why do we care about intercepting arbitrary protocols? We've got these tools for intercepting HTTP and HTTPS
02:42
and that should be enough, right? Realistically communications in various forms are fundamental between most of the applications that we actually care about these days. You know a standalone application is mostly not particularly interesting. So most of that traffic is, well a good chunk of it is using HTTP and HTTPS and we've got a
03:05
good set of tools for working with that starting from web scarab almost 20 years ago but now more recently the burps and the zaps etc. are doing a pretty good job of handling HTTP and HTTPS related protocols. The question then arises though what about all of those
03:29
applications that are not using HTTP and HTTPS? I think a lot of people go I don't really know what to do about this. They go yeah it's using SSL let's call it good.
03:43
You know I can't get in the middle of this you know what do you want from me kind of story. Whereas a version of that communications could in very many cases result in the kind of
04:00
pwnage that we saw back in the early 2000s where we saw things like hidden fields used to store you know prices for shopping cart items and that sort of thing and those things I think in many cases may exist simply because nobody's looked at them. So there's definitely great
04:23
opportunities for looking at these other protocols and just seeing what's actually going on. Okay so on the one hand we've got a nice orderly protocol like HTTP which is text based
04:46
human readable request response very simple one party asks a question the server responds and that's the end of it. Very neat and tidy and orderly and you know it's a simple thing to
05:03
okay I'm going to say simple but it's relatively simple to intercept yes there are variables like chunked encoding and web sockets and etc etc but in the main HTTP and HTTPS is a simple protocol to deal with. On the other hand we've got all of these other protocols
05:23
which may have very different rules for how they operate for example who speaks first does the server speak first or does the client speak first if you look at SSH for example they both speak at the same time they both start sending out a message of what protocol version
05:43
they send so understanding those rules becomes a fundamental requirement for building proxies for a particular protocol. Other details that may arise things like response ordering
06:04
if you send a request does the response have to come back in order or do you number them number your request like for example IMAP you send a request in IMAP you can send a numbered request number one numbered request number two and get response two back before you get response
06:23
number one they come back with a number attached to them so that you can re-associate them but you know it's a very different thing to what we may be familiar with using HTTP proxy interceptors. Other details that arise things like timeouts in your browser you've got a very
06:43
forgiving client how many of you have left burp intercept on come back a day later and found a thousand requests stacked up waiting to to be fulfilled and when you release those requests what happens does everything go oh no i died i timed out this is a disaster no they just
07:02
accept the answer and carry on going you know it's no big deal in other protocols it can very much be a big deal with these timeouts built in and those timeouts can be on the client side as well as on the server side i think some of you may have heard of attacks like slow loris
07:23
so slow loris is an attack where you connect your atdp server and you start typing your request really really slowly so it uses resources to process this connection so you can tie up the server with many many long-lived connections so the server starts implementing timeouts if you
07:41
take too long to type your request or to send your request that connection is going to drop you don't see that when you're interacting with with the server with burp and zap etc because that connection to the server is actually closed while you're intercepting a request it only establishes a new connection once you say go establish the connection gets the answer it's
08:05
quite happy to close that connection immediately afterwards the acdp protocol supports that kind of operation it doesn't require these long-lived tcp connections okay so just to contrast some of
08:20
the the approaches and the the complexities involved in dealing with protocols other than the the familiar okay so as a result most proxies for protocols other than acdp and https tend to be the sort of one-off complete clutches
08:41
you know just enough to test that one thing that the person was trying to do no nice user interface or flexibility built into it and you know there may be post pasted as a gist on github or something along those lines but you know there's never any kind of community or reusability or anything like that uh built in and in many cases you'll find that these
09:06
intercepting proxies are a wrapper around a tool like socat the uh the real swiss army knife okay so there is prior work um people who have tried to create frameworks or intercepting
09:20
proxies and put them out there oh was proxy is a tool i wrote around 2008 um it was a programming library so i kind of abandoned that fairly early on because it really didn't fit in as a tool for people um to use rather it was a you know supporting infrastructure if you like martin swender
09:42
wrote hack it proxy based on top of oas proxy um that didn't get very much much traction um malory is probably the most well known of the available in um intercepting proxies for arbitrary protocols that was created around 2010 i think the last update for that is
10:02
around 2011 so it's kind of abandoned and dead um and it kind of suggests that you know this is a hard problem if the people who wrote the tool aren't even using it themselves you know you would expect them to be making those updates available um a bunch of others all of these ones
10:23
with a slash in them are referring to github projects um most of them again you know kind of clunky and difficult to get set up um malory is probably amongst the worst culprits in terms of difficulty and getting it you uh getting it working you've got to set up a virtual machine
10:41
and then you've got ip tables rules that you've got to worry about and routing tables and all of that sort of nonsense and it's really a complicated thing just to get started and i think that's part of the problem right nobody wants to go through the pain just to intercept their first packet so my some of my objectives when um when creating mallets was to get around
11:08
those pain points and make it simple for people to actually get started using mallet you know once you actually sort of get that initial gratification hey i'm intercepting a packet now you can work forward and start understanding the complexity of the tool um if necessary you
11:25
know if it's if it needs to get more complex in your case and obviously that's a protocol dependent thing right the complexity of what you're trying to analyze will depend on the complexity of that protocol the other objective that i was trying to to avoid or trying to
11:47
a lot of these existing proxies come with a very very limited support for a variety of protocols and what that means is that if you want to intercept those protocols or interpret them from a byte stream into something meaningful you have to create those protocol decoders yourself
12:06
and you know i don't know about you but i'm lazy i would much rather not do that work myself if somebody's done it already let's leverage that and uh and make use of it so a few years back i stumbled upon the neti project has anyone used neti
12:23
anyone here okay one over there all right so it's a really cool project um it's a an amazingly designed framework a really high performance it's used by some of the largest companies in the world for extremely high volumes of data
12:41
processing i saw a report by um i think it was trust in lee or norman mora one of the um the maintainers of the neti framework that apple uses it to transfer something like four petabytes of data a day so yeah there's a lot of thought which has gone into it and it's really nice and cleanly designed um okay so this is a sort of overview of their architecture
13:07
um suffice really to say that there is a large existing body of protocol support and it's a really neat design that allows you to simply compose your protocol by including
13:24
handlers at appropriate places and as i say there is a large body of existing handlers that you can simply make use of rather than having to write all the all of those from scratch the other neat thing about neti is that it is transport independent so it supports a variety of different transports you can intercept tcp udp sctp is um serial so
13:48
uart um communications or uart transports available um and as i say i think it's definitely possible to create a bluetooth transport and that would allow us to do like bluetooth low energy
14:00
man in the middle um work all right so one of the fundamental concepts that neti um builds on is this concept of a channel pipeline the transport represents uh constructs a channel so it's a connection from one end to another and then there is this pipeline of uh of handlers
14:23
and the the pipeline starts at the network layer you'll have your socket read and then a byte buffer an array of bytes essentially gets passed into the first handler which can then choose to process those bytes in some way and then pass the result onto the next handler
14:43
in the pipeline and so you can see you can construct fairly arbitrarily complex pipelines you might start off with an initial ssl handler so it takes in encrypted bytes does that hand shaking as an example of the inbound outline handler number two you've got encrypted bytes
15:03
coming in the handshake happening with the client so more encrypted bytes going out and eventually you'll get some plaintext out of that and those plaintext bytes will get passed up the pipeline for other handlers to do to to process so how do we turn this into a proxy it's really
15:24
really easy all we do is we bridge two pipelines together or two channels together and we do that with um what i call a relay handler it simply takes messages coming in or coming out the end of one pipeline and copies them over to the input of the next pipeline
15:42
so those get written back out to the network any bytes coming in the other direction the same thing happens so really simple um really what you're doing here is just providing that relay mechanism as well as a a mechanism to observe what those messages are okay so i moaned about
16:04
how difficult it was to set up malory with ip tables rules and um you know routing and virtual machines and all of that um there's an existing protocol can anyone tell me what talking about here socks okay so socks is a um a protocol designed for specifying to a proxy
16:27
where you would like to connect to and for a ccp connection that is the simplest possible thing that you can that you can achieve makes life really really easy there are um many ways of of activating a socks proxy whether it is by choosing a socks proxy in
16:47
your user agent like your browser they've got built-in support for socks um or using a an intercepting library like um t socks or proxy chains on on linux or osx or um on windows
17:04
there's this really nice library called um called proxy cap that you can use to specify precisely which application it is that you're trying to soxify and intercept that application's traffic
17:20
okay so this is an example of the default graph that mallet gives you when you start it up the first come first thing at the top is a um a socket channel or a listener that's listening on a tcp port and that is just waiting for an inbound connection after that the rectangular blocks are
17:44
various handlers which are in place along that pipeline and then finally the sink represents that outbound connection to its destination so having a listener is all very well but you know if we receive an inbound tcp connection we don't necessarily know at this point where we wanted
18:04
to go now where should we be connecting our outbound to so the easiest way of figuring that out is to to add the sox initializer and that wraps a bunch of different handlers that you don't really need to care too much about suffice it to say that it figures out what the destination is and makes that available to the relay handler the next rectangle in red
18:28
is an intercept handler this is going to intercept the various events that neti generates that allows us to see the requests the various messages and the state of the connection at any time next is the relay handler that just copies those
18:47
messages across from one pipeline to the other and then you can see the arrows are now blue so that represents the outbound connection sorry the arrows are now red on the on the
19:00
outbound connection we have one more intercept handlers that's now representing the events that are happening on that outbound connection and we can capture those as well so just for a note the event capture and recording happens in an intercept handler those get reported up to the user interface this is an example of some of the methods that
19:27
you have available to you if you want to manipulate the packet or the data or the object in transit channel active is fired when the connection is established and when your handler is added to the pipeline so you can use that for setting up some initial state or maybe
19:45
triggering an initial message channel read we have received a message from the network here it is right we have a message going outbound to the network this is the message you can now process it at this point and then user event triggered just gives you a little bit
20:03
of a mechanism of passing state messages or event messages things like ssl handshake successful or handshake failed or something along those lines may be passed as a result of this event handler
20:22
so here is a really really simple event handler this is taking an inbound text web socket frame replacing the text with the uppercase version of it so a you know a dead simple example but it illustrates that i don't need to worry in my
20:43
handler about parsing i don't need to worry about have we read enough bytes to actually read a complete text web socket frame that's all being handled by other handlers not my problem when we get to this point i've got a complete text web socket frame and i can process it and
21:02
pass it on and there is very very little complexity at this level that you have to worry about okay demo time okay so for my demonstration i'm going to be showing you a
21:34
very simple text-based json connection so it's easy for us to see
21:43
it's going to make a connection to a server that i'm running on my local machine on port 9999 and if i provide some details i can send it off to the server and it does some calculations and gets a message back okay not particularly exciting
22:08
um let's take a look at what's actually going on so let's close that and intercept it
22:25
that's all that's needed to do to actually intercept the messages that are being generated by this client so now if i bring my mallet across so that you can see it
22:48
and i go back to my client so this is the default graph that i showed you your tcp connection it's this thing for a sox connection intercept relay intercept and done
23:04
and we can see the data that gets captured on the connections tab source and destination ip address and a bunch of other fields a number of events that have been seen when it was opened and it was closed it's a little bit difficult to show on a small screen so that's not all that
23:28
important anyway okay so let's submit that and we can see here is our connection and there are a whole bunch of events which were triggered and what that's showing channel register these
23:46
are all reflecting the neti state in essence so a number of them you can ignore probably the most useful ones would be things like the user event triggered which gives us some information hey we've got were requested to connect to localhost 9999 that was triggered by our sox connection
24:05
and then we'll see things like this channel read message and though that's probably going to be your most interesting that is the data which was read from the network and in this case we can see we've got a curly brace and i'm going to skip that because it's the right the
24:21
outbound right on the other channel then we've got a line of data a snippet of json and more json etc etc etc this is not particularly useful i mean it's all broken up into into
24:41
lots of different pieces so it's a real pain to work like this rather let's try and frame this message into you know beginning and end and then combine that into a single thing that we can work with so neti amazingly has an existing handler for this not reinventing the wheel all it does
25:04
is it counts braces so you'll have a an opening curly brace and as you count as you open more curly braces and you get nested further down it just counts up and as those braces get closed counts down when it gets to zero end of message and it's got some logic for handling you know
25:21
quotes strings with curly braces in recognizing that it's in a quoted context and doesn't affect the count so let's do that i'm going to open a demonstration graph not that one and what we've done is we've just added this additional module or additional handler
25:57
uh this json object decoder class that i've just mentioned
26:02
so it's going to to decode these uh these bytes and just amalgamate them into a single message so i added it on the inbound connection and i've added another one on the outbound connection as well just in case the server is sending um chunk you know fragmented messages as well
26:24
there it is okay let's run that again so we can see that inbound connection but now instead of having multiple byte buffs byte arrays or messages read you can see we've got
26:45
a ton of reads and read completes no data until that json decoder has actually finished reading the whole packet and there we can see the entire message has been amalgamated into a single message so that now makes it a lot easier to work with we can we can process this in one
27:09
piece without having to like step between all the different parts of it but you know it's json right yeah json's a text-based protocol we don't really want to deal with it using an array of bytes that's not particularly friendly so what we can do now to to simplify our life
27:31
is to translate that message from an array of bytes to a string a java string so we've got our
27:46
json object decoder that we added previously and then once we've decoded that entire message we're going to pass that through a a string a pair of string encoders and decoders the string encoder and decoder the decoder will simply take an array of bytes and transform that
28:05
into a java string in the opposite direction you'll have a string coming out we want to convert that back to bytes because we can only write bytes to the network not strings so the string encoder will encode that string as an array of bytes for us and
28:21
you can specify constructor parameters or constructor arguments that that are needed by our string encoder in this case we're going to consider our array of bytes to be you to be encoded using utf-8 encoding which is an important detail
28:42
so you can see here we've got the class definition and that is simply the name of the built-in neti class that is that is available for us to use so any existing neti handler classes we can make use of in our pipeline just by you know dropping a new new object into our graph and
29:06
choosing the the class name and any parameters that may be required okay let's try that again so now you can see that instead of receiving an array of bytes
29:36
we've done a very simple thing we've dragged and dropped these additional handlers into place
29:41
and now we've converted those bytes into into something that we can actually start using and i'll point out that there's no programming involved here we actually haven't had to write a single line of code in order to actually do this the the graph construction is a matter of
30:02
you know choose the handler that you want from the from the palette drag it into place like i want to add this in between here and it inserts that into the pipeline and then you can choose the class that you want in this case channel duplex handler is the
30:21
sort of base class and abstract class well not abstract it's a complete class and then you can just override the various methods so this is a do nothing instance but it's it's illustrative of of the process right okay back to our connections so now we can see that we've received a string
30:43
which now makes our life a lot more easy we can see what's going on and we can manipulate this in a much more friendly manner so if we were to turn on intercept go back to our
31:00
client you'll see that this connection is now being held and we've only got a couple of events that have been triggered initially and we can now send those events on and oh damn what happened time out exactly right so our client
31:29
was unhappy about how long we spent making this to to forward on that that connection and as a result it's disconnected so it illustrates once again what i was talking about
31:45
the complexity of intercepting protocols that are non um non-http you have to be aware of things like timeouts and in some cases you may have to be fairly snappy when it comes to
32:00
responding to these these intercepted messages so the ideal situation is really to script any changes that you have to make and i try to make that easy for you and i'll show show how that works all right let's try it again and try and be a little bit snappier about it
32:31
okay let's send those on client is happy here's our intercepted data and i can now change it
32:43
and change some of the data that was sent and turn off intercept and there goes the data and instead of 46 years old now it thinks i am 26 year old okay so i hope you can see that
33:11
mallet actually makes it pretty simple to construct these pipelines and um you know and start the process of understanding what the protocol looks like not every protocol is going
33:24
to be as simple as this obviously you know we're going to have binary protocols you know maybe smb for example and i would be kind of hesitant to try and intercept smb but who knows you know you might actually find something interesting there would be a fair amount of work i'm not trying to take that work away from you but what i am doing is i'm giving you the
33:43
framework to try and make that as easy as possible to avoid you reinventing the wheel rebuilding this user interface um you know this mechanism for you to start drilling down and understanding what it is that you're doing okay so there's a detail
34:04
that i have not shown you here is our server it's showing the information that was just received but it's also showing that the checksum that it received no longer validated so now we have
34:23
a problem we've tampered with a packet and we haven't reconstructed that md5 sum that was being sent as part of that json message what to do well i mean obviously there's a detail you need to figure out what goes into that checksup in this case it's the username surname and date
34:45
of birth concatenated together and then converted to bytes and checksum calculated but you know then if we know that we can maybe go out to the command line and we can recalculate that and paste it back in its right place but that's a mission right it's not something
35:03
that you're going to want to do on every single request you're going to want to automate this so let's look into how we're going to do that it's all very well having a string version of a json
35:22
message but in some cases we're going to want to parse it into something a bit more intelligent like provide an actual java representation of our object so let's take a look at the next graph what we're doing in this graph is we are adding a script handler
35:53
that is making use of the jackson java library the jackson library is a json parser or json
36:03
library that is designed to make it easy for you to interact with json objects so it can deserialize and re-serialize json objects and that's great that's what we want once again this isn't code that i wrote only code i wrote was this tiny little snippet that tells us
36:23
when to invoke the jackson library so here we have on an input strip we receive an array of we convert that into a stream and we tell jackson to read a json object from the stream we then
36:42
emit the json object into the pipeline so that it can be processed further and similarly on the outbound step we're going to encode the object as a sequence of bytes
37:01
simple enough back to our client not the client whoops come on so let's take a look at what we got as a result again we weren't intercepting this one
37:28
but here you can see we've actually got now a java object that represents that json string we had previously and of course nobody wants to worry about writing or creating
37:44
a custom editor for every kind of object although it can be useful not every object is the easiest thing to drill down into using java reflection but as a default i've provided this reflection editor that interrogates the various properties of the java object so you
38:08
can see the various properties of that that json object was which was being passed around all right let's intercept now play it one more time and here is our intercepted json node
38:29
i'm just going to turn that off and once again having modified the json object we've
38:53
re-serialized that it's now gone out as a stream of bytes finally and come back and we can see that once again the client is happy but unfortunately our server is not so it's still
39:07
seeing that it's invalid we've done nothing at this point to update the checksum next step our final graph added one more scripted handler that is now updating the the json
39:41
object so when it sees a json object coming in on a right being written outbound receives the message it checks that it is in fact a json node so it's a bit of defensive programming making sure that the object that you're dealing with is in fact what you're expecting and then it gets the various properties which are required to calculate that checksum
40:04
update calculates the checksum and updates the appropriate node so now i can modify any json packet or json object coming through that uses this scheme obviously details right
40:21
that uses this encoding scheme that has a name surname and date of birth and a checksum field and i don't have to worry about that checksum because it'll be automatically recalculated for me before it gets written out to the network so taking too long there we have it okay so
40:55
once more i can drill down into that object change whatever i like
41:08
send that off and back it comes with a ridiculous date come on
41:30
but importantly the server is happy with it so this demonstrates reconstructing the protocol to ensure that it retains its integrity that the server all the checksums and anything else
41:43
that's required by the protocol is retained and the process that you can go through in order to establish you know drill down and see what's actually going on in this protocol
42:03
so mallet can create graphs of fairly arbitrary complexity including branches which is a nice detail i think so for example i'm going to open the the sort of demo graph and let's zoom out
42:32
so in this case we've actually got two independent graphs going on one is a TCP listener and the other is a UDP listener the TCP listener is accepting
42:43
SOX connections as we've seen and then there is a branch that branches depending on the port that is being requested so this could be a regex max or match on a particular host name but in this case i'm only caring about the port component so if you're connecting on port 443
43:02
it'll go down this branch port 80 it'll go down this branch and then anything else will just get simply relayed as we've seen already for port 443 the first thing we're going to do is SSL interceptor and mallet does have the capability by default it will automatically
43:24
generate a certificate that matches the host name that was requested so much like we've seen with burp and web scarab etc in the past you don't have to worry about you know clicking through certificate warnings all you need to do is install the CA cert that mallet generates
43:44
in your browser or in your user agent or your certificate store and you can make these outbound connections without oh sorry you can intercept connections without triggering those warnings following on from that we have an HTTP server codec what that's doing is it is decoding the inbound bytes and transforming those into an HTTP request object
44:08
well it's actually two different types of objects the first because neti is a high performance library it parses out the headers separately from the content of the messages you don't
44:21
necessarily want to have to buffer an entire 2 gig file upload into memory so neti by default will stream chunks of content up the pipeline allowing you to process that dynamically for our purposes if we're only going to be dealing with small messages probably chunking those up into a single single object is a bit friendlier to work with so there is this
44:46
additional HTTP object decoder so what that does is it just aggregates the messages and the those content chunks back into a single object the next step is in this case a web socket
45:03
upgrade handler so just demonstrating once again so complexity of of protocol handling and the ability to do pretty pretty amazing things with a graph what this is doing is it is listening for a successful web sockets upgrade message so if you were to start negotiating a web sockets
45:24
connection with something it would be able to monitor that hey there is now a successful web socket protocol upgrade if i still sit here with an HTTP handler i'm going to corrupt these messages entirely and i'm going to fail to parse them so it switches out the HTTP codecs
45:42
and replaces it with an appropriate web socket frame decoder instead and similarly on the outbound connection we've got that web socket frame upgrade handler we've got HTTP object encoders in this case a client encoder it's taking an HTTP request object and writing it
46:04
out as bytes and then eventually the SSL client it's last item down at the bottom the plain HTTP is very similar just without the SSL components installed on the right hand side we've
46:24
got a UDP protocol handler demonstrating a manipulation of some DNS messages if you were to take a DNS client and point it to port 1053 on localhost you would send a request for say google.com and you'll get back an answer for sensepost.com it's changing the request or the
46:45
details that you requested in that message to something completely different and then rebuilding that DNS response with a certain amount of protocol knowledge obviously it did take me a while to figure out that my client's complaining because it's receiving a response for sensepost.com
47:04
when what i was actually asking for was google.com you need to reconstruct the response and say hey this is actually a response for your request to google.com anyway so we have a fixed target handler here which says at a UDP level we're not using SOX so it needs to know where to
47:24
send that outbound connection to so here we just say connect to port 8888 google's DNS um the script handler is just setting up the DNS um parsers uh and the reason for that is
47:47
actually i could have i could have dragged those on as um as blocks it's not really a good reason for them not being there um intercept handler logging handler dns rewrite this is a handler that i wrote in java you can use a variety of scripting
48:08
languages as you've seen there was the script that was written in groovy that did the update of the json objects um this it's sometimes easier to write it in java
48:20
simply because you get the the syntax highlighting and code completion and i'm lazy um and so yeah so that's just that a simple demonstration of a dns packet and let me show you how it works there we go sorry and it's look up versus dig i do not have intercept on why is
49:27
this not working demo gods don't fail me now yeah on 1.053 i'm not connected to the network so that's a good explanation for why that's not working all right so if i had run that successfully
49:45
you would have seen a response coming back for sensepost.com instead of google.com and so that is the end of my presentation if you have any questions i am not going to be running
50:05
away you can also get hold of me at rogan doors on twitter the repo is at sensepost um the github repo is at sensepost slash mallet and please send me pull requests if you have any problems um this is thanks for the presentation um i was wondering whether you maintain some
50:27
global state uh between the different uh sessions so you you here in this case you only had one connection going on to the proxy and so on do you have a case where you have where you have multiple maybe concurrently from different clients you actually maintain some state that
50:43
can be then manipulate manipulate it using a script right so each connection has what they call connection attributes i provide a binding to script you know for the scripting scripted handlers to access and you can store state in there and that will get passed
51:04
between connections so for example you could use that to implement something like an ftp interceptor you've got a control connection that is saying you know please let me connect to a
51:23
particular port and then you can make a note of what those what that port is that was that it was told to connect to and then in the next connection coming in you could then put an appropriate protocol decoder on that on that connection based on the state that
51:41
you've maintained anyone else thank you thanks for coming