Hiding Wookiees in HTTP: HTTP smuggling is a thing we should know better and care about
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 93 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/36282 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
DEF CON 2429 / 93
4
6
7
11
15
20
26
33
34
35
36
39
40
46
49
53
58
62
63
66
68
72
79
90
92
93
00:00
WikiRoundness (object)Demo (music)Computer animation
00:33
Dependent and independent variablesFormal languageMaxima and minimaCache (computing)SpywareStack (abstract data type)Open sourceInformation securityFormal languageHTTP cookieInformation securityWeb 2.0MereologySoftware developerDemo (music)SoftwareFreewareMultiplication signProcess (computing)Revision controlVector spaceInjektivitätWordSystem administratorComputer animation
02:11
Software testingOpen sourceEmailObservational studyServer (computing)Data storage deviceServer (computing)Open sourceQuicksortReal numberUniform resource locatorAbsolute valueThread (computing)Observational studyEmailComputer animation
03:19
Dependent and independent variablesMessage passingWeb browserLevel (video gaming)Exploit (computer security)Open sourceServer (computing)WindowHacker (term)Disk read-and-write headTDMAProxy serverFront and back endsContext awarenessDependent and independent variablesWeb browserFlow separationComputer programmingConnected spaceLine (geometry)Proxy serverFront and back endsReverse engineeringQuery languageMessage passingMultiplication signCache (computing)InjektivitätMedical imagingPortable communications deviceError messageLibrary (computing)Computer fileDemosceneRight angleRadical (chemistry)Order (biology)Level (video gaming)Revision controlComputer animation
07:25
Message passingTransmitterDiffuser (automotive)Dependent and independent variablesProxy serverCrash (computing)Density of statesShift operatorStreaming mediaSpywareHTTP cookieQuery languageDependent and independent variablesCrash (computing)Heegaard splittingStreaming mediaReverse engineeringDiffuser (automotive)TransmissionskoeffizientComputer programmingType theoryProxy serverQuery languageExploit (computer security)GradientCache (computing)QuicksortComputer animation
09:05
Content (media)Data transmissionEmailRadio-frequency identificationExploit (computer security)Large eddy simulationComputer animationLecture/Conference
09:34
Data transmissionEmailRadio-frequency identificationExploit (computer security)Content (media)Line (geometry)MultiplicationHeegaard splittingCodierung <Programmierung>Heat transferSpywareDemo (music)Front and back endsDependent and independent variablesProxy serverHTTP cookieReverse engineeringExploit (computer security)HTTP cookieDependent and independent variablesError messageSystem identificationDoubling the cubeQuery languageMathematicsLengthLogical constantWell-formed formulaSpacetimeLine (geometry)Real numberComputer programmingMessage passingOptical disc driveArrow of timeData transmissionTransformation (genetics)Demo (music)Binary codePressureHeegaard splittingTrailRevision controlProxy serverReverse engineeringCodecContent (media)Heat transferConnected spaceSystem administratorRing (mathematics)SpywareEmailComputer animationLecture/Conference
14:46
Content (media)Raw image formatSpywareDependent and independent variablesComputer programmingConnected spaceFront and back endsSpecial unitary groupHeegaard splittingCache (computing)Proxy serverReverse engineeringComputer animation
16:00
EmailContent (media)SpywareDemo (music)Large eddy simulationRevision controlDependent and independent variablesTransmitterStreaming mediaFront and back endsInformation securityReal numberMultiplication signFrame problemError messageConnected spaceContent (media)Dependent and independent variablesRevision controlDemo (music)Streaming mediaHTTP cookieCommunications protocolUniform resource locatorLengthOcean currentType theoryMedical imagingDigital rights management2 (number)Computer animation
17:47
Streaming mediaContent (media)Computer-generated imageryDemo (music)Asynchronous Transfer ModeMessage passingRange (statistics)Cache (computing)WritingReverse engineeringFront and back endsProxy serverTelecommunicationMedical imagingMessage passingDependent and independent variablesServer (computing)Cache (computing)Roundness (object)MereologyCase moddingMUDSquare numberDigital electronicsNetwork socketContent (media)Computer programmingReal numberAsynchronous Transfer ModeFerry CorstenSocket-SchnittstelleQuery languageError messageComputer animation
19:11
Dependent and independent variablesCache (computing)Heat transferCodierung <Programmierung>Price indexContent (media)HTTP cookieRange (statistics)Revision controlComputer-generated imagerySpacetimeQuery languageSoftware bugMereologyArithmetic meanHeegaard splittingError messageHeat transferLine (geometry)Asynchronous Transfer ModeCache (computing)MUDRoundness (object)Computer animation
20:18
Dependent and independent variablesCache (computing)Demo (music)CodeExploit (computer security)Sample (statistics)Proxy serverReverse engineeringIdeal (ethics)Query languageStreaming mediaLengthBuffer overflowServer (computing)Group actionAttribute grammarProxy serverDigital electronicsReverse engineeringReading (process)CASE <Informatik>Optical disc driveDependent and independent variablesMedical imagingAbstractionHeegaard splittingConnected spaceOpen sourceError messageImplementationComputer programmingComputer configurationCommunications protocolConfiguration spaceMessage passingFront and back endsMultiplication signSpacetimeDigital rights managementInheritance (object-oriented programming)PlastikkarteCartesian coordinate systemGoogolSoftware testingKey (cryptography)Case moddingWeb pageFlow separationINTEGRALInformation securityRegular expressionGradientNumberDoubling the cubeQuicksortResultantFinite impulse responseExploit (computer security)Data transmissionLine (geometry)MereologyUsabilityContent (media)Cache (computing)1 (number)Streaming mediaWeb browserReal numberProjective planeHeat transferDemo (music)Uniform resource locatorLibrary (computing)NP-hardRobotRevision controlSocket-SchnittstelleBitQuery languageAsynchronous Transfer ModeEmailCodeWritingComputer animation
Transcript: English(auto-generated)
00:00
So we are all set. Let's learn about HTTP. Let's give our uh rather nervous speaker a uh big round of applause and make him feel comfortable. Okay so we should add some demo but maybe it won't work. Uh no demo. Mic. Like this. Okay. So it's Sunday morning I hope you are all
00:27
well enough. So this would be about HTTP. So why Wookiees? I hope you are well enough enough. Um Wookiees because this is about smuggling and Wookiees are smugglers. And uh also
00:43
because the Wookiee language is a thing. Uh quite hard to understand. Um easy to misinterpret. So this was the nice thing for HTTP flow. So what are we gonna talk about? Um something about the um HTTP that you need to know. Uh what is HTTP injection?
01:08
Uh some recent some some recent attack vectors. HTTP zero dot nine which is on old versions. Um maybe not the demo. You'll get to trust me. And uh maybe we'll talk about
01:21
the tool I wrote which is called HTTP Wookiee. So who am I? Uh Regileiro because it's French. Um French. Um yes it's not Wookiee but maybe sometimes you won't understand my words. Uh I would I work on the very small French uh free software web company which is
01:43
called Makina Corpus. Uh fifty people. I'm a dev op. I've always been a dev op so it means I'm a sys admin and I'm a developer. Um web security is just a very small part of my job and it's part of my spare time also. And this is important because if I can
02:04
do it on my spare time uh some other people might be doing it uh better than me. So why did I start testing my tools? Uh I work everyday with open source HTTP servers and I
02:21
like these tools a lot. And I used to trust these tools a lot. But I found two very interesting papers. The first one is HTTP host header real world attacks. And this is about uh very old stuff in HTTP which is the absolutuary in location. Um there are some
02:44
tricks with this so you can attack the host header on HTTP. And there are real uh threats with this sort of attacks. Um 2015 for example. And this is about HTTP smuggling. The other one is a study from 10 years ago. 11 years ago which is HTTP
03:06
smuggling. And everything is inside. Most of the things I can tell you today are already inside but it's like everyone forgot it. So what is HTTP smuggling? Uh it's a
03:24
possible level attack. So it's injection and everything always is injection. It's like here we are gonna inject HTTP in HTTP. So we gonna have more requests or more responses than we should. Um to do this we craft low level HTTP messages. By
03:50
definition you cannot do this with a browser on or an HTTP library because um they do not make mistakes. So the goal is to make mistakes. Um when you make mistakes usually
04:08
you get errors but not always. So before we start the first things on HTTP is the very old version of HTTP was one TCP connection for each resource. Like you want a file, you
04:25
want an image, you want a CSS file, you need one new TCP IP connection for each resource. Then you do the same thing. You make your request, you get your response and then you try to close the TCP connection and there are problems when you try to
04:44
close because we are not sure you get the the answer. Um this is a big performance killer of course because you may need a lot of TCP IP connection. So we are going to they may keep alive on HTTP one on one. That is you will reuse the same connection and
05:02
when you get the new response you can make another request. You'll get the new response and you make another request. And you can in fact you can usually your browser will use six TCP IP connection and do this. And unless you you're using HTTP two it's what you are. Um there's also some other things that you can do. Um
05:25
something that most people doesn't know that's pipelines. So instead of waiting for each response before doing the next query you can send every query and wait for the every responses. On the example you can see that I made four queries and I get three
05:46
response only which is allowed. But the important thing is the other of the responses. So if one of the response is huge you get to wait for this response before
06:00
having the next one. So this is called out of line blocking and this is also a performance problem. And this is one of the reason of HTTP two. And the other reason of HTTP two is smuggling because you depend only on the order of the messages. Um
06:23
in HTTP two you've got a real binary multiplexing thing where each response can be associated with the request and it's better. If you get a reverse proxy like an SSL terminator or reverse proxy cache uh the things if you make pipeline to the reverse
06:45
proxy the reverse proxy will talk to the back end without the pipeline. It will make just a simple um HTTP one dot zero or one dot one keep alive connection but you will not use a pipeline when you talk to the back end. The problem is the back end doesn't know
07:05
that there's no pipeline. So if you can make the reverse proxy play something which looks like a pipeline the back end may see several requests and send several response but the reverse proxy doesn't want several responses. This is smuggling. Um
07:29
you've got one example here where I send something which should be only one query but the final um step has a split splitting issue so now you've got two queries and you've
07:44
got one extra response and the problem is on the question mark. What do you do when you've got one extra response? And you've got two type of things which are the transmitter which transmit the bad syntax and you've got the splitter which splits on
08:01
these bad syntax and the two are problem problems but the splitter is the real problem. So why do you do HTTP smuggling? Maybe just because you want to hide a query. So maybe the reverse proxy would like to prevent this sort of query to happen but it
08:24
doesn't see the query. Uh you can make crash you can crash a lot of things when bad syntax are used. You can try to shift the response stream like uh if you insert one extra response on the pipeline you may really shift the re the stream and do cache
08:46
poisoning and you may also try to attack another user credentials on HTTP by using incomplete queries um this is uh the thing on the on the demonstration. All these things were
09:01
described in 2005 already. So we also need exploits. Oh it's not me it's the Mac it's my
09:36
day. So the exploits it's all about size because we are in the pipeline. We have got we've
10:02
got several messages in the pipeline and each one has its own size so the goal is to alter alter the size. You can make double content length errors so people doesn't know which one is the right one and this is strictly forbidden. You can play with content length
10:20
and chunked transmission and you should not play with both. Um on chunked transmission you do not have a length for your message you say I will send things and when you see the end of chunk marker it's the end of the message. Uh you could use invalid errors or invalid values like the contents length with a space is not a valid error. Uh you could
10:46
use also invalid end of lines like the first one the real end of line is CRLF you could try to use LF also it's almost valid but CR is not valid as an end of line. Um you could
11:03
also try to use very old features of HTTP which are still there like HTTP 0 and 9 like multiline errors which are in the RFC. So the first demo um at least you can read the
11:23
thing um here this should make an hijacking credential exploit. So here you've got some real world uh exploitation like this was working on node before the version 5.6 it's a
11:41
splitting issue um you should always have CRLF for the for end of line but for node if you have the CR they thought that they do not read need to read the next character because it's always LF but if it's not you've got problems. So on on the example
12:05
you've got the dummy header and then you've got the CR on a Z character which is not an LF on any other thing so for everybody this is just one header which is dummy header CR Z transfer encoding chunked it means nothing but for node there are two headers. The
12:27
first one is dummy header which means nothing and then there's a transfer encoding error. So for everybody this request is one request and in blue you've got the body of the request and you can make a get request with the body it's allowed. So this is just one
12:46
request with a binary body that nobody reads but for node you've got transfer encoding chunked um there's a pri priority and chunked always has the priority on content
13:00
length so it doesn't it does not read content length it reads chunked and the first bytes of the body is the end of chunked marker so after the end of chunked marker you've got a new request in the pipeline. You've got a post request and this post request is
13:24
please direct user 2. Um there's another thing on on the second request there's a partial there's a partial header at the end so um it's an untaminated request so
13:40
there's a post request. Node is still waiting for the end of this request so what could happen if you've got a reverse proxy in front of node with a keep alive connection with node um the first proxy get the first response like you think is thinking this is one
14:07
query I need one response so you get one response then node is just ring waiting for second query and another user like the admin user is there sends a a query any query and
14:22
the query gets appended to the in untaminated one with the new user credentials like the cookie like and actually be authentication credential and then the admin user is doing a post user direct 2 without requesting it so uh I'm not sure if you guys are
14:49
I cannot show you because there are problems with the screens but at least I can show you uh what happens um I'm the attacker uh I send a request to vanish which is a reverse
15:03
proxy cache and this request is the request you just saw like you've got in fact 2 requests but it's just 1 and the request goes to node, node has a splitting issue so it's in no things there are 2 requests and it sends back 1 response then another user
15:26
came for uh new request which is request C vanish reuse a keep alive connection with a backend so send the request C which is appended to the untaminated request and then it
15:44
responds B which was my hidden request um the user 2 is deleted uh only if you do not have um CSRF protection on the post request so this is a way to run this attack it's just a
16:04
request I run it uh on Linux with a printf and netcat just to put the contents in a TCP IP connection and I do it uh 150 times just to feed every vanish um TCP IP
16:21
connection uh in the pool of backend connection with uh node and it works believe me you can try you've got everything on the CD if you want to try it it worked um before it was fixed on node um there was a second demo um on this one I'm not sure if you
16:45
remember the first one is about HTTP 0.9 so HTTP 0.9 is something awful which should not exist uh it was the very very first version of HTTP where you do not have any errors
17:02
so you've got an example of what is HTTP 0.9 1.0 and 1.1 on 0.9 you do not have the protocol version it's just the method on the location and no errors no errors it means there's no errors on the response but the security of HTTP is in the errors like the
17:24
content type the cookies the content security policies the content length everything is on the errors on HTTP 0.9 you've got just the body of the answer so it's just text stream um if you've got text stream maybe you can make this text stream looks like a real
17:44
HTTP response with errors so for example you could make an image and the content of the image instead of being a binary thing would be uh an HTTP response with errors and if
18:00
you request this image in 0.9 mode it may looks like an HTTP response you could also try to hide uh these parts this HTTP message in in the exit data of the image and make a wrong query to get only this part of the image so you get a real image and if you
18:23
request through the the right part of the image you've got an HTTP response but you should not be able to do a wrong request in HTTP 0.9 mode so I never figured out how another problem uh is the no cache poisoning like maybe you know about cache poisoning in
18:45
HTTP but for cache poisoning you need a cache no cache poisoning is in fact socket poisoning like TCP IP sockets and you hide response on the sockets and another user when
19:01
the socket is reused get the response so this is a real thing on Apache for example so this was the thing we could try like here we've got uh a splitting issue on Go uh where Go
19:23
is fixing your um your syntax like transfer encoding with a space is fixed as transfer dash encoding uh so for everyone uh the line transfer encoding chunked means nothing but for Go it's a real line so same as before the blue part uh is no new request um there's
19:50
the no cache poisoning will use it uh there's another bug on Go which is that we can ask for HTTP 0.9 which does not exist and this makes a 0.9 query it should not work this way and
20:06
there's a third bug on Go where we can make a wrong query in 0.9 mode it should not exist because in 0.9 mode we should not read the request errors so you make a request uh with a
20:23
splitting issue on the Go long reverse proxy for example here I've got to run apache reverse proxy which transmit the request to Go I've got a splitting issue on Go so Go is doing this request uh against Nginx it targets an image and takes only the HTTP part of
20:52
this image which gets back in HTTP 1.1 mode but for Go we need to make a request for go it's an HTTP 0.9 request so it removes the errors of the response and sends you back
21:03
the real thing uh it's hard to understand without the demonstration but it works it means I can um request with the Go an HTTP response which is hidden in an image on a server um inject this response on the streams of response but there's another
21:25
problem for apache if we get back there was only one request and it gets one response then there's a new response which scans back which was the response hidden in the image and
21:40
for apache there's nothing to do with this response so nothing's happened and the response is stored on the TCP IP connection then when a new request is coming for apache from anybody apache reuse the same TCP IP connection with the back end send a new
22:03
request and read on the TCP IP connection to see maybe I I have a response inside and you see that he has a response and takes it and send it back to the user just before getting the real response which was hey no you should not reuse this connection this is
22:22
an RST on TCP IP because there was a 0.9 response so you should not reuse the connection but it's too late the the the message was stored and was sent back to the user so you can store a response in apache and get this response with maybe an XSS or
22:45
maybe everything you want like any error you want any security policy error and sending back to every user so this is the way to do it if you want you need to run a lot of requests because you need to feed every TCP IP connection and I cannot show show it to you
23:08
but it works what can I say about it the real problem on these things are the splitting issues and if we get back just a couple of minutes and then we can
23:24
just return here this is the real problem on Go and it's fixed now as soon as you have a splitting issue bad things happens like having an extra response sends back and for
23:41
apache the fact that you've got the no cache poisoning is a public issue it's not a security issue because they consider that there was a security issue for Go there was a splitting issue and what happens after that is not their problem it's a little bit of
24:00
problem but it's complex to manage if you think that the the back end is doing bad things you cannot do anything so for every splitting issue you should get a CVE because for me it's quite critical but usually project leaders do not like doing CVE on these
24:26
things and maybe sometimes they do not understand how you can exploit the injection and because also they do not want people to try to do things with those issues the
24:42
other issues are transmission of the strong syntax like yeah I got something which is quite very strong syntax like transfer encoding with a space you should not have any header with a space in the header title and a reverse proxy could detect that this header is invalid and could reject your query but usually they do not do this so there's a big
25:11
problem of responsibility because everyone is trusting every other one and no one wants to take the full responsibility of the problems on things which looks like smaller
25:24
things for security researcher a warning you will not earn money with a HTTP smuggling because you cannot test it like an XSS thing you cannot take a big infrastructure and try
25:43
to break everything because you won't be the only one impact impacted on this you may crash everything you may send back response through people that have nothing to do with you so it's very hard to test on public servers I earned one buntys with a go-long
26:07
issue from Google but it was unexpected and it's very hard to usually explain to people that maybe they should not get their servers the other thing is that we should have more
26:23
people reading the code of HTTP servers like there are a lot of issues which are still there and we need people which really do not trust blindly the code things get better
26:41
years after years the issue gets fixed and you should really try to upgrade to avoid problems some other exploits maybe because we cannot see demos so we can take time on exploits I had some exploits on nginx like their integrals overflows this cannot be
27:06
used with nginx as a reverse proxy but if you've got nginx as a backend you can try strange things with very long numbers this was fixed last year there's another issue which is fixed only trying currently which is a 0.9 downgrade using HTTP 65536.9 or
27:36
nginx so you get no errors in the response this is usually not transmitted by your reverse
27:47
proxy to nginx like to exploit these sort of things you need to get nginx as a backend and you need a reverse proxy which transmits this bad syntax and this usually doesn't happen but in the past HAProxy was transmitting these to nginx and HAProxy is one of
28:08
the best HTTP tools against smuggling um another one here it's a severe against vanish uh we are there's here enough line was valid and you could also use double content line
28:24
failures uh there's another one but I can't remember what what what I can't read it and over one on Apache like here it's on the chunk size attribute so when you do chunk transmission you've got chunks with a size um there was a truncation like only the 31
28:48
first character I read so you can use zero character and try to alter the size of the chunks this is now fixed of course um there are all the other exploits I showed
29:03
previously in Go in Apache like the no cache poisoning in node and there are no there's also a verse people other peoples we've fixed which fix overflows like we had one in uh Python URL lib recently and uh on node there were overs so how do you
29:28
protect against this the first thing is to use the last RFC which is uh 7230 and a lot of people are still using the very old RFC for HTTP one on one which is
29:41
already very very old you should really try to reread the RFC because on this new version there are a lot of things against smuggling like they said you should try to avoid uh chunked and content length you should really avoid rewriting your own
30:05
reverse proxy because it's very hard stuff but in case of if one day you try to write a reverse proxy the goal of the reverse proxy is to rewrite all errors in a very clean way do not take blindly the errors I sent and give it to a backend uh you should
30:25
try to read books on TCP IP socket connection because it looks like something like an abstraction which is simple but in fact it's really not simple you should also think about browser and browser do not make HTTP errors uh a lot of HTTP server are still
30:48
allowing a lot of things like HTTP 0.9 because of uh monitoring tools which use 0.9 or because of bots or because of bad implementation on the wide but I think you should
31:02
reject bad implementation and you should restrict uh uh what you allow in HTTP uh you should be intelligent and this is one of the problem there is this sentence uh which was in general an implementation should be conservative in its sending behavior on liberal in its
31:22
receiving behavior but maybe it's wrong maybe the the right thing is not lib it's not being liberal it's being robust like you should think that you may get some bad things and you should handle this thing maybe you need a special error message maybe but do not fix
31:43
the things bli- blindly and there's a new RFC uh which is called draft Thompson postal was wrong which is talking about this thing we should now be interior on on the particular implementation I think also that you should be able to get more option on on
32:04
HTTP server configuration like you should say I do not want to support HTTP 0.9 because even today you may get an HTTP 2 server uh in go for example you've got very nice HTTP 2
32:21
server but there's still the HTTP 0.9 support inside and you should also be able to get the keep alive thing without the pipelining so there's no option to say there's no pipelining on this server you could also try to get back to HTTP 1.0 but you'd get some
32:45
problems of performance um you may also think I was thinking this that adding a reverse proxy will secure your installation and maybe in fact it's the contrary every time you
33:02
add another layer maybe you add another issue the reverse proxy trusts the backend response and it's the way it works and we never think it works this way it means if you've got a reverse proxy and you've got a backend uh with a PHP application and you
33:22
let everyone write PHP application everyone can break your reverse proxy simply by sending several response several responses so the reverse proxy will always trust the things from the backend and we do not think the this is the case um think about things like
33:48
Azure proxy uh because it's a very good tool to to remove everything which is strange. Engine inks as a reverse proxy is also quite nice uh the next APHC mod proxy will
34:01
have the strict protocol option which is nice. Is HTTPS protection no it's just a layer so it's not a full protection because inside after the SSL thing you've got just HTTP 1.1 um and maybe when you add an SSL I mean I thought you just added 1.1 and
34:23
never uh HTTP server but HTTPS is great of course. Is HTTP 2 protection yes of course it's better against smuggling but you can always ask HTTP 1.1 for an HTTP 2 server it's
34:42
just another transport layer. Just the last thing I wrote a tool which is called HTTP Wookie uh it's an open source tool I will release it uh today or maybe tomorrow uh with uh
35:03
tests the goal of this thing is to test uh reverse proxy implementations and to fuzz HTTP I need to remove some of the tests which are not fixed yet everywhere but uh if you need to fuzz your own server you could use this tool. Uh if you got any questions I will
35:27
be outside uh and that's it and thanks for for everyone.