Smartfuzzing The Web: Carpe Vestra Foramina

Video thumbnail (Frame 0) Video thumbnail (Frame 4770) Video thumbnail (Frame 6657) Video thumbnail (Frame 8717) Video thumbnail (Frame 17860) Video thumbnail (Frame 19008) Video thumbnail (Frame 20536) Video thumbnail (Frame 22182) Video thumbnail (Frame 24278) Video thumbnail (Frame 26612) Video thumbnail (Frame 28047) Video thumbnail (Frame 32161) Video thumbnail (Frame 36145) Video thumbnail (Frame 37469) Video thumbnail (Frame 38571) Video thumbnail (Frame 39515) Video thumbnail (Frame 42915) Video thumbnail (Frame 45149) Video thumbnail (Frame 47927) Video thumbnail (Frame 51777) Video thumbnail (Frame 53087) Video thumbnail (Frame 54029) Video thumbnail (Frame 55799) Video thumbnail (Frame 56739) Video thumbnail (Frame 57781) Video thumbnail (Frame 61001) Video thumbnail (Frame 61951) Video thumbnail (Frame 63292) Video thumbnail (Frame 67071) Video thumbnail (Frame 68774)
Video in TIB AV-Portal: Smartfuzzing The Web: Carpe Vestra Foramina

Formal Metadata

Title
Smartfuzzing The Web: Carpe Vestra Foramina
Title of Series
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2013
Language
English

Content Metadata

Subject Area
Abstract
It can be scary to think about how little of the modern attack surface many tools cover. There is no one best tool for the job and on top of that some tools don't do a great job at anything. Often in the hands of general users the capabilities and limitations are not even thought of during testing. Point, click, done. The attack surface of modern web environments as well as their protection mechanisms have become more complicated and yet many tools have not adapted. Hey, Y2K called and it wants some applications tested. There is certainly no shortage of vulnerabilities in modern web environments but we should be looking beyond low hanging fruit at this point. In between fully automated scanners and manual testing lies a sweet spot for the identification of vulnerabilities. Some of the juiciest pieces of information are not found by vulnerability scanners but are found by humans creating custom tests. This is why semi-automated testing space is so important. All of this complicated blending of protection mechanisms, services, and RIA technologies means that moving in to the area of semi-automated testing can be fraught with failure. We detail how these failures can be avoided as well as provide a tool that solves some of these problems as well as provides analysis for your own tools and scripts. Your web applications have moved on, don't you think it's time your tools to do the same? Nathan Hamiel is a Principal Consultant for FishNet Security's Application Security Practice. He is also an Associate Professor of Software Engineering at the University of Advancing Technology. He spends most of his time focusing in the areas of application, Web 2.0, and enterprise security. Nathan has been a speaker at security events around the world including: Black Hat, DefCon, ShmooCon, ToorCon, SecTor, OWASP and many others. He is also a developer of several open source security projects including the pywebfuzz and RAFT. Gregory Fleischer is a Senior Security Consultant in the Application Security practice at FishNet Security. In his spare time, he likes to find and exploit vulnerabilities in web browsers and client-side technologies such as Java and Flash. He has an interest in privacy and anonymity and has worked with The Tor Project to identify potential issues. Justin Engler is a Security Consultant for FishNet Security's Application Security practice. His focus is on the security of web applications, web-backed thick clients (desktop and mobile), databases, and industrial control systems. Justin is currently working on the open source RAFT project. Seth Law Seth Law is a Principal Consultant for FishNet Security in Application Security. He spends the majority of his time breaking web and mobile applications, but has been known to code when the need arises. Seth is currently involved in multiple open source projects, including RAFT. Twitter: @sethlaw
Ocean current Principal ideal Slide rule Mobile app Context awareness Multiplication sign Regular graph Focus (optics) Perspective (visual) Information technology consulting Statistical hypothesis testing Software bug Coefficient of determination Software Software testing Lie group Information security Traffic reporting Associative property Vulnerability (computing) Area Adventure game Focus (optics) Software engineering Software developer Bit Cartesian coordinate system Statistical hypothesis testing Web application Universe (mathematics) Quicksort Information security Electric current
Slide rule State of matter Token ring Direction (geometry) Multiplication sign 1 (number) Set (mathematics) Port scanner Statistical hypothesis testing Latent heat Web service Different (Kate Ryan album) Term (mathematics) Internetworking Software testing Vulnerability (computing) Enterprise architecture Service (economics) Bit Special unitary group Cartesian coordinate system Statistical hypothesis testing Web application Personal digital assistant Website Right angle Figurate number
Scripting language Multiplication sign File format Port scanner Web 2.0 Sign (mathematics) Different (Kate Ryan album) Single-precision floating-point format Personal digital assistant Abstraction Information security Vulnerability (computing) Scripting language Injektivität Email File format Software developer Open source Data storage device Electronic mailing list Bit Regulärer Ausdruck <Textverarbeitung> Flow separation Statistical hypothesis testing Demoscene Type theory Web application Process (computing) Order (biology) Right angle Pattern language Authorization Quicksort Resultant Electric current Spacetime Web page Mobile app Sequel Flash memory Mathematical analysis Data storage device Web browser Statistical hypothesis testing Latent heat Computer programming Authorization Energy level Software testing Authentication Dependent and independent variables Mathematical analysis Interactive television Counting Client (computing) Basis <Mathematik> Cartesian coordinate system Uniform resource locator Word Object (grammar)
Server (computing) Information Multiplication sign Weight Blind spot (vehicle) Electronic mailing list Directory service Electronic mailing list Directory service Client (computing) Mereology Statistical hypothesis testing Application service provider Web 2.0 Word Word Search engine (computing) Website Website Mathematical optimization
Server (computing) Computer file Code Multiplication sign Connectivity (graph theory) Directory service Electronic mailing list Client (computing) Mereology Event horizon Web 2.0 Word Exclusive or Robotics Set (mathematics) Website Weight Computer file Feedback Electronic mailing list Directory service Application service provider Web application Word Googol Website Resultant
Injektivität Web page Sequel Token ring Multiplication sign Electronic mailing list Instance (computer science) Plastikkarte Cartesian coordinate system Statistical hypothesis testing Goodness of fit Mathematics Different (Kate Ryan album) System identification Software testing Quicksort Error message Vulnerability (computing)
Building Scripting language Parsing Multiplication sign View (database) Demo (music) Web 2.0 Data management Mathematics Different (Kate Ryan album) Information security Scripting language Programming paradigm Touchscreen Building Software developer Open source Bit Funktionalanalysis Instance (computer science) Sequence Statistical hypothesis testing Web application Data management Interface (computing) Volumenvisualisierung Website Quicksort Slide rule Proxy server Open source Sequel Dependent and independent variables Connectivity (graph theory) Mathematical analysis Heat transfer Web browser Plastikkarte Focus (optics) Event horizon Statistical hypothesis testing Sequence Software testing Proxy server Traffic reporting Dependent and independent variables Focus (optics) Demo (music) Interface (computing) Mathematical analysis Cartesian coordinate system Web browser Component-based software engineering Graphical user interface Loop (music) Personal digital assistant Routing
Application service provider Wechselseitige Information State of matter Weight Multiplication sign View (database) Zoom lens Web 2.0 Medical imaging Different (Kate Ryan album) Scripting language Touchscreen File format Electronic mailing list Range (statistics) Bit Message passing Emulator Mixture model Artistic rendering Volumenvisualisierung Arrow of time Website Web page Computer file Open source Link (knot theory) Image resolution Motion capture Web browser Plastikkarte Login Coprocessor Internet forum Internetworking Green's function Software testing Configuration space Form (programming) Default (computer science) User interface Dependent and independent variables Interface (computing) Limit (category theory) Cartesian coordinate system CAN bus Uniform resource locator Software Normed vector space Document Type Definition Window Library (computing)
Web page Execution unit View (database) Consistency Closed set Instance (computer science) Web browser Term (mathematics) Proof theory Web application Lie group Website Gamma function
Wechselseitige Information Intel Sequel Variety (linguistics) View (database) MIDI Coroutine Value-added network Front and back ends Googol Information Data type Scalable Coherent Interface Execution unit Software bug Clique-width Data storage device Range (statistics) Menu (computing) Database Greatest element Repeating decimal Inclusion map Type theory Moment of inertia Search engine (computing) Royal Navy Information security Hydraulic jump Local ring
Web page Mobile app Matching (graph theory) Programmable read-only memory Demo (music) Menu (computing) Port scanner Word Differenz <Mathematik> Different (Kate Ryan album) Motion blur Information Information security
Web crawler Dynamical system View (database) Flash memory Virtual machine Web browser Client (computing) Login Event horizon Web 2.0 Revision control Hooking Different (Kate Ryan album) Software testing Office suite Codierung <Programmierung> Form (programming) Task (computing) Authentication Covering space Addition Interface (computing) Data storage device Coma Berenices Bit Basis <Mathematik> Cartesian coordinate system Sequence Cross-site scripting Web application Uniform resource locator Website HTTP cookie Quicksort Local ring
State transition system Slide rule Code Multiplication sign Maxima and minima Online help Mathematical analysis Information technology consulting Statistical hypothesis testing Software bug Information security Traffic reporting Software developer Open source Electronic mailing list Mathematical analysis Funktionalanalysis Cartesian coordinate system Backtracking Web application Googol Software Computing platform Right angle Window
Web page Server (computing) Implementation Service (economics) Dependent and independent variables State of matter Multiplication sign System administrator Computer-generated imagery Sheaf (mathematics) 1 (number) Port scanner Mathematical analysis Web browser Number Medical imaging Single-precision floating-point format String (computer science) Modul <Datentyp> Cuboid Electronic visual display Circle Endliche Modelltheorie Error message Hydraulic jump Formal grammar Dependent and independent variables Information Demo (music) Software developer Mathematical analysis Bilderkennung Line (geometry) Cartesian coordinate system Regulärer Ausdruck <Textverarbeitung> Social engineering (security) Type theory Personal digital assistant Configuration space Fuzzy logic Convex hull Right angle Cycle (graph theory) HTTP cookie Flux Resultant
Demo (music) Connectivity (graph theory) Demo (music) Plastikkarte Correlation and dependence Bit Web browser Plastikkarte Markup language Sequence Web browser Statistical hypothesis testing Sequence Component-based software engineering Type theory Cuboid Computer worm Software testing Object (grammar) Fuzzy logic
Mobile app Application service provider Decision tree learning Touchscreen Demo (music) Open source Mapping Weight Interface (computing) Demo (music) Electronic mailing list Set (mathematics) Web browser Directory service Focus (optics) Statistical hypothesis testing Template (C++) Web application Uniform resource locator Personal digital assistant Cuboid output Resultant Computer worm
Web page Point (geometry) Intel Raw image format Application service provider Dependent and independent variables Weight View (database) Web page Length Maxima and minima Letterpress printing Color management Demoscene Statistical hypothesis testing Number 9K33 Osa HTTP cookie Resultant
Laptop Execution unit Dynamical system Token ring Weight Token ring MIDI Event horizon Sequence Statistical hypothesis testing Sequence Insertion loss Aerodynamics Computer worm
Default (computer science) Dependent and independent variables Dynamical system Freeware Demo (music) Local area network State of matter Maxima and minima Web browser Shape (magazine) Cartesian coordinate system Login Sequence Process (computing) Software Intrusion detection system Hypermedia Configuration space Computer worm Hill differential equation Pattern language Quicksort HTTP cookie Form (programming)
Web page Application service provider Intel Identifiability Image resolution Computer-generated imagery Statistical hypothesis testing Web 2.0 Sic 9K33 Osa HTTP cookie Execution unit Dependent and independent variables Demo (music) Venn diagram Menu (computing) Mass Bit Ripping Cartesian coordinate system Cross-site scripting Uniform resource locator Moment of inertia Software repository Website Simulation
Web page Scripting language Email Uniqueness quantification Multiplication sign Maxima and minima Similarity (geometry) Statistical hypothesis testing Statement (computer science) Fuzzy logic Pattern language Escape character Scalable Coherent Interface
Dialect Presentation of a group Multiplication sign Web browser Mereology Rule of inference Software bug Number Pattern matching Hooking Cuboid Matching (graph theory) Demo (music) Content (media) Data storage device Instance (computer science) Cross-site scripting Uniform resource locator Software Personal digital assistant Website Convex hull Single sign-on Quicksort Arithmetic progression Local ring Resultant
Web page Slide rule Execution unit Sequel Building Multiplication sign Software developer Data storage device Data storage device Cartesian coordinate system Mereology Cross-site scripting Web 2.0 Cuboid Quicksort Codierung <Programmierung>
Web page Slide rule Random number Proxy server Code Multiplication sign Web page Disintegration MIDI Code Mathematical analysis Port scanner Group action Statistical hypothesis testing Software bug Component-based software engineering Error message Function (mathematics) Lie group Interface (computing) Right angle Traffic reporting Proxy server Traffic reporting
so a little bit about us we work for fishnet security but we're not salespeople so you really don't have to worry about that portion of it I'm a principal consultant on the app sec team I'm also am an associate professor of software engineering at the university of advancing technology so they have a booth over there in the vendor area if you feel like talking to them and even though we didn't change the slides that's actually j-rok not just an angler we went through our entire team and gave each other hip-hop names so that's j-roc okay na dog known eat dog I'm Seth i'm also a principal consultant fishnet i'm greg i'm a senior security consultant fishing as well i'm justin i am a regular consultant at fishing at security so kind of what our talk is about here is we're going to provide a little bit of an overview of problems with current testing tools that that people may or may not be aware of so kind of our modern landscape is that we end up with people who may be in a QA or security testing role who may have come from a different background like they may not have had development experience so when they run into problems or when they run into technologies they may or may not understand they might not find vulnerabilities that are fairly easy to find and some of that is a problem on the tools perspective like not being able to handle modern web applications so we'll go through some of the current workarounds and how that you know how people are handling those will go through a little bit of proposed solutions like how those can be fixed and then we wrote a tool to start addressing some of these issues so what will go into we'll skip some of the stuff and get right to the demos and show you the tool so what we aren't going to do which this is kind of a lie so we aren't going to beat up on a particular vendor so that's kind of sort of not true but we're going to we're going to try we didn't say tools we've said vendor yeah it's adventure yes we also currently can't solve every single problem that we outline but we're working on it and we're definitely not going to sell you a solution so our goals for this is one to raise awareness for people who actually test applications we want to put focus back on the tester and not so much on the tool and that's what our tool allows you to do I know that sounds kind of weird like we're giving you a tool so you don't have to use tools so it sounds kind of strange but you'll get it by the end of the talk I promise if not you can punch me in the face afterwards and also to get you to submit bug reports for raft I remember a couple times when some tools came out I don't want you guys to do what I did so I would download the tool and give it a try and then I think something wouldn't work and I'm like this sucks i'm not going to look at it anymore so don't do what I've done in the past I realize them you know not being a good advocate of that but we will fix something and you know we are we will take feature requests and try to work things into the tool so a little
bit of clarification throughout the talk we use terms fully automated and semi-automated and sometimes we use those interchangeably so that causes some confusion and we're going to continue to be confusing about that so sorry so this slide is going to try to explain what we mean so if you think of a fully automated testing tool like your enterprise application testing tools those are like the mac-10 on the right hand side right so basically what you're doing is you're loading up a bunch of bullets and you're just spraying them in a current or anybody who's ever shot a mac-10 knows that you can't hit anything with it they might as well not even have any sights on it you just pointed in the general direction and you put holes and stuff not always the best solution so semi automated testing or how it should be which is what you'd think of sending sending data to something like running through a bunch of different test cases that's more like the the semi-automatic sniper rifle so you're honing in on a problem and you're really trying to focus on that problem and find vulnerabilities based on a specific set of test cases and that's mostly what we're talking about during this during our this talk is we're talking about the left-hand side we're talking about the sniper rifle okay so I'm going to kind of talk
a little bit about the current solutions that exist out there I'm not sure how well you guys can hear me basically we test for a living right we're looking at web applications and we've kind of figured out that they all fall down in some way or another I mean you get a you know the the fully automated tools when you click the start button and it's supposed to find every vulnerability under the Sun my end up spending two to three days configuring the thing and it comes back and it tells me there's you know SSL is in configured misconfigured or some like that so the automated tools fall down I mean there's the fully automated ones the semi automated ones they have session and state problems you've got scanners that will run and they you know pull what you did and the next time that you go to the site and have the tool run it's out of state and they can't figure out that it's out of state so you've got hundreds and thousands of requests they're coming back in the tool or that the tool is making to the website and did you start valid because they're all returning you know 302 redirects or some something like that they have problems we thought you know these complicated applications that modern technologies we already talked about you know see surf tokens are rich Internet applications web services the tools just don't understand them very well furthermore all this data
that is collected is in disparate locations right you've got your your proxy that you're using while you're testing you've got the full-blown Afghan or web inspector sorry I probably wasn't supposed to mention that but the fully commercial scanners I mean getting data out of those tools can be just one huge pain in the ass as in as you go further you've got all this this data that you've collected that there's no analysis that's run on it after the fact right you've got a single request response the scanner goes in it makes its assumptions about what's happened and then it basically discards that data there should be some sort of analysis that goes on after the fact as testers we need more interaction not abstraction right we need to be able to understand the application in order to break the application or in order to find the vulnerabilities and if it's not I mean if the tool is basically the point-and-click tool you don't understand what it's doing behind the scenes all the vulnerabilities that I find typically are because I'm in the application I'm actually looking at the request of the responses on a low low level not at the level that is being presented to me by the tool itself furthermore we miss portions of the application if you think about the mobile applications that exists in the space that's out there now when your iPhone makes a request you get a different application than you do when you're using your Firefox web browser if the tool doesn't understand that hey it needs to fuzz the accept header or the user agent header to actually get into portions of the application you're going to miss maybe fifty percent of the application is developed by the developers and there's some risk there's we could go on forever if you if you really want to know about the problems we really beat up within the white paper that we present it can anybody tell like anybody who tests web applications so say you have an automated testing tool can anybody see from the screenshot why it might have a problem there you go it says sign out a lot of a lot of automated testing tools look for a regular expression to tell whether it's in-state or out-of-state so the application is clearly asking the user to authenticate yet it says sign out like they're already logged in so a lot of tools will continue to send their their their tests and fail based on that then you've got things like this risk-based login that we're talking about the beginning you know financial applications depending on where you're coming from if it's a new browser that hasn't seen before they're going to ask you for more layers of authentication and you know the first time you step through it with your tool it may you know ask two or three different questions and it would be different the next time that you hit the application so these tools are just you know they're basically killing us when it comes to application testing their simple features we're missing request times authorization checks storage locations the new html5 spec flash objects things like that especially these tools that were built in you know 2001-2002 they don't understand any of that new technology so now that we've talked about why the existing tools can't do a good job on the whole picture we're left with trying to figure out at the end of the day when I have to do an assessment what am I going to do so even though a lot of tools don't handle the whole picture there's some that can handle pieces so we run a bunch of separate tools that do little pieces and a lot of them don't have any analysis of their own sometimes will write our own custom scripts to to do something custom to generate a whole bunch of requests but then we don't have any way to analyze what we just did and not only do we not have the way to analyze just the one thing from one tool we've got all this stuff from all these different data formats and we don't have any way to get them all in one spot and then look for commonalities another problem with doing it this way is that most of these tools even when they do have analysis you can only do it on the stuff you just ran if you've got data from a bunch of tools from last year and now some new type of vulnerability came out and you want to check hey do I still do I have that in any of my stuff you're going to have to scan everything again you can't just take the results that you had and and run the analysis so instead you could try testing manually but when you look at the scope of the assessments at least that we get there's not really any chance that you would be able to get anything meaningful done by manually clicking things and manually looking at the responses we've got you know thousands of pages to look at in the course at two weeks you just you're not going to get it done you need to have something that helps you reduce that burden so just manually doing it isn't going to work if you've got a crazy tool that almost does what you need sometimes you can modify something to do what it wasn't supposed to do but even that can be painful and you're spending time writing scripts when you should be spending time testing the program that you're supposed to test as anyone in here ever had to use like windmill or selenium to do a security test anybody called poopy pants ok so that's kind of what we're talking about like you know windmill and selenium are more or less QA tools they're not really made to find security vulnerabilities so you might be looking for something specific like let's say you modified and you had some selenium scripts and you were looking for sequel injection well you're kind of focused on sequel injection but you might miss a whole slew of vulnerabilities and other data in the same request that could be easily found if there was proper analysis done so the other problem is many tools were fine when they were first written so that you could you know present at defcon a couple years ago but they don't they've never really been kept up to date or they don't adapt and so just like our picture you need to stay up to date with the times or you will become useless I would argue I don't think she's ever been useful but it was just a funny picture so anybody here use Nick
to on a regular basis for doing web application testing raise your hand please you can get some kind of count okay and how about der Buster a couple more hands all right okay so I'd say that was maybe ten percent so if people haven't figured it out yet Nick to is just a piece of crap if you actually look at what it is it's just a list of web request URLs that get sent and it has some pattern matching it comes back there's no intelligence and in at all I mean about the only thing it's valuable for is testing a very broken laughs and a derp Buster we're going to talk a little bit more about dura buster so those word lists that you
guys are using in your door buster tests if you're using dirt Buster or if you're importing them into another tool those haven't been updated since 2007 and if people haven't noticed the web has moved on since then and there's a lot of common words that we see all the time that aren't in those lists so there's a couple reasons for this when those lists were first generated they were generated by going out to websites and spy during the web site seeing what directories existed and pulling words down well if you think about that you're interested in parts of the website that don't exist not the parts that do so if you're just depending on values that come back you're going to be missing all the stuff that you think you should find but aren't and there's a lot of bad words in there so you're wasting time because search engine optimizers do keyword stuffing so you're going to end up with all these strange words that are completely useless so here are some
common words that you know we see in our assessments on web servers that aren't in the small and medium list I mean asp.net asp net client that's pretty important you know the vti directories there's good information that can be pulled from that so that's just something to be aware of i mean these are common things that are missing so if you're depending on these tools and not actually looking at what they're doing you have these big blind spots and this
is kind of you know we were reviewing the dirt Buster lists and we're like what what what the hell is this Jeremiah Grossman I mean I'm sure people know Jeremiah either through his reputation or at events like this hey you know when was the last time you found your gross the Jeremiah Grossman directory on your web server really you know but that doesn't make any sense why are you he just stopped by to say hello yeah but the thing I love is that that asp net under our client isn't in there but Jeremiah Grossman is yeah yeah it just it's it's like really does nobody look at this stuff so that let us say well we
need to generate our own word lists you know how do you approach this problem and we said well let's think about some way that we could go out and find words that people are telling us not to look for and if you're familiar with the robots.txt exclusion standard basically webmasters go through and they mark parts of the site that they don't want Google spidering you know maybe there's sensitive data there or maybe its underlying web application components so those are kind of the kind of things that when we're doing an assessment we're very interested in so we went through and we pulled down the robots.txt file from a lot of sites we gently combine the Alexa and a quantcast top million site and pulled down about 1.7 million we made about 1.7 million requests and found 350,000 unique files and we went through when we generated word lists based on how prevalent certain words were so it's kind of like we crowd-sourced what people are telling us not to look for so we've been using those on our assessments and we're seeing that they were getting better results back than one depending on the dirt Buster list so those are out in our svn on google code you can pull them down right now they're just in a 7 zip file so pull them down look at them you know if you think they suck there's probably some stuff in there that doesn't make any sense you know let us know give us feedback
so one of the things that I was like to say is that tools tools don't find vulnerabilities like people do so tools should be there to assist two assists in identification of vulnerabilities not exactly point them out so if you have a tool that's telling you something as vulnerable the tester has have the knowledge to eat to look at that data and say yep that's that's an actual vulnerability or it's not a vulnerability so we decided that there's
too many tools out there with absolutely no intelligence when it comes to the sort of fuzzing or fault injection testing of applications so if you if you had a like say you were testing for sequel injection you have a good sequel injection list that you want to test with different values with modern applications a lot of times those fall down so if you've right clicked on your favorite tool and said send to insert semi automated testing tool here if there's a si sirve token that changes every single time the page is laid out that means every single one of your test is going to fail it's going to come back and say you know yup not vulnerable or depending on the error message it may come back you might be able to identify that but that's a big problem so because often you can use you know automated tool automated tool and it might not find an instance of sequel injection that was very easy to find if a person would have tried to test manually but they're using these semi automated testing tools and it's failing and they're assuming that they can move on to other tests so a smart tool a smart
semi automated testing tools have several components it should have session management obviously because we want to make sure that our tool is smart enough to stay in session it should have sequence building and running so if you have a difficult test case you might need to run a sequence of events prior to sending your test case and even a sequence of events after you've ran the test case so for instance you may need to run a sequence of events to log you in run a test case and then log out there are crazy weird applications like that a lot of international money transfer applications have weird functionality like that to try to make them more difficult i guess so security through obscurity but any tool that can handle those for things can test them rather easily also content discovering and support from modern technologies so we all know that something new comes out developers want to use it it always takes testing tools time to catch up oh so so here's our
tool we're going to start talking about how we solve these problems in how you can use our tool to help you solve some of the same problems so a little bit of history about raft it was written it stands for response analysis and further testing which is actually on the next slide I should have said that but you're probably wondering why there's a big red raft in the center of the screen right so this tool was created because I was on an engagement and I had to write some custom scripts to test some functionality of a web application and I got to thinking there's this data I'm collecting I'm looking for something specific sit well quite simply I wanted to be able to see the data syntax highlighted I wanted to be able to see the data rendered in some sort of web view and I wanted to be able to parse elf scripts and comments and in all the general things so I created a simple QT interface that allowed me to do that of course the tool today looks absolutely nothing like the beginnings of it so yeah it used to be just a basically a sequel I browser yep with with a commune focus towards web technology so basically it's not an inspection proxy so that might throw throw you for a loop a little bit but we decided to take a different route and kind of change that that paradigm that everybody's used to because if you think about it you're just chaining responses through another another device or another application so that's really important because almost all of the workflow that you see on all the other tools as well you'll set up this inspection proxy use whatever browser you want to go browse through the site and then come back and look at the tool again we decided to just cut out that middleman and instead you can import data if you already have it or you can just use the browser that we have ourselves so we actually built WebKit right into the thing so it works just like your safari or your chrome does renders things the same way so a lot of tools that have their own browsers they often have something that's kind of not as full-featured or is just a little weird this one is going to work just like you expect it to we also a big piece of this is we made a custom analyzer engine so you can write whatever it is you want to find easily and then run it against all the stuff you have it's all open source it's Python and QT and it's designed for testers this is not a fire-and-forget click the button and your report is done tool this is designed for someone who already knows how to test web apps but they just need something to help out so now we're going to have the demo what you've all been waiting yeah we got through like the boring technical stuff so we'll see if we can get this going
all right this is the user interface and
we're sorry about the screen resolution green resolution looks like crap yeah so this has a little bit different workflow like Nathan talked about this started out as a way to look at data from other data sources we have our own capture format that we have defined it's an xml-based format and what we have provided a DTD for you and there's a URL lib to module for people to do python that you can just plug it in as a processor and it automatically generates this format so we'll look at some data we've captured and you know I just want to point out here there's we're off the internet anyway but there's an important thing that we discovered is when you're looking at rendered data from old assessments you know you may have a limited time window for do your testing anything you're not supposed to be interacting with the site so we have this black hole network so if you're looking at old response data there's no traffic being sent out to the internet but when you have captured data if you have all the references all the images anything that was a originally referenced then the built-in rendering will pull that out of your capture data and display the web pages it originally existed we have responses here I'll zoom them as soon as you guys can see we have the zoom feature which is really handy you can see the request the response i don't know if there's any scripts on this but we pull them out i'll find some with comments links we plot all the links so you get a quick view of what all the references are any form values you get those we kind of we kind of go through and parse out of the DOM and generate the list of forms and then one of the really handy features you know if you do a lot of assessments especially with highly dynamic applications you know that you have to do view source a lot you generated source so we have the generated source that we render the render the page and pull out of the DOM and any of these like references like links and forums that are dynamically added those also get included here so that's pretty handy so that's imported data we also support art from the ramp format we also support logs Burke state files if you use Burke pro you know the save state files that's then the XML saved XML and web scare of in Paris message log formats we don't do there's their storage but we're working on that because it's a not a good interface we also have our own built-in web browser
so you can go and pull up sites your let me copy one here I'm just type that in
yeah don't don't get into my my porn yeah definitely so we just have an instance of the broken web applications running locally here so this is a simplistic one bending over like that that just looks strange this is a simplistic view of our browser it's made as a proof of concept so in the future we're going to add there is and there is a the question we got the mostest there is a back button you just have to right click in the page to go back that's terribly intuitive yeah yeah we definitely have some you I you know we're not do we designer so we have some like you I inconsistencies so you just have to work through those but as a close that and if you I scroll down here
and we'll see that those any requests that we've been making yeah we have
zoomed in view so those requests that we
were just making are now being saved in
our storage we use sequel light in the back end so those are just getting saved to a local storage database other interesting tools we have a little
search engine even though you can go
through and type in like let's see one woodford HTML comments so this one I'll have some comments in there and you know we offering a variety of search routines there's a built-in differ let's see if I
can find some pages that are similar now I'll just pick two that aren't just so you can see the differences
this is like a really bad example but with syntax highlighted it's using the built-in diff live in Python if you're familiar with that so it's based on word matching and not bite positions we're
going to cover some of these things like the analyzer the requestor you know it like simple tasks like requesting a whole bunch of URLs that other you know it's like going through and saying I'll copy my sitemap and re request them with a different authentication sequence that's like really tough to do in a lot of tools but you know here we have it all template it out and you just copy the URLs in here pick a sequence and rerun it our crawler is a little bit different it has the traditional web spidering approach but in addition to just pulling down and analyzing the raw HTML it also renders anything so highly dynamic web applications that you know that are based on Ajax or some other rich client technology we can go ahead and pull out any dynamically generated links and follow those we generate Mouse events text events submit forms do clicking all on generated dynamically generated basis we have an encoder probably the most interesting thing was you tf7 use generating malformed UTF 7 there's a lot of laughs out there that check for you tf7 and cross-site scripting attacks but they don't check for malformed versions and web browsers are more than happy to render you tf7 if it's malformed we have a data bank it's going to hook into this botha spidering and any sort of sequence if you do replacement dynamic data replacement we'll cover that a little later sitemap and you get a view the view of the site cookies in addition to our younger traditional cookies we also have offer views of Flash cookies so this is pretty uninteresting Nathan cleaned all the porn office machines so all this Flash cookies got deleted right now you can only view them but we're going to give you the ability to edit them that's really important like if you're testing risk basic applications with risk-based logins that are storing data in those Flash cookies you know getting a good way to go in there and edit those values and html5 local storage will all go through a little bit of a demo of that later and that's about it for now we're going to start covering some of the other interfaces j-rock doesn't know how
to use a mac so sorry guys to drive it
for them so as the regular consultant I get all that boring slides too so we run on mac we run on Linux we run on windows we've mac works pretty well windows you've got to do some compiling I'm not going to go into the boring details the easiest way to use this right now if you have backtrack 5 you have to do one apt-get install of q scintilla and then just download our stuff and it works we've been trying to keep everything that we need packaged in with it there's just a few things that we don't but this is the list it's up right now in google code we will eventually have packages so for those of you on mac and linux it's and windows it's easier for you to to just download it and run it and please please so how many of you are web application testers like us show hands ruff okay how about functional testers of things that might use tools like this for security a couple more okay so everyone who knows how to write documentation we need those we have to so all you guys that are testers whether it's functional or application your specialty is telling developers how much their software sucks so we need your help in telling us bug reports on what goes wrong and and then hopefully some help on how to fix it too but even if you just tell us what went wrong that'd be great yeah you're probably going to want to wait like a week or two so we
can get back from Vegas and fix some of the problems that we found while we've been out here we're you know we're like 12 our development like running down you know 10 minutes before we had to present writing code so you know that's the way it goes yeah we presented at black hat and we didn't see the outside of our hotel room pretty much the whole time okay so now we're on to the analysis engine right it's obviously in the title this is a big portion of raft we wanted something that would
actually analyze everything that we currently had right I don't want to have to spider the side again to figure out if their comments that may you know have some data that are interesting yeah I can go back through the burp state or whatever but it's you know you always find yourself writing another manual tool to pull more data down but if I've already got all that data let's actually just analyze what's there so our model here for analysis is something modular right we want to be able to you know write one time and have it analyzed all this all this data that we have we want to be able to analyze sessions as a whole not just you know single request responses you know if the first request is different from the last request but we made the same or the responses are different but we made the same request we want to know that and we want to know why so we want to find what others ignore we want to look at timings like how long it takes a page to respond we want to do some image analysis if you guys have looked at Google Images lately they now actually pull out exif data and will display where exactly an image was taken or how or what camera was used for at that type of information and that's not something you typically look at during an assessment but it could be useful information especially for a social engineering engagement something along those lines so the possibilities are really endless these analyzers are extremely easy to write um let me show you at least the demo right I guess
we've got anal going on right now but right now there's nothing in the analyzer we haven't run the analysis or the analyzers yet but we are looking at actually hooking some of the scanner and fuzzies end to the analyzer so it would kick off specific analyzers when when we make a request from things like that but currently we have to actually click this you know circle button up here which which runs the analysis and we get back in this case 120 results now these analyzers this is currently what we've written some are in flux some do more than others do we've got you know we're looking for error messages insecure cookies i mean the low hang fruit what's currently out there we're analyzing some redirects to see you know if if there is more information behind a redirect than is actually displayed to the browser I mean your browser takes a redirect it just takes you to the next page but I've seen applications where people actually allow you to spider the admin section because the developers didn't write the redirect portion of the application correctly timing analysis thanks a lot PHP developers yeah you've ride on men the timing analysis looking for the denial-of-service pages you know anything that would cause the server itself to spin longer and to take up more cycles so we could you know potentially execute a denial service attack so these are ones that we currently written if you've got other ideas for what can be implemented let us know we did implement simple regex and strings so all you have to do is change the configuration add your regular expression that you want to look for in all of these requests and it'll display them to you so currently I think we've got you know we're looking for personal information so in Alterra mutual it found some private information here you know phone number and I'll actually tell us that's in the response i believe yeah so it finds that finds the phone number and it'll show it to you so it's easy to scope through there and i think when we were building this we decided hey we want to know if XSS has been found it took us all of 20 minutes to write the XS s finder to see if it if an alert box popped and if it did then if it was in the response in the request then XSS is obviously within the applications it and we'll get we'll do an example that in just a minute yeah so that that's the analysis engine jump back over to keno
so a little bit about our smart testing components because we basically so far we've been talking about data you've already collected so now you've already collected the data you want to do some additional testing based on the data you've collected so we created a requester and a fuzzer and those are template eyes and we'll get into those in a second we're about to do the demo but we also have the ability to run sequences so you can launch the sequence builder build your sequence out and then import it into or just select it from the drop-down box when you when you're doing your testing and of course we have a browser object so that browser object can be utilized during the testing as well so this template approaches probably better just to show you versus explain about it so we'll go through a
simple example of using of using the templated approach to fuzzing so we're
going to grab our URL here make a
request for the resource so we want to
we want to actually replay this request first so we're just using to open their OS broken web application we know that this is vulnerable to XS s so it's a good place to test and we're using our own built-in web browser so any of the requests that you make sure here end up in your data set automatically so here's
the request that I just made your name equals test easy enough to send that over to the web browser and as you can see the templates here so there's you know some automatic templates but there's a payload drop-down box for you know where you want the payload to go and here's the mapping screen so you can map payload names to different sources yes the two in there or hard-coded that will not be hard-coded for long we needed something for the demo so we'll have a directory where you can load all your favorite lists and those will be automatically available to you in the interface so yeah so in this case I need to fuzz the name equals test variable so I'll add the marker just there at the end of the URL this will be explained better our documentation so because we're working this out right now so our copious documentation is yes all right so all we have to do is start the attack you see it went really quick again you know these are hard-coded lists we have you know it's not everything and anything but you can actually view the results what it did all of them that it sent so if we're looking at sea yeah so you can look at each of them separately
or we can go back and look at them in
the response view now at this point we've you know we've done these tests we want to see if it found any excess s so we're going to run the analyzer again and now we have instead of 120 results from the first run that we did we've got 162 and all those are in the XS s finder it looks like we've got a couple here if I actually render the page and it was successful i get the XS s pop up because i am rendering it i mean we're actually running the WebKit engine behind behind the scenes
okay now so we're going to have a sequence buzzer and we were looking at his laptop is kind of funny because it locked on this so we don't know where we are so we're going to have a sequence buzzer and that sequence buzzer is going to allow you to tag data in sequences we call it dynamic data replacement so you can import a sequence of events tag that C surf token that elusive si sirve token that's making all of your requests fail you can tag that and then place it into your payload so now all of those previous tests that you were running that we're failing I will now become successful and in the future we're going to have the ability to do any kind of
dynamic data on the Dom so they're really really really really really
difficult to test applications well you'll now have visibility into doing those without having to do them by hand
yeah so I'll just do a quick a sequence builder demo this is obviously not completely functional yet but at least give you an idea because we're still in the process of getting our dynamic data replacement features to work so here we have a pretty typical login form we'll just login foo foo is that submitting we should be start seeing cookies getting captured if that ever comes back we'll see the prime network is down Oh in the network is down yeah the local network and we're in bad shape but you can come through here and by default all any sort of media responses are excluded from sequences this is where you'd configure the dynamic data replacement we're going to offer the ability to run the sequence in a web browser so you can literally render the whole thing we discussed earlier about some of the sessions state problems so we offer both an in session pattern in an out of session pattern so that you can figure out well do I have some specific requests that's causing a problem so let's see I think get a log out log out people that are leaving to be really upset because we're giving away free cookies yeah ah what am I looking for here well again log out just sign out yeah could be no it's log out all right so that one you know if you've configured other tools sometimes it can be a pain to figure out so we actually search through dynamically and a mark it up for you yep so how many people in here would like to search for Dom based XSS without having to use a browser
plugin or having to send your website to some application off on the web and have it test anybody that should be pretty much everybody so we have a built-in Dom fuzzer so it's integrated into our tools so we can identify things like Dom base cross-site scripting without you having to use another tool we're going to do demo that repo so I have a imported a couple of web pages that are I know vulnerable to dom base cross-site scripting because I wrote them I'll show you their response here so if you look in the response it just
does a simple doing this on a little bit this resolution is just killing us yeah so that's just doing a document dot write location.href so that's going to be vulnerable in some circumstances this
one is also doing something similar yeah this is doing a nun escape so if you come over our Dom buzzers very basic but it still finds stuff so if you look at these tests we've generated some unique script statements those are quite common alert that would go into a string if you're doing a string and as a pattern that's going to be pretty much unique on the page so what we'll do is we'll go through and a run the the fuzz are based on the that is the absolute first time
that that's happening yes where it you
know that sounds like a joke but it is
the first time it's ever done then so I guess we need to fill out some bug reports yeah I guess so so what this is doing is it's taking the saved HTML data and it's loading it into an instance of WebKit and it's setting the the base URL to the modified value so this is not making any sort of network requests at all but it's still replaying those values and rendering it as if it was on the original site so we should start seeing some results coming through and then we're doing a couple of matches here the first one is high we hook the alert box so if we see an alert box pop up with that number in there we say bang found some Donbass cross-site scripting in other in other cases we're just looking for pattern matches in the rendered Dom content so like if we look at this guy you can see that's written out and if we render it boom right there so we found Donbas sprout site scripting without sending any network traffic at all and is it are we yeah okay so this we have screenshots of in our presentation but I'm going to try it you know if this works oh yeah i'm not sure but one thing rules don't normally do is give you visibility into thing like things like sso or LSO's and allow you to modify those to test to see what would happen yeah this is this is kind of a a twitchy one so the html5 demos if I look at this and will render it so you see to come over here you should see the local storage now has a value in it yet so if we set that this is the part that doesn't always work you're not supposed to tell them yeah well like we said this is still kind of a work in progress but think no other tool allows you to do this so being the only one that cut in and it kind of sucks right now is better than being yeah yep we may disagree and that's okay that didn't work so we'll switch back over what happens is it gets cached the browser's cache in that local storage reference in we're directly
modifying it through sequel light so you
don't get to see it so here's here's the way it actually looks should the time that it did work yeah it works every time you just have to restart it which is annoying so you modify the value and then when you go back and rear ender the page you get the pop-up box so that's actually modifying the local storage and some some people asked us well what is the attack scenario there and we say well it's really no different than any other situation where one part of an application accepts data and it doesn't properly sanitize it and then it gets replayed later so let's say up a web page that's storing some sort of user preference in a client-side storage i mean you know developers are starting to use this not even realize it like they're importing j storage and using it without understanding where these values are actually being stored and then later you come back and it pulls it out of there and doesn't encode it or sanitize it and then you get have cross-site scripting so let's say you do some sort of forced browsing to send the value to that web page and get it saved and then force the user to rerender that page and they themselves find out to be vulnerable so we have some more slides
and we have no more time so what we're going to do is as far as documentation goes there's some on the project page as well as our our slides and a copy of
these slides is already already available on our project page so if you
have any questions now's the time to ask
them we're going to be at we're actually going to be in the QA room as well so we can actually go through some of the stuff we weren't able to cover so thank you for coming out we hope you use the tool please submit bug reports please submit feature requests we had some people come up to us and say that I that they don't code but they like to help out well we need people to submit bug reports and help us with documentation right Doc's please oh I mean we aren't an inspection proxy thanks bye
Feedback