Video in TIB AV-Portal: Sol[IDA]rity

Formal Metadata

Collaborative reverse engineering
Title of Series
Part Number
Number of Parts
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Reverse engineering is an exercise of exploration and digital cartography. Researchers slowly unearth bits and pieces of a puzzle, putting them together to better understand the bigger picture. Binaries, like puzzles, can be put together much faster in collaboration with others. And with services such as Google Docs, Office 365, or Etherpad, it is easy to recognize the power and effectiveness of real-time collaboration in the digital space. Unfortunately, reverse engineering as many know it today is almost exclusively an individual experience. Our present reversing tools offer little in the way of collaboration among multiple users. This can make reverse engineering tedious and wasteful in a fast-paced team setting. In this talk we’ll be publicly unveiling Sol[IDA]rity, the newest collaborative solution for the popular disassembler IDA Pro. What started as a simple plugin to sync IDA databases between users in real-time, soon evolved into an interconnectivity platform for IDA with endless potential. Join us for a glimpse at the latest generation of collaborative reverse engineering.
Collaborationism Computer animation Information Multiplication sign Series (mathematics) Flow separation Identity management
Order (biology) Student's t-test Disassembler
Collaborationism Context awareness Game controller Software developer Multiplication sign Projective plane Motion capture Real-time operating system Line (geometry) Event horizon Degree (graph theory) Revision control Latent heat Mathematics Pattern language Endliche Modelltheorie Reverse engineering Directed graph
Game controller Process (computing) Code Digitizing Binary code Right angle Reverse engineering Library (computing) Physical system
Information Mehrplatzsystem Projective plane Data storage device Shared memory Database Real-time operating system Bit System call Abstract syntax tree Word Centralizer and normalizer Right angle Endliche Modelltheorie Sinc function
Word Functional (mathematics) Group action Computer animation Demo (music) Computer file Multiplication sign Variance Database System call Metropolitan area network Directed graph
Group action Multiplication sign Database Instance (computer science) Computer architecture Form (programming)
Web page Trail Complex (psychology) Collaborationism Server (computing) Functional (mathematics) State of matter Multiplication sign Projective plane Database Event horizon Cross-correlation Hash function Right angle Spacetime
Server (computing) Centralizer and normalizer Information Integrated development environment Differential (mechanical device) Database Client (computing) Endliche Modelltheorie Event horizon Computing platform Computer architecture
Module (mathematics) Collaborationism Server (computing) Block (periodic table) Projective plane Client (computing) Event horizon Equivalence relation Computer programming Message passing Component-based software engineering Core dump Software design pattern Software framework Endliche Modelltheorie Computing platform Window Plug-in (computing) Near-ring Row (database) Library (computing)
Server (computing) Functional (mathematics) Real-time operating system Client (computing) Event horizon Web 2.0 Goodness of fit Computer configuration Core dump Endliche Modelltheorie Data structure Information security Computing platform Physical system Module (mathematics) Collaborationism Demo (music) Projective plane Binary code Shared memory Database Maxima and minima System call Data management Loop (music) Computer animation Software Right angle Routing
Different (Kate Ryan album) Projective plane Database Right angle Directory service
Server (computing) Computer animation Gravitation Instance (computer science)
Collaborationism Server (computing) Dependent and independent variables Computer animation Database Event horizon Usability
Computer animation Personal digital assistant Projective plane
Service (economics) Sound effect Database Real-time operating system Instance (computer science) Computer Distance Event horizon Computer animation Finite difference Operator (mathematics) Right angle Directed graph
Multiplication sign Tap (transformer) Projective plane Shared memory Bit Real-time operating system Field (computer science) Type theory Database normalization Computer animation Hypermedia Different (Kate Ryan album) Normal (geometry) Data structure Reverse engineering
Functional (mathematics) Group action Computer animation Different (Kate Ryan album) Weight Selectivity (electronic) Database Right angle Event horizon
Point (geometry) Slide rule Context awareness Hoax Decision theory Multiplication sign Projective plane Event horizon Power (physics) Broadcasting (networking) Computer animation Single-precision floating-point format Order (biology) Finite-state machine Diagram Endliche Modelltheorie Reverse engineering Physical system
Mathematics Arm Computer animation Synchronization Fourier series State of matter Multiplication sign Database Sanitary sewer Event horizon Physical system
Multiplication Server (computing) Building Functional (mathematics) Personal digital assistant Block (periodic table) Database Right angle Function (mathematics) Graph coloring
Module (mathematics) Pixel Demo (music) Assembly language Binary code Mereology Machine vision Type theory Mathematics Different (Kate Ryan album) Energy level Series (mathematics) Data structure Window
Server (computing) Group action Computer animation Information View (database) Projective plane Identity management Directed graph
User interface Game controller Server (computing) Information Projective plane Directory service Web 2.0 Mathematics Computer animation Personal digital assistant Different (Kate Ryan album) Form (programming) Task (computing)
Area Slide rule Beta function Multiplication sign Planning Open set Twitter
Game controller Server (computing) Multiplication sign View (database) 1 (number) Database Event horizon Mathematics Video game Right angle Quicksort Row (database) Physical system
so the bed and time and then that
in a the end the and
so when we have a talk on collaborative solutions for ident to start things off and said yeah so if it's the thing several information now here last series and so on solidarity I
worry about my name's markers does only a mysterious offered here Microsoft and I the Bird and I am currently student at RPI
and so I don't already this talk is almost purely about order I Prodi don't know it's a disassembler of has developed by hats eraser here somewhere so I all I'm sure all of you know today is you probably all have a license overlaid
on so just to give some background on a so far the project in the history of the collaborative reverse engineering in the context of our approach and we have a little time line here but I don't know the specifics of the individual projects too much because this is only the top so way back in 2005 that we 1st sizing by pattern of that was a very simple solution so some you as in Google somebody's if you want it's pretty interesting to look back on history collaborates 1 that everyone probably knows and talks most about they're all like they you know that 1 in ideas sinking solution that was really crappy International time so and was like a eagle on the this and break the so what's interesting about this you project so that this was way back from and these projects attempting real-time sinking of their account of what we call it a push a push based model but then we started seeing other solutions coming through such as being contrary i'd that synergy and of the all these solutions were whole slash control SPN get based on you would commit or you could push your changes up by hand and other people have to pull them down my hand and it was very strange and you know kind of what happens here and I have to say we stop dreaming right come but it also might have been the fact that I wasn't quite ready at the API was in in a place where could support real-time of collaboration plow all real-time event capture to the degree that was needed for accurate and events sinking or how how developers would need it and so version
control and you know that talking about
libraries intros bad is that this shows a medium to document the process of creation right for resource control on your developing Toda writing code but reverse engineering
is an exercise in discovery and digital cartography right we're trying to understand what this binary is with this blob is you trying to document that you're not writing code not developing a system so you
know pull inverting atrocity and get that seemed very strange for a medium of no storing and maintaining notes in 1 hour so here we are again 2016 and we are unveiling solidarity of personal project that we've been working on various about the year have you have you have about a about a year since about last year and
so solidarity so we start off with these initial pose are here and you know for for t central calls was a lot of same I databases nail right but not a CTS were very fast paced and we you know we we need be able share information instantaneously on and we also want to say hey can be syntax trees in real time to and you know that was something that collaborate there's no way higher ability that primary tenet touched on some that's up a little bit but then again there also was more of a kind of a poll the base model I'm and I said I want user curses advance on ITER is now out there and so I is now been is the thing on the top line and you've seen is always usually blue or whatever on most of us ignore has to have but I was like I wanna see multiple users working on the same database and I wanna see where they are in the database on an have and then the last 1 was it has 2 word that's how I want I 1 another tool that you know kind have words and doesn't and what not so as we started out you know we took the 1st goal this is June last year and this is all Jeff of
it he's the variance and very quick demo but I word then it is just a very simple demo approve concept you know we sat down relate in the couple largely of man like notices plate a sink in time user actions between databases ridges so it renaming a function call here and really qualit so easy will be done by
death and files uh said that was June 2015 and we are still working on this so and but they are the 1st
attempt was very simple you have to write a pro instances or solidarity finds little form for now this was like the 1 of solidarity were gone through probably up 3 huge architecture overhauls and and so I was thinking time these are action between databases that is actually really easy and if you put in some time and effort
on but then in are you know
and this is just between 2 databases are who to users right it is both for the same database in both there are at step in each other's that use in fact and that you know that wasn't hard but there were a
lot of added complexities you start to realize that OK the you don't actually want to sing databases there's a lot lot more of like how do you ensure the that everyone starts out in insane right so you just have this very simple idea if we just sent every event everything will be OK but how do you even make sure that everyone is on the same page before you start sinking events right people are going to be of going to sleep waking up like during CT slight having different states of the either the database you know you're the you need the same basis out from that and so that you know that is 1 thing correlating in routing multiple sessions at once what if you want multiple people working on different challenges at the same time processing server what if there and you know how do you have like 5 people working on the same binary terribly do that with the you know that single pipeline we had before so in capturing only the track database event so i'd is notorious for a it's auto analyzer at least in the space of of collaborative yeah have elaboration yet all at almost every project is struggled with the auto analyzer because the auto analyzer is helical a one-way hash function only you say hey define a function here and it will go through database and just start modifying the database from India illustrate the stuff and there's no way to go back and that's why there's no 1 do you only go forwards and so on and a lot of people struggle to actually
differentiate between what the user is doing and what the auto analyzes do into the database but so you know in our naive and initial attempts we we had no idea about the auto analyzer and all accuracies south of and so that was 1 of the big issues that we have worked through in that everyone else that has worked in this has had a work through that would come up with some good solutions of but we'll get to that that so what happened is this product and of evolving into more of a we recognize that we needed a a platform to build this collaborative environment that we were seeking 1st CTS and the and it instead you know just sensing information in data and events between databases we wanted a platform that you could really work together on you know and so this this became a platform to enable either interconnectivity of and so over in a give a brief can have a high-level overview of the architecture then zooms down and so you know you just
have a basic central server and client tenor model going on right I'm all year either clients of phone into the server In the server bounces everything and rats traffic accordingly based on different sessions that are going on and all that you know it's there's nothing too crazy there
an but then we start to get to you can know what we actually call the platform so you have to you know 2 major components to Solidarity had the client on and the server has and so this is how the relationship we see between the 2 halves so the idea client this in the windows that whatever running either problem and and then you have a server us who have mostly been using a bond to uh but it's it's essentially we have a twisted I'm a Python twisted but I have never been him with like that on the server will look at the the and in and often on companies the server the client were able to extend them with marginals so yeah this is that the out so we have in the the plug-in which is built on the bottom row which is the what we call the solidarity client itself and then at the salaries are in this this whole get upper year is the platform up but then we what we do is we actually build modules on top of the platform so it's we actually made in either plug-in that then has plug-ins so that's kind of funny and the server has equivalent plug ins on its side and so it's very interesting framework that you know will will you'll you'll see false and so
don't build a more detail about the client near but it's written in the Python entirely the reaction that extend ITER Python and flesh it out a lot early on because the event handling stuff was incomplete when we started last year and now I think as of 16 6 9 it is more complete are if not finished at this point and so the plug-in that plant plugin is entirely in Python which is different from collaboration so the past projects which were built in the sea and we actually use of the client is using the actor programming patterns of if you guys know about that and it's of visible using the Pico Python library and so the actor and program pattern essentially is it breaks up things into all these different little nodes are models of and these modules communicate with each other they pass messages between each other in act on these messages and so you know we we have a core kind of module on and this kind of helps load and organize all the other modules and all these models can pass messages to each other based on their needs in what Anderson doing and and so it's pretty sweet it's all asynchronous nothing blocks anything else and then then there's more models so on the
server side of the year we have also it's written in Python but it's using twisted they sink I O 3 twisted to do the networking from event loop and to keep things from blocking that way and so we can handle a goal that from clients coming in and and wrote the events basically and it for the database model were using SQL up to me to to do the like me the actual on or modeling and then so the purpose of a server to rout between quite clients obviously but then we also implemented by a user management system and is with a web server to help you sorta like manage the entire system better and we'll see a play yeah the and also just something that's kind of funny to note is that all of the not mediation is overkill some 22 in really only should you have little option there too late made optional real like 0 no spirit good security it's actually force that on so you can turn that off and say that we actually you know trend in instill some good things and so was certainly was getting to this is that we don't have a wider and we have lots of really cool just 3 guys which is basically a ordinal so was certain about the modules that extended on the platform you know that really made up the core of what you guys care about you know his thinking and all the really cool stuff so the 1st thing we really needed once we once-feared you know can establish the platform was some way to correlate you know these users and sessions in binaries so if you have a binary right you need you you want you want to work have like 5 people working on this binary so you know we basically can a call this a project and and this is you know what became the becomes a foundation for collaboration our and so the project's model offers a few features so even if you don't want to necessarily work in real time with other people you might wanna share databases of other people so projects module offers minimal functionality of you can easily upload databases and download databases and to server and it provides a very easy way with an eye to access in push and like Paul databases super conveniently is no longer any what's right here I mean Akamai IDB you get on you can just do all this directly from within out with either so this is a a little demo
of it and so were able to come from within i'd actually go and create a project through the and the log actually is a center director structure that can go
in and create new directories and projects in and then from there
actually come the upload the database of a working on right now so that other people can then use it and I once it's been uploaded yeah and so you can specify since details about the projects that are about the entity so that people can see it and then we just uploaded great to server the and will become instantly available to everyone else who is using the platform and and different than the show the analogous to this the next 1 we're downloading and an
opening so we have a blank ITER instance and we're going to open the from the server directly that so here's you know users that upload something someone else they are only gravity you know
during and so this downloads and so now you can very easily and
that axis there are some houses the and IDB that they want you to get without having to do anything too complicated as directly from the there so I mean you know even without talking
about the sinking events and you know all this and that were already in assigned to see some some great solid of you know collaboration opportunities right
which is you know the ease of being able to share databases across you know anyone that has access the server silicon ICT team or the
instant response team in size and company in the supercontinent at
and complete just mention uses the and the what do you think
I yes so we are because you were my absence divided these and stuff we implemented a system so that you have opened projects of prior public private projects are private projects so you can add users to prior projects and only they can access it just as a protection in case you have sensitive stuff inside a company the only serve you can access and so on so some some basic
permission rights and I will talk a little bit about real-time or you know with a lot of people care about the database events thing right so this is just
does some some more trivial
of it so to we have 2 different i'd instances here Bo for connected I can mention that there aren't from computers but is easier this way the up and so were able to do
and up under different operations in ITER and they will be directly a between both uses in real time so where you have a distance and basic remains in this 1 of your images and operand those injected 20 can on both sides and services or recall more the easy sinking just it because user I'm pretty simple operations they don't have any side effects really up on their own they don't
taps are added to have a so
this is you know stepping up a little bit more this is like structure creation in changing the the types of the different fields and structures and this is a little bit more interesting older more out of the norm very useful that we construct user thing very useful by half a reverse and great guilty share them in real time with people working on the project with the yeah so this is just structure creation in defining some filtering in some stuff on Talbot quick and and uh so this play original time of redundant and so they're pretty pretty corporate media um changing some type some in on the fields and Renault actually Delaney entirely is Chen it's going people are in Canada and
Sevilla more advanced here is were actually engage all analyze lit by a on defining functions In all sides have undefined in the same way so now we can go back and start redefining code and let the
odel analyzes do its stuff BC there still unable able to think the same events even though at is creating a ton of
actions that is doing to its own database so now we're starting to on defining the final selection of weights from which is also all interesting it's different on defining defining selection is actually a friend from this hitting you and like on defining and a you know from from where you are downwards so yeah it's it's cool you know where were screwing up all of the year but it's insane right that's that's what we care were
trying to minimize evidences as much as
possible so yeah that uh so there's a little more than so every
summer secrets fur the events in the novel on the full struggled on I want to more slides in here but
i ran out of time so you only have 1 slide as some of the secret sauce is you have to really respect ideas also analyzes so it might not have been the best design decision way back analyzing the ill thought-out about that how way back in the day they were hoping to have an AI built into either to help you know system with reverse engineering and and so it's you really have to respect it and the you know don't mess of that late when when it needs to do stuff you let it do stuff and you back off and so the other you know the other uh super-secret hint is utilize the cute subsystem like is there for the taking and you can do almost anything you want to either few people realize that but I is so so customizable you changes about any aspect if you guys have seen some crazy all late all the you know customizations people have done but for all a dividing 1 dollar crazy menus and what not like you could do the same exact thing without a few people actually utilize ideas API or it's you know that the power of the T system so also and the last point is providing context to the uh signals and hoax and events that uses art that are made available in order of collaborating some the other projects just captured them in broadcast of them the thing is some of individual signals mean a whole lot you need to understand them in context in the I wish I had time to put together a diagram to better explain how we handle this but some or events sinking self how we can deal with and watching understand the auto analyzer and what the users doing is actually is a finite state machine to walk through we walked is finite state machine based on the singles were receiving so 1 single could be interpreted by different ways and you know it it could mean by different things but it really depends on the context you capture you and that's how we try and interpret things and only been bitten by the analyzer yeah so by stevens's on the models here so replay a very useful if
all year friend is doing a bunch of reversing and you went to sleep a something and now he's done a bunch of changes and I you wanna come back on line and get those changes so basically what we're showing here is a and make changes to 1 of the big databases that's connected and the other 1 is currently not open are connected so then once we've made some changes we will connect the other 1 and it will sink up to the state of the arms the 1st 1 and it tried we will be back in sync with the other 1 you can see the changes having background in our back and sync again the and so if you think you might think 0 what if there's a ton of states or we have a system where I'm snapshots of the entity be uploaded you bring you download that and then rebates from there so it's a very few men of events the actually the sink and so it make you really keeps you enabled the Fourier really able to keep up that that the sewer ordering on time so
the human so satisfied so in either you
have I saw the satisfying the bottom and like while the so much potential so I started from you know doing crazy things with qt and out of the outputs of them there and so I did so I felt like there is no user status and you can easily select which everyone expects you I'm in case you have multiple maybe play with multiple seats CTF teams the companies whatever CT just narrative dismissal thing the kicker server what not and and but we are given is around so
invites so I divided toast right by we would toast notifications so that if you can right click somewhere in the database and OK like your some across Romania upon the bodies in this function is 0 what functions are not going to say x for 1 B at 7 3 at any more I just say I click as a invite me here and it is it's not efficient them you can just click that notification and he database that's 1 over there I'm alternatively you can also you know if you were busy reversing something the salute you have an efficient at him and on and so building blocks found with the colors mean but we have done
so curses I said I wanted puts up on the now banned x-rays give you 280 eyes but 1 of them refreshes the now and the other lets you draw a single 1 of pixels and up and other refresh API doesn't actually work so I had 1 of had 1 API and the duty subsystem to do this so you can see these are different users were working with if you hover over that we actually can us your names which is fond so you can kind see where people are working binary and the spread then you know practically no real probably ever users but this is part of the vision that we're trying to establish a user experience for you and
TextRazor so we said I do have series of us are gonna have our on while it is definitely feasible this demo some of it is just a little bit old so we are doing us some different type changes and structure manipulation within the actual x-ray window and its thinking up to the other person is also with any X rays window and so that way you know have to work at assembly level you what yet so I mean we all use have serious so much nowadays of it is still to be good at at module to specify the elusive readable but I know there is no you know we're working on Sally Thinking Hats race is step where feasible but if you guys could open up the microcode that would be great but awesome or as Ck API spread that the awesome amount had a reverse x-rays and that you know they have to respond itself and I had sometimes to understand how some in the eyes of work and then
yeah so now and very quick just wanted show the server this is because I did you use can be a little tricky maybe no 1 open identity like had someone to project or something so yeah
and so what we have is it's basically a reversible server here I you can see like all the projects that you are on the recent actions that have happened is like an overview the
projects and the information like uh these all have sink enabled balance private and so on I can get like a view of the
directories so they can kind of visualize that better and then further you can
go to a project and like see very detailed information about what's been happening in it so you can like get an idea of how the person's going and as well as we'll see like all the entities that have been uploaded in case you want to like take 1 of the system and use it personally or something and yeah and and so the
other thing is we really wanna like fine-grained control through the web interface is much easier to enter in and that kind of the but like control through all the different forms and everything so labeled this is the easiest way to add uses a projects like change details everything's enough ago indicted to this kind of tasks they don't actually involved reverse engineering so that's very much the web server and you so that there is
a you know there's actually a lot of there's a lot more that we do not cover all we only have 30 minutes so we know we're out of time basically what we we showed a bunch of and there's a lot more planning that we would like to release is a beta we submitted for a black patina open releases are a bit out and by that and and out so it'll be debated soon of that's all the promise of of there's a lot of other really cool stuff the retreat tradition guys but hopefully soon so you know falls on Twitter and the what's area pushups soon just you know so there's some central resource and you have all the this today can be yet and will you know put this slide up and death of that so that's
all we've got the and
I have yep the like that that all and I the the yeah yes so yes so the as and yearly around the repeat the question yes yes I am your views wondering if there is some sort of like Abbas control for the actual like on databases as your monkey somebody like completely messes up the database and you can do it so I and as I mentioned earlier we have a system that will make make snapshots of ITV's as as you going on so that you have to think of as many events and so you could easily tell solitary to use 1 of those as the most recent 1 and then just move on from there the other ones will just be like put aside and you can download them and do whatever you want but they will be used for anything just yet so you know we maintain a history of all the events a final deserver of that you know are more detailed precise events such as the stuff that was you cannot collaborate but we there we we maintain this this whole lot of them as well as various snapshots of databases that automatically get uploaded from a user in that session so every 500 runs say a database that uploaded and so any time someone else joins a session now that database it's pole down and they you know baby joined the session it is what it all down and in any event they have missed that have been you know records and and will be played you know which can be used you do the same thing and with any of the database is pretty much and there have been like the snapshots and that way you can get based go back in time if you really need to know so any other questions yeah you know what the this so divergencies offline is difficult right like there is no good solution to that but so so 1 thing and if you have to make a lot of changes offline you can basically create any snapshot then and people who want to work from that Alex fork of the thinking then go on from there but if you do the ideas you doing most the stuff online so ideally you would be able to connect and send like record the events that you've been making otherwise the server doesn't know what you've been doing yes the simple answer is basically a for sale and on today I don't think we really have any more time enforcing over and but you know his up will be around for the rest the day of our life that was his son for the