Merken

Keeping your tools safe IDA

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
I wrap add ham ham hammer man com mood Ombu who idra home I rolling along a path in 17 I
and so this this goes to this is to say so my over just a tiny bit bi-gram 1st so I'm man is on 0 I joined the bunch crazy people in 2011 and I were not so far from here on a project that some of you know uh when I joined the highest I had no idea exactly what we what I would be working on but something interesting happened is that and had just been released by just 1st versions in using Qt as the as the as a toolkit and unfortunately the guy who did the water then up a study whom some of you know I actually left the company at that time giving a lot of meaning to the bus factor thing because it was the you guy and nobody else that actually really knew how to handle the UI so uh there was maybe with a problem there so 1st of all just to to make it clear we have 2 big categories of bagging to could be categories of bugs either we have 1st a kernel bugs which typically can affect the loaders and processor modules erasing they're about the disassembly itself and we have another class I would say about which is EU-wide bugs which is everything user interface related happy crashes and things like that and this talk is about and the actual uh the UI that's not doing that what we call the kernel that's so inside not the OS kernel obviously so this is about you that and how we deal with those so do you why when I'm talking about you i is not about pages button is misplaced or anything like that we know they are right this is this is this is about basically rendering pipeline and how the that the distance that use your these actually rendered on a screen that's that's called what's what we call the rendering pipeline um this is about that I was not particularly inclined to go down that road but then again it's not as stupid as positioning buttons so it's actually fairly interesting and anyway we still have bugs coming in and somebody had to take over because well uh the situation was not exactly great especially we had the blanket problem where and when I fixed 1 thing in 1 place it exited the broke something the other place and of course then you rush the place you fix it and brought in the 1st place which you can do that forever if if if that's your thing but it's we decided we would try to do better so I whatever I wanted to do whatever I 1 where a fixed we wanted to make sure that this was fixed properly and did not happen in the future and the best way to do that obviously is not regression tests none regression tests for you however are not exactly trivial given that we actually wanted to have a platform independent and testing framework we obviously want them to ought to be automated and we don't want to have to we change all the tests every time the UI changes a little so we want to be able to access the parts that were interested in for a specific test and and deal with that and also yeah the results must be easily interpretable that's 88 text files uh so that we don't text files where we're interested in and that's good because then it's in it's in our servers and repositories so we know the history of tests we can do really relevant in operations on those that just a quick parenthesis here in my previous job I was working on something and uh literally a team of guys in Atlanta was uh where the HQ what the company was working for was they were literally just recording test you eat because a lot of you I testing is done through what they call you recorders so you just launched a petition Euclidian places and then it does typically screenshot comparisons um up unfortunately the problem the product was under heavy development which means 2 guys literally and I'm not making this up they were just re recording all the test every week that's all they were doing and the where a small team at X rays and that we we um we we don't want that and in in fact I don't think anybody should ever ever want that and this is indecent and job in my opinion but anyway so we we investigated a few options to 2 to 4 for tools that would help us achieve that goal we we set out to achieve the have a moment all of those that that the 3 that I used here 1 was less brain that than the other uh off the hook up out of the box it was school squished by a company called from logic is scriptable in Python among other things because it the it supports like 6 or 7 languages so you you get to choose whichever you want and the very important part is that it has an easy access to the qt which a tree which is can what we wanted the so there's a lot of buzz going around in squish if you go to their websites you may get notions at the at the at the end of the word salad that the stuff in there uh and um but never mind that duty which of 3 again this is what we wanted and we got it and the time so the but disagree granted that he's test recorder things don't use them and their stupid little stupid even squishes this recorder because they have they have to uh and if you if you start kicking places and they will it will actually have to put sleeps in a in order to make sure that the test will run again just fine and in fact it's not even a guarantee is just hopefully is going to run again on no no no way that this stuff is on this stuff independent of course that is if if you run the test on the west and obviously the widgets would be at different pixel coordinates that uh so my my advice would be just stay away from it it's completely done and another completely that and also a lot of you like testing is done by comparing screenshots which I let you guess if that
works well there is this whole page on the squish our uh the from logic website and explain like what the this is unknown reinforce creative differences obvious reason for speech of differences in other words screenshot comparisons don't work just don't do that because then what they start doing is but the comparison of screenshots and then of course there are some items that you don't want the these the day positives because of the fuzziness so this is exactly what you don't want to be doing in our opinion at least so and
just be very very brief on the uh on the speech architecture the uh it it basically and it runs test Biopython tests in our case we chose Python and it sense uh and it sends commands to a to a to a 3rd party to let's call this which server and that actually is in charge of launching a petition and then dispatch and according to the application um the they did this is not that interesting uh this is typically what happens if you have the test uh scripts on the left then you have the the discretion server and that close his eyes and so if you wait for a special widgets for example is going to say OK wait until this which it is available but is it's visible and active and then it's going to return a pointer so speaking if you call vertical for bore to get the scroll bar on the particle which have been is you send in Vol gets called Bob basically on the on on that which it to the server the server will do its thing in return another pointer and so on so forth so basically it's just it's an RPC mechanism really RPC by wrapping the and that the the Qt interfaces in our particular case so and that's great
uh the way squish itself is built is that when you want to build it from source and you need to provide a cutie uh source tree because what is going to do is that to portal the headers and generate ideas therefore for indefinite interface official language for all of the headers that it encounters and it's just gonna it generates automatically a lot of stubs for you like this the vertical scroll bar method called they showed above it knows that that thing has a vertical scroll bar method at all because of this idea of and therefore you have an easy access to a to the existing qt uh widgets which is great but I'd has a lot of custom widgets and we don't bills which against this in fact needed that that's the way the rule we should have taken that is injecting our own widgets in it before building it's where we chose a different approach the problem is that our on which it so what I call the rendering pipeline typically things such as uh this like this is a summary you uh deep did these these widgets are complete black boxes to switch which means what ever you do um near my doing here right
well and where is gonna where is my mother the the I'm sorry the the 3 treating
like to the right a much more left brilliance I did not see who you are but you will definitely get a free coffee after uh this is really cool ahead sh a date so again whatever operations you want to be doing in our own widgets I will not go back there because otherwise I have to do 3 fingers once again uh this is a course which will be able to do if we record that is to say OK click at this position and then dried for example not gonna work for us again because the font size differences or platform differences a lot of possible all really no you may say about stick to just 1 font size no don't do that because some tests anyway will need to use different font size is precisely because in some cases we wanna check upon sizes were fine so what uh and so uh what we introduced this just things to 1 thing that's called acute object protocol on our own widgets we have just 1 and point of this we call the squish facade of it's just again shorter returning uh um the QT object that is in trouble and that can be introspected and that's squish tend to so it cannot called the original method of or widgets but through this crucial side we don't have bridges basically to our methods that means now we have an entry point to calling stuff in our own widgets even though speech itself is not known actively about our own widgets and that's actually pretty cool and this is this is the uh this is this is the 1st other the over a type of which we call the customer and basically has this and uh in this it has 1 property like the renderer type whether in the flat model graph view for example and the and you as you can see that the the slot there what they call a slot is something introspective all uh and the differentia type would just return resource renderer type in fact so it's just literally just a bridge it's usually what it does is just it's just a trampoline so to speak and so this is a somewhat I move everything uh from the customer just because initially I started putting those discoverable properties and slots on our widgets with the quickly became a mass so that's why I extracted them into a facade that I could just retreat and work on them uh and so you have you have something like this on a widget now we define a public schools for property which expands to blah not so interesting but but it's basically just a bag of properties and accesses right um sort of a namespace if you will and we can call that could that if you want and they just do worry them very little they just query so there's just 1 hour of all use of squish with our own widgets and once you have that you would typically called it like this so you would wait for the for for the for the for for a specific kind of you in this case I have you ways of is something you and then you would you would retrieve the switch sad then call on this switch out and that cause into our application and retrieves our own custom widget specific stuff the uh and then of course you wrap it up into more or less well-defined API is because we don't exactly know where we're going 1st so are at the eyes of a are a bit of a mess in terms of coherency that doesn't really matter what matters is that they work so uh things acute object protocol this kind of thing is feasible and this is really really really cool so then we started writing tests that we wanted to have a pretty small scripts because otherwise it's down uh and we want the results to be of text files just as easy to to to to stuff into an ASEAN and then do comparisons over and uh this is an example tests for example the the 1st AI system areas but then I call on the viewer to say OK go flat mode then jump to a certain uh address and then go up and then go to graph and then make sure that the the current IEEE makes sense for example right because there might have been a problem there no this is a this was extracted from a test um so I it's that out at the at the end which is typically what will be end up into the listing file that we will actually submitted for so that would be the test results basically have so a few numbers uh and so we we started actually writing stuff in April 2013 and we've been added since then uh at the time of this writing which is the this is a small presentation idea some time ago actually um there were 400 test scripts but uh in all tests the descriptors are basically themes and in all of those tests we have multiple what what what I call phases so if you take into consideration all of the phases of all the test scripts religion to test for thousands of things which is not too bad and and get some pictures some numbers rather this this as you may guess is that and that the curve of the tests as we we started writing and so as you can see in the beginning we had a bit of a hard time setting it up setting it all up way with question everything so finding our Morris and then it started at some points in a in a in a note 2013 by the end of 2013 and then we have a huge curve going up there uh this is basic in these venues should lot of time writing tests and and then that fortunately and and the curve soft and some of its which means that paid there are probably less but now right and 1 thing that's probably worth mentioning is that around that time around here there was and this is a little bit of an achievement I'm proud of there was a massive refactoring in ITER between OK so that the red line here is the ITER really is
um 6 6 . 8 service back and the other red line is either 6 9 and in between there was a massive refactoring the why and when I say massive it's a massive and thanks to the test we hardly noticed real world problems 1 at during our beta testing and even more so after the release so it actually works right it's it's announcing it's a significant amount of work but it definitely is worth it in my opinion at least and I believe I'm not the only 1 thing in that so uh the typical the the the typical average output looks size again this must be readable by a human and the comparisons with the physical so we end up with logs that on overage are 1 that 1 about 200 lines the average test sizes to be if 74 alliance perscript which is not big and they work well I mean that is and they work use across platforms we have less of stability on our stand and windows in fact the my my platform of choice thing and this is where I focus the most and this is where tests are by far the most stable and the fastest and as he was he II getting at I have a certain advantage of by working on this and on this 1 but right so difficulties and I believe that you like this thing is actually fairly hard and other companies on even bother because it's it's pretty hard but I'd has a fairly large surface of actions and if you don't test that it's easy to not even notice that you broke something and then the reuse goes other people saw complaining because I am using that all the time now is broken yes sorry we denote so another companies I think that even bother and if they do they do it in the way that the recorded tests and quite frankly no uh um yeah so we have a it is the 1st we have squish and as the as I said I I introduce the notion of discrete facile over on just images and then on top of that we want above that we had to slap our own scripts helpers so as it this is this is the total amount of lines just will have a person that is all the tests were to be useful helpers I have about 11 thousand lines right there and just to have a cross platform kind of user friendly interface for writing tests and yet so that the tests are pretty darn stable 1 minutes and as I said last on the windows and and but they do run and a colleague of mine Troy uh added recently bench tests for I as the beginning and obviously those must be run from my platform because Apple uh so let us as a reference platform has 1 very very great advantages eggs with the suggested by muckily Ramiro here uh and EVA them if you know what it is but it's basic an X server and therefore the limits of visual interface um except it's in memory and it the inputs and outputs which is cool because then we can actually run the test disk which tests actually require input and output that is the new the mouse they will literally going and click on buttons and things so I um that's pretty annoying when you're working in you run a test and you have to have the everything happening on your screen this is a little bit of upsetting so uh exhaustive B is is super because not only does it let you run tests in its review and therefore does not interfere with your work that's the 1st bombers but you can parallelize it so now we run the test you by this of 16 and their ego so uh at the time we have a right now we have maybe a little less than 500 tests and run them all takes about half an hour I would say which is not that big a deal and we get a sense of usage and continuous integration now driven by a test driver I will mention later uh we did results every now and then I like multiple times a day of justly uh saying that hey this just broad and it's it's a great help and we can find any way to have an equivalent on Windows and on the west and we have a kind of a clutch analyst and it didn't really work uh but uh wrote running running on Exif the emperor lies and then just made our life so much better and so yeah uh this T 2 fingers I should have just driver uh which helps run a lot of things including the UI tests but not only um and and with this test driver I have added a few things such as the the flag look for trouble which means that we are underspecified tests and endlessly until it finds the failure or a crash or any any kind of trouble really uh that's J is the casinos as would make some multiple processes uh video captured that's cool because when you run an exhaustive B and something goes wrong high-volume have visual feedback but now with visual capture using ffmpeg fully capture and player typically for the playback I can actually in C frame-by-frame what's going on and that that is super helpful and then there is also a way for debugger which is very very cool because whenever um whenever something goes on at Disney ability together with the uh with that and with x of the Disney the ability to launch and lastly the tests and as soon as I starts the plots the gdb into it and and and so we would specific commands so that if the test crash well it stays right there but if the test succeeds and I DAGs it's gdb except as well so it's automated that again in UI testing because it's events base their thing some the hashes are very very very hard to reproduce that tends to this uh and I mean I could for example and opening engineer type this is
something that happened so I I I ran this particular test call tracing Mr. uh looking for trouble any and sj 1 meaning in its with the decision of the end but still and that then does the value which is short for wait for the better and so the test as you can see ran quite a few times here and then at some point eventually they receive the sex succeed and I have a backtrace uh proper badges and I don I should investigate all the stuff that he's um so that's like
massively helpful I mean yeah so the future directions of ego or Michael it's suggested may be doing some kind of funding but it needs to be heavily directed because otherwise is just a waste of time but I believe that the ones who are a little bit more comfortable with all that stuff uh yeah it's definitely definitely possible uh so yeah whenever you reported by depending on the nature of bugs is as I said in the beginning you will fall into 2 big buckets either it's what we call that kernel but it is to be deposited module unloaders known stuff what analyzes or to you in which case 1 of us uh will actually add UI test for its 1st before fixing it obviously uh and and yet you'd be surprised by the amount of things that we can test there are some limitations to you Whiteson but usually it's incredible the amount of things that we can test already and that we have we we have this it'll don't have a 100 per cent coverage but it gets better every day literally and uh this is uh this is very very helpful and and we have so much more confidence in what we're doing so um yes thank you very much for his and his the latter half lives there is again a
questions they have to answer them the the thank you little uh do you have any um API is our example so that the individuals could user this make this now we haven't published those uh as I said uh briefly before its time interval the little bit organically it's a bit of a mess they're not pretty they're not uh and at the moment we don't really have plans to publish them uh but uh maybe I mean it's among other things because we don't know if people would be interested so thank you for suggesting it I don't know firm if the pressure becomes unbearable we were just really is that at some point but uh there is no there is no plan at the moment and as I said it relies on it it yet relies on the other on a boatload of scripts that are in a state that I mean I wouldn't show my mother so um yeah thank thank you the no thank you yes
Resultante
Kernel <Informatik>
Bit
Konfiguration <Informatik>
Momentenproblem
Formale Sprache
Versionsverwaltung
Euklidische Ebene
Geneigte Ebene
Computeranimation
Entscheidungstheorie
Homepage
Kernel <Informatik>
Netzwerktopologie
Komponente <Software>
Softwaretest
Prozess <Informatik>
Typentheorie
Lineare Regression
Hook <Programmierung>
Speicherabzug
Emulator
Metropolitan area network
Softwaretest
Nichtlinearer Operator
Teilbarkeit
Prozess <Informatik>
Dokumentenserver
Kategorie <Mathematik>
Spieltheorie
Debugging
Programmierumgebung
Biprodukt
Teilbarkeit
Konfiguration <Informatik>
Datenstruktur
Funktion <Mathematik>
Suite <Programmpaket>
Client
Server
Kategorie <Mathematik>
Projektive Ebene
Programmbibliothek
Ordnung <Mathematik>
Versionsverwaltung
Prozessautomation
Super Nintendo
Server
Web Site
Ortsoperator
Quader
Wasserdampftafel
Klasse <Mathematik>
Content <Internet>
Systemzusammenbruch
Mathematische Logik
Framework <Informatik>
Datensatz
Modul <Datentyp>
Widget
Widget
COM
Booten
Abstand
Coprozessor
Operations Research
Softwareentwickler
Touchscreen
Beobachtungsstudie
Binärcode
Benutzeroberfläche
Pixel
Booten
Stochastische Abhängigkeit
Systemplattform
Paarvergleich
Elektronische Publikation
Modul
Coprozessor
Programmfehler
Portabilität
Netzwerktopologie
Bildschirmmaske
Schnelltaste
Zustand
Mereologie
Bus <Informatik>
Wort <Informatik>
Term
Objekt <Kategorie>
Server
Web Site
Subtraktion
Ortsoperator
Programmverifikation
Sprachsynthese
Kartesische Koordinaten
Diskrete Gruppe
Mathematische Logik
Computeranimation
Homepage
Widget
Skript <Programm>
Skript <Programm>
Zeiger <Informatik>
Schnittstelle
Softwaretest
Kraftfahrzeugmechatroniker
Architektur <Informatik>
Datenmodell
Paarvergleich
Paarvergleich
Fuzzy-Logik
ATM
Client
Server
Wort <Informatik>
Computerarchitektur
Partikelsystem
Schlüsselverwaltung
RPC
Blackbox
Formale Sprache
Schlussregel
Quellcode
Programmierumgebung
E-Mail
Computeranimation
Open Source
Generator <Informatik>
Knotenmenge
Körper <Physik>
Datumsgrenze
Widget
Widget
Operations Research
E-Mail
Term
Schnittstelle
Retrievalsprache
RPC
Schreiben <Datenverarbeitung>
Bridge <Kommunikationstechnik>
Computeranimation
Softwaretest
Code
Fahne <Mathematik>
Protokoll <Datenverarbeitungssystem>
Skript <Programm>
Auswahlaxiom
Phasenumwandlung
Gerade
Schnittstelle
Softwaretest
Namensraum
Sichtenkonzept
Oval
Schlüsselverwaltung
Krümmung
Kategorie <Mathematik>
Ruhmasse
Debugging
Ausnahmebehandlung
Ereignishorizont
Motion Capturing
Dienst <Informatik>
Rechter Winkel
Zahlenbereich
Festspeicher
Server
Objekt <Kategorie>
Subtraktion
Stabilitätstheorie <Logik>
Systemzusammenbruch
Diskrete Gruppe
Systemplattform
Viewer
Open Source
Informationsmodellierung
Reelle Zahl
Flächentheorie
Widget
Datentyp
Hash-Algorithmus
Widget
Skript <Programm>
Videospiel
Protokoll <Datenverarbeitungssystem>
Vererbungshierarchie
Systemplattform
Elektronische Publikation
Portabilität
Körper <Physik>
Debugging
Bridge <Kommunikationstechnik>
Mittelwert
Resultante
Bit
Punkt
Prozess <Physik>
Adressraum
Regulärer Graph
Kartesische Koordinaten
Systemzusammenbruch
Login
Softwarekonfigurationsverwaltung
Videokonferenz
Umwandlungsenthalpie
Font
Datumsgrenze
Bildschirmfenster
Volumenvisualisierung
Visualisierung
Kurvenanpassung
Funktion <Mathematik>
Umwandlungsenthalpie
Nichtlinearer Operator
ATM
Physikalischer Effekt
Benutzerfreundlichkeit
Plot <Graphische Darstellung>
Programmierumgebung
Ein-Ausgabe
Spannweite <Stochastik>
Mustersprache
Funktion <Mathematik>
ATM
Phasenumwandlung
Kategorie <Mathematik>
Refactoring
Rückkopplung
Total <Mathematik>
Ortsoperator
Gruppenoperation
Virtuelle Maschine
Zahlenbereich
Sprachsynthese
Streuungsmaß
Kombinatorische Gruppentheorie
E-Mail
Term
Data Mining
Histogramm
Physikalisches System
Message-Passing
Mittelwert
Mini-Disc
Inverser Limes
Operations Research
Bildgebendes Verfahren
Hilfesystem
Basisvektor
Meta-Tag
Touchscreen
Elektronische Publikation
Graph
Zeitabhängigkeit
Namensraum
Rechenzeit
Kontinuierliche Integration
Paarvergleich
Mailing-Liste
Paarvergleich
Sichtenkonzept
Gerade
Quick-Sort
Objekt <Kategorie>
Druckertreiber
Flächeninhalt
Term
Softwaretest
Arithmetisches Mittel
Punkt
Systemaufruf
Computeranimation
Entscheidungstheorie
Softwaretest
Bit
Punkt
Momentenproblem
Natürliche Zahl
Automatische Handlungsplanung
Regulärer Graph
Computeranimation
Programmfehler
Kernel <Informatik>
Richtung
Eins
Unendlichkeit
Druckverlauf
Programmfehler
Suite <Programmpaket>
Framework <Informatik>
Bereichsschätzung
Last
Skript <Programm>
Inverser Limes
Richtung
Aggregatzustand

Metadaten

Formale Metadaten

Titel Keeping your tools safe IDA
Untertitel Lightning Talks
Serientitel REcon 2017 Brussels Hacking Conference
Teil 07
Anzahl der Teile 20
Autor Diederen, Arnaud
Lizenz CC-Namensnennung 4.0 International:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/32399
Herausgeber REcon
Erscheinungsjahr 2017
Sprache Englisch
Produktionsort Brüssel

Inhaltliche Metadaten

Fachgebiet Informatik

Ähnliche Filme

Loading...