recheck und der Stein der Weisen
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Untertitel |
| |
| Serientitel | ||
| Anzahl der Teile | 94 | |
| Autor | ||
| Lizenz | CC-Namensnennung 4.0 International: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/45647 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FrOSCon 201930 / 94
7
10
12
14
15
16
17
20
22
23
26
27
28
29
30
31
32
33
36
37
38
39
41
44
45
48
52
53
55
56
57
58
60
61
62
64
66
67
74
76
77
78
79
81
83
84
93
00:00
SeidelOpen SourceFreewareDemo <Programm>XMLComputeranimationVorlesung/Konferenz
00:37
EmulationTreiber <Programm>AusnahmebehandlungZeichenketteWeb-SeiteZählenSpezielle unitäre GruppeLokales MinimumHilfesystemProgrammierumgebungSoftwaretestBildschirmfensterFontDatumsgrenzeVerhandlungs-InformationssystemWurzel <Mathematik>MenütechnikQuellcodeLeckMathematikSoftwaretestStellenringBildschirmfensterBildschirmmaskeResultanteTreiber <Programm>E-MailCASE <Informatik>Graphische BenutzeroberflächePunktWeb-SeiteBrowserElektronische PublikationElektronisches ForumElement <Gruppentheorie>Computeranimation
02:29
SichtenkonzeptLesezeichen <Internet>DialektOvalSoftwaretestZeichenketteWeb-SeiteTreiber <Programm>AusnahmebehandlungFontMehrrechnersystemCALIBAN <Programm>Web-SeiteElement <Gruppentheorie>TouchscreenDemo <Programm>Computeranimation
03:08
SoftwaretestSoftwaretestProzessautomationWiderspruchsfreiheitProgrammfehlerKontrollstrukturDemo <Programm>SoftwareVersionsverwaltungJSONXMLVorlesung/Konferenz
03:41
SoftwaretestVersionsverwaltungKontrollstrukturCodePortscannerIntegriertes InformationssystemMathematikSoftwaretestLineare RegressionProgrammfehlerHilfesystemSoftwareRechter WinkelEinsCodeDynamisches SystemInformationKonfigurationsraumVersionsverwaltungLaufzeitsystemLaufzeitfehlerInstantiierungXMLComputeranimation
04:58
SoftwaretestVersionsverwaltungKontrollstrukturCodeMathematikOrdnung <Mathematik>SoftwareHydrostatikSoftwaretestVersionsverwaltungLaufzeitfehlerComputeranimationXML
05:34
ZeichenketteVererbungshierarchieSinguläres IntegralEin-AusgabeStreaming <Kommunikationstechnik>SoftwaretestAusnahmebehandlungWurzel <Mathematik>VersionsverwaltungTransformation <Mathematik>SoftwaretestKomponententestElement <Gruppentheorie>Wort <Informatik>Message-PassingComputeranimation
06:43
ProgrammverifikationSoftwaretestEINKAUF <Programm>ZeichenketteSinguläres IntegralEin-AusgabeStreaming <Kommunikationstechnik>AusnahmebehandlungSoftwaretestProgrammierspracheProgrammbibliothekFormale SpracheInstantiierungProgrammverifikationComputeranimation
07:18
VersionsverwaltungWurzel <Mathematik>VererbungshierarchieSoftwaretestKontrollstrukturIdeal <Mathematik>SoftwaretestVersionsverwaltungMathematikIdeal <Mathematik>Klasse <Mathematik>DifferenteLoginSchlussregelInstantiierungPunktRechter WinkelSuite <Programmpaket>Software Development KitComputeranimationXML
08:58
MathematikSoftwaretestTouchscreenComputeranimation
09:49
Ideal <Mathematik>Graphische BenutzeroberflächeFirewallInstantiierungWort <Informatik>DatenloggerBenutzerbeteiligungInterface <Schaltung>Vorlesung/KonferenzXML
10:35
Elektronische PublikationRegulärer Ausdruck <Textverarbeitung>InstantiierungKlasse <Mathematik>LoginDatenloggerInternetworkingElektronische PublikationAttributierte GrammatikMathematikSoftwaretestWeb SiteBenutzerbeteiligungComputeranimation
11:18
SoftwaretestRegulärer Ausdruck <Textverarbeitung>BildschirmfensterOvalTreiber <Programm>AusnahmebehandlungZeichenketteWeb-SeiteGraphische BenutzeroberflächeAppletProgrammierumgebungSichtenkonzeptLesezeichen <Internet>Element <Gruppentheorie>DialektFontDatumsgrenzeMathematikTypentheorieSoftwaretestBildschirmmaskeDivisionVerschlingungFontInstantiierungRegulärer Ausdruck <Textverarbeitung>Attributierte GrammatikComputeranimationFlussdiagramm
12:47
DigitalfilterWeb SiteMathematikFilter <Stochastik>Inhalt <Mathematik>InstantiierungDifferenteDemo <Programm>JSONXML
13:27
PasswortBildschirmmaskeLoginSichtenkonzeptLesezeichen <Internet>SALEM <Programm>IRIS-TDemo <Programm>KreisringGraphische BenutzeroberflächeOpen SourceWeb SiteDienst <Informatik>Plug inComputeranimation
14:02
PasswortBildschirmmaskeLoginLesezeichen <Internet>PaarvergleichSALEM <Programm>Demo <Programm>HilfesystemSichtenkonzeptBewegungsunschärfeDatenmissbrauchCookie <Internet>TermAttributierte GrammatikErwartungswertElement <Gruppentheorie>Inhalt <Mathematik>Suite <Programmpaket>SoftwaretestDualitätstheorieCliquenweiteVorzeichen <Mathematik>FontCulling <Computergraphik>MathematikInklusion <Mathematik>PASS <Programm>PaarvergleichGraphische BenutzeroberflächeVerkehrsinformationDifferenteMathematikInstantiierungGraphfärbungFontKlasse <Mathematik>Computeranimation
15:08
DatenmissbrauchCookie <Internet>TermLesezeichen <Internet>PasswortBildschirmmaskeLoginSichtenkonzeptSinusfunktionSALEM <Programm>FlächeninhaltSpieltheorieDemo <Programm>WinkelInstantiierungRechter WinkelMathematikVerkehrsinformationComputeranimationFlussdiagramm
15:43
Vorzeichen <Mathematik>PASS <Programm>Attributierte GrammatikErwartungswertElement <Gruppentheorie>Inhalt <Mathematik>Demo <Programm>Suite <Programmpaket>SoftwaretestInklusion <Mathematik>StellenringCliquenweiteFontCulling <Computergraphik>Innerer PunktE-MailMathematikKommandospracheMathematikFilter <Stochastik>SoftwaretestGreen-FunktionPixelInhalt <Mathematik>InstantiierungKlasse <Mathematik>GraphfärbungSchlussregelOrtsoperatorComputeranimation
16:47
BildschirmfensterBildschirmmaskeLoginPasswortLokales MinimumSichtenkonzeptLesezeichen <Internet>Ideal <Mathematik>ViereckVerschlingungLaufwerk <Datentechnik>DialerDoS-AttackeGleichheitszeichenDualitätstheorieMathematikSoftwaretestDigitale VideotechnikPixelDifferenz <Mathematik>Rechter WinkelGleitendes MittelVerkehrsinformationElement <Gruppentheorie>PaarvergleichVorlesung/KonferenzBesprechung/InterviewComputeranimationXML
17:41
SichtenkonzeptLesezeichen <Internet>DatenmissbrauchCookie <Internet>Suite <Programmpaket>Inhalt <Mathematik>ErwartungswertElement <Gruppentheorie>Inklusion <Mathematik>SoftwaretestAggregatzustandStellenringFontCliquenweiteGerade ZahlAttributierte GrammatikArbeit <Physik>VerkehrsinformationGraphische BenutzeroberflächeMathematikMereologieRandverteilungFontZweiComputeranimation
18:21
MathematikVerschiebungsoperatorSoftwaretestLineare RegressionVisualisierungCASE <Informatik>InstantiierungPixelWeb SiteSchlussregelNegative ZahlDifferenz <Mathematik>XMLFlussdiagramm
19:26
Suite <Programmpaket>MinimumVorzeichen <Mathematik>PasswortSoftwaretestErwartungswertHecke-OperatorAttributierte GrammatikHyperbelverfahrenPaarvergleichInformationsspeicherungElement <Gruppentheorie>Differenz <Mathematik>Web SiteBenutzerbeteiligungCodeMathematikKette <Mathematik>InstantiierungKlasse <Mathematik>GraphfärbungVorlesung/KonferenzComputeranimation
20:14
Deterministischer ProzessQuellcodeInterface <Schaltung>SicherungskopieMereologiePhysikalisches SystemResultanteSpeicherabzugCASE <Informatik>Offene MengeFramework <Informatik>Open SourceDifferenteSchlussregelDienst <Informatik>Demo <Programm>Interface <Schaltung>GeradeGraphische BenutzeroberflächeComputeranimation
21:15
Produkt <Mathematik>Web-SeiteInformationSoftwaretestRückkopplungAusnahmebehandlungTreiber <Programm>ZeichenketteSoftwaretestElement <Gruppentheorie>InformationMathematikTypentheorieWeb SiteRechter WinkelComputeranimationXMLFlussdiagramm
22:20
SoftwaretestElement <Gruppentheorie>Attributierte GrammatikSystemidentifikationTablet PCSurjektivitätSuite <Programmpaket>SoftwaretestMatchingElement <Gruppentheorie>Wrapper <Programmierung>Funktion <Mathematik>XMLFlussdiagrammDiagrammComputeranimation
22:57
HilfesystemLokales MinimumProgrammierumgebungFeuchtigkeitAttributierte GrammatikElement <Gruppentheorie>ParametersystemDialektLogischer SchlussInformationE-MailCOMUnendlichkeitWurzel <Mathematik>Innerer PunktDiskrete-Elemente-MethodeDefaultKategorie <Mathematik>Hecke-OperatorRahmenproblemWeb SiteDifferenz <Mathematik>Konvexe HülleBellmansches OptimalitätsprinzipDatumsgrenzeAppletSystemidentifikationSoftwaretestRechenschieberMathematikKontrollstrukturComputeranimationProgramm/QuellcodeXML
23:40
TaupunktUnendlichkeitInformationSichtenkonzeptSuite <Programmpaket>SoftwaretestAttributierte GrammatikErwartungswertFontWurzel <Mathematik>Diskrete-Elemente-MethodeRahmenproblemKomponente <Software>DualitätstheorieInformationsmanagementSystemidentifikationElement <Gruppentheorie>SchlussregelDifferenz <Mathematik>Hill-DifferentialgleichungProgrammierumgebungPROMDefaultEmulationInnerer PunktMenütechnikArithmetisches MittelRechenwerkBildschirmsymbolSIMA-DialogverfahrenLokales MinimumEinfügungsdämpfungVakuumFeuchtigkeitInstantiierungMathematikSoftwaretestKontrollstrukturElement <Gruppentheorie>ComputeranimationXML
24:31
PixelInformationSystemidentifikationSoftwaretestRelationentheorieElement <Gruppentheorie>MathematikFlussdiagramm
25:06
CodeFlächeninhaltAttributierte GrammatikMatchingElement <Gruppentheorie>DiagrammFlussdiagramm
25:50
Demo <Programm>CodeTreiber <Programm>Element <Gruppentheorie>Attributierte GrammatikSystemidentifikationInstantiierungPunktURLRechter WinkelJSONXMLUMLComputeranimation
26:36
Attributierte GrammatikCliquenweiteE-MailTreiber <Programm>Element <Gruppentheorie>Suite <Programmpaket>SoftwaretestFontHilfesystemInformationsmanagementMenütechnikInnerer PunktProgrammierumgebungErneuerungstheorieMIDI <Musikelektronik>GammafunktionZeichenketteLokales MinimumSchlussregelAttributierte GrammatikSoftwaretestMathematikKontrollstrukturWeb-SeiteXMLComputeranimation
27:31
Treiber <Programm>BildschirmfensterZeichenketteFontChatten <Kommunikation>Gesetz <Physik>MIDI <Musikelektronik>E-MailWechselseitige InformationAttributierte GrammatikCliquenweiteSoftwaretestStochastische AbhängigkeitVersionsverwaltungDifferenz <Mathematik>SoftwaretestDatenunabhängigkeitSchreiben <Datenverarbeitung>MathematikSoftwareProdukt <Mathematik>VersionsverwaltungInstantiierungElement <Gruppentheorie>MultiplikationsoperatorDifferenz <Mathematik>LastPhysikalisches SystemMailing-ListeDifferenteComputeranimationJSONXMLFlussdiagramm
28:41
Framework <Informatik>SoftwaretestKnotenmengeTreiber <Programm>ZeichenketteSimulated annealingAusnahmebehandlungWeb-SeiteGesetz <Physik>BildschirmfensterProgrammierumgebungLokales MinimumHilfesystemSynchronisierungSpezielle unitäre GruppeSoftwareVersionsverwaltungMultiplikationsoperatorSoftwaretestBitRechter WinkelKontrollstrukturVorlesung/KonferenzXMLComputeranimation
29:19
AppletSoftwaretestAusnahmebehandlungZeichenketteTreiber <Programm>Element <Gruppentheorie>ProgrammierumgebungLesezeichen <Internet>Web-SeiteBildschirmfensterQuellcodeLokales MinimumSoftwaretestOrdnung <Mathematik>Treiber <Programm>InstantiierungPunktKugelkappeFramework <Informatik>MultiplikationsoperatorComputeranimation
31:06
Lesezeichen <Internet>Elektronische PublikationTreiber <Programm>SoftwaretestAusnahmebehandlungZeichenketteBildschirmmaskeWeb-SeiteProgrammierumgebungSoftwaretestElektronische PublikationMultiplikationsoperatorComputeranimation
31:39
SichtenkonzeptSchlussregelLesezeichen <Internet>SoftwaretestProgrammierumgebungTreiber <Programm>Graphische BenutzeroberflächeAusnahmebehandlungZeichenketteWeb-SeiteBildschirmfensterAttributierte GrammatikBetrag <Mathematik>AppletRegulärer Ausdruck <Textverarbeitung>Element <Gruppentheorie>FontElektronische PublikationMultiplikationsoperatorSoftwaretestInhalt <Mathematik>InstantiierungKlasse <Mathematik>Web SiteDifferenteElement <Gruppentheorie>FontCASE <Informatik>TouchscreenOrtsoperatorComputeranimation
32:55
Attributierte GrammatikMenütechnikElement <Gruppentheorie>FontProgrammierumgebungMathematikUmwandlungsenthalpieDivisionEin-AusgabeGruppenoperationp-BlockSkriptspracheFunktion <Mathematik>Web-SeiteElektronischer DatenaustauschAppletElektronische PublikationTreiber <Programm>SoftwaretestAusnahmebehandlungZeichenketteBildschirmfensterAttributierte GrammatikMathematikSoftwaretestE-MailInstantiierungKlasse <Mathematik>Web-SeiteSpielkonsoleElektronische PublikationWeb SiteDifferenteTouchscreenElektronischer ProgrammführerElement <Gruppentheorie>Computeranimation
34:32
ProgrammierumgebungSoftwaretestAttributierte GrammatikSystemidentifikationElement <Gruppentheorie>RahmenproblemKomponente <Software>Differenz <Mathematik>VektorrechnungInformationElektronische PublikationWeb SiteFehlermeldungFunktion <Mathematik>Kategorie <Mathematik>AusnahmebehandlungBildschirmfensterZählenE-MailQuellcodeMathematikE-MailIdentifizierbarkeitCLIZweiSoftwaretestGraphische BenutzeroberflächeProgramm/QuellcodeComputeranimation
35:09
Demo <Programm>ProgrammierumgebungRahmenproblemDifferenz <Mathematik>Komponente <Software>Attributierte GrammatikElement <Gruppentheorie>Web SiteSoftwaretestSystemidentifikationE-MailElektronische PublikationQuellcodeFunktion <Mathematik>AppletKategorie <Mathematik>AusnahmebehandlungMathematikSoftwaretestCASE <Informatik>Web-SeiteVorlesung/KonferenzComputeranimation
35:53
PaarvergleichBildschirmmaskeFunktion <Mathematik>Kategorie <Mathematik>AusnahmebehandlungBildschirmfensterFramework <Informatik>SoftwaretestKnotenmengeTreiber <Programm>Demo <Programm>MaßerweiterungGraphische BenutzeroberflächeDigitalfilterAbelsche KategorieInhalt <Mathematik>Stochastische AbhängigkeitSoftwarewartungOpen SourceMaßerweiterungGraphische BenutzeroberflächeDienst <Informatik>SoftwaretestMereologieComputeranimationVorlesung/Konferenz
36:55
SoftwaretestStochastische AbhängigkeitSoftwarewartungQuaderAttributierte GrammatikFormale SpracheMathematikProgrammierspracheSoftwareSoftwaretestMaßerweiterungVersionsverwaltungNormalvektorInstantiierungGraphische BenutzeroberflächePunktBrowserAppletVerkehrsinformationElektronische PublikationWeb SiteDifferenteElement <Gruppentheorie>EindringerkennungMultiplikationsoperatorRechter WinkelDienst <Informatik>BenutzerbeteiligungWeb-ApplikationWellenpaketLastExpandierender GraphWeb-SeiteDokumentenserverSoftware Development KitComputeranimationVorlesung/Konferenz
41:26
Suite <Programmpaket>SoftwaretestArbeit <Physik>Attributierte GrammatikErwartungswertDifferenteElement <Gruppentheorie>TouchscreenBenutzerbeteiligungComputeranimationVorlesung/Konferenz
42:14
Suite <Programmpaket>SoftwaretestStochastische AbhängigkeitSoftwarewartungBildschirmmaskeLokales MinimumMaßerweiterungGraphische BenutzeroberflächeInformationsspeicherungBenutzerbeteiligungRückkopplungProjektive EbeneVersionsverwaltungComputeranimationVorlesung/Konferenz
43:18
Open SourceFreewareXMLComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:07
Thanks for coming. So, early, it's the first talk today. I'm glad you're all here. And what I'm going to talk about is ReCheck and why we internally call it the Sorcerer's Stone.
00:22
Because it can turn Selenium into adamantium, so it can help you make your tests unbreakable. And I want to give a short introductory demo of that, just so you know what you have to expect. Oh, and I changed. Of course, I changed something here.
00:44
OK, so this is a typical Selenium test. Can you all see it? So you have a setup method. Here, I used a local HTML file, because I didn't want to depend on Wi-Fi. But then you find an element, in that case, by its ID.
01:01
And you send it some keys. And at some point, you press Submit. And then you check a result. So it's a fairly typical, very small Selenium test. And if executed, you will see that a browser window pops up.
01:22
So here's the Chrome browser. And as you can see, it entered something into the form. Press Submit, and the test is green. However, the problem is if I now change that page. So this is the form page that was called.
01:41
Here is the ID that was being called email. If I change that to something else, or if I completely delete it, then when I execute the test, it will fail. Because what it does is it identifies the element
02:01
it wants to interact with the ID. The ID changed or isn't there anymore. So obviously, the test has to fail. However, if you use recheck, just wrapping your driver, so as you see, it's the same test overall. If you execute that test, it will now not fail.
02:25
And this is some magic. Yes, it makes sense. And I will show you why. So as you can see, it still can interact with the web page. It still can find the element. It identifies them correctly, executes, and is green.
02:44
And this is what I will be talking about, how we achieved that, and how you can do too. So I got you all interested. Very good. No, no, no.
03:00
I will come to that. OK. So this is what I will be talking about, how to make your tests unbreakable. And this was a teaser demo. First of all, I want you all to think about how automated tests are not actually tests, because an automated test
03:20
doesn't tell you about inconsistencies. It doesn't tell you about bugs that are already in the software. An automated test only makes sure if you change something here, then this doesn't break. This is what automated testing gives you. And in that regard, it's not a test. In that regard, it's more like a version control.
03:43
So you use an automated test to make sure that you get notified about changes. If there is a change where you have an assert, you could still say, oh, that was actually a correct change, an intended change. I'll update the assert. So the test automation doesn't help
04:01
you find bugs in the software. It only guards against introducing new ones. That's why it's called regression test, right? And what you typically do when you use version control, for instance, if you use Git, Git tells you where the code has changed. So it tells you, here is your test automation.
04:21
Here's your configuration. That changed. But unfortunately, there's more to software than code, because software consists of the dynamic runtime information. This is what your users see, right? This is what you write. This is what you create.
04:41
This is what your users see. And for that, you need data. You need a runtime system. You need much more than just the code. And this is not version controlled by Git. This is. So there's a gap. There's a problem there. Because what you want to know about is where has the software changed.
05:04
But as I just said, Git tells you where has the code changed. And this is what we use tests for. Tests close that gap. Tests turn the dynamic runtime of the software into a test, into a static artifact, into code that you can govern then by version control.
05:23
This is how we currently close the gap. This is why you have to execute the test in order to do that. So, as I said, automated tests are really version control. And now we'll have a look at a typical test, like at a unit test. So you have, this is a test that tests an XML transformation.
05:46
So you have an XML that goes in. And you have a remove element transformer that transforms that XML. And you want to make sure that the remove is gone and that the keep is kept. So, in other words, this is what you put in. The above XML is what you put in.
06:02
Then the test executes. And this is what should get out, right? So this says, okay, the keep tag is still there. The remove tag is removed. A fairly typical test. However, the problem is, the way the test is written, if you execute it, it will pass.
06:20
And this is not even valid XML. But it fulfills all the assertions. Because it just says, keep is still there and remove is not. So this test passes. Which is bad, right? This is not what we want. We want that only that XML should pass. We want that this complete thing be verified.
06:43
And this is actually what is called gold master testing, derived from the LPs where you had an audio recording that was a gold master and all else was copied from that. And there are even existing libraries for that. For instance, approval test is very fairly well known
07:01
because it's available in almost any language, a programming language. So what this gives you is, you now can change your test and say approvals, verify XML, and then you transform that to string. And it makes sure that this test will pass
07:22
only if the XML looks like that. And really only. So it checks for every whitespace character even. So as I said, automated tests are version control. Then, if an automated test is version control, an assert is a blacklist of a change.
07:40
Okay, so you say this, for instance, the remove tag has to be there unless I update the assert. So I specify a single rule that is checked and instead gold master testing is whitelisting of changes. So you say, you check, so imagine you execute a test without any assertion, right?
08:03
Then you check nothing. If you don't put any rules there. If you don't ignore anything, then with gold master testing, you check everything. And obviously, there are some things that you don't want to check, like with Git, you have also logs and classes and target folder and stuff like that.
08:21
So there are always stuff that you want to ignore. And you can do that with those rules. And there's an ideal amount of checks. So you want to put some asserts there. It doesn't make sense to have a test without any assertions. So you put some asserts or you ignore some stuff. So at some point, you end up somewhere in the middle.
08:43
But the thing is, imagine what happens if you forget to ignore something or imagine what happens if you forget to assert something, right? That's a very vast difference. And it's even better because you can't check
09:00
for things that you don't expect. So this is a problem that they had at Google. At Google, apparently, they mark changes for testers. So they have manual testers. If you're a manual tester at Google, you're supposed to test the things that have a dancing unicorn next to them more thoroughly than the rest because those things have been changed.
09:22
However, someone forgot to remove the marking and the customers ended up having dancing unicorns on their screen because they didn't have a test that checked whether there are dancing unicorns on the screen. So if you don't know what to check for, if you don't expect a change,
09:41
then you can't assert for it, right? So the question is, where do you err? You don't wanna err, you want to err on the side of caution. In other words, blacklisting, for instance, a firewall is a bad idea. You wanna, if you create a firewall, you wanna blacklist and say this port is free.
10:02
And also, what we learned when we did this is that typically, you want to check rather more than less. So it makes sense to start from here because you have less work to ignore individual things. Okay, so what we created with Retrack essentially is a Git for the GUI.
10:22
Actually, it's more because you can use it even, you can implement it for XML or log files or anything. So you could even say it's a Git for any interface, but we concentrate it now. For now, we concentrate it on web. So it's a Git for the GUI. And with Git ignore,
10:41
as I just said, you ignore stuff. So you give it rules, you ignore log files, class files, and everything in target, for instance. And with the golden master, for instance, with Retrack web, you wanna ignore, for instance, class attributes. You say, okay, I don't care if I check a website, I don't care about the class attribute.
11:02
Or you can ignore with a regex. You can say, okay, I wanna ignore every change. And actually, this is what I did. So I cheated on the example, which was remarked in the audience already. So that test shouldn't have passed earlier
11:20
because what I did is I just ignored every change, which probably is not a good idea, so it just let me remove that. Now, if I re-execute the test, it will still execute. It just will not pass because it will say there are changes, but it will still correctly execute.
11:43
It fills in the form correctly, but now it fails, saying there are changes. This is what you want. But obviously, for a teaser, it looks better if it passes.
12:04
So you can ignore changes, attribute changes, based on regex. And on type, for instance, you can say, I wanna ignore the link type, the link tag, and so ignore every link tag.
12:21
Or via XPath or ID, you can say, I ignore everything that has the ID, and there should be only one thing, but anyway. Or ignore everything on a specific XPath. And of course, you can combine it. So you can say, ignore every font for a div with an ID, such an ID, whatever.
12:43
So you can combine the stuff for XPath or whatever. And we implemented something we call filter, because if you don't use any ignore at all, and you change the website,
13:00
it often happens that you get a lot of changes, and it's then hard to understand which changes caused what. So for that, we implemented filter, which worked by the same syntax. So you can have different filters and say, okay, for now, I wanna ignore all CSS changes and only know what content changed, for instance.
13:22
So, and then I wanna, again, demo. So, ah, and by the way, to give you a better impression, we created a Chromium plugin to test this. The plugin itself is also open source, but it uses services from us, so these services are, well.
13:45
So here are two websites that are fairly similar, but not quite, as you can see. So I'll just use the plugin, get my, ah, right, create my gold master.
14:03
Then I have another site, and I wanna compare against that. So I wanna compare. Now my difference report was created, and now I can open that report either with a CLI or with a GUI.
14:20
I will now open it with a GUI because it's much more visual. So as you can see, there are a couple of changes, and you see everything that changed. So you see, there's a change in outline, there's a change in class, apparently it was before button primary, now it's button secondary, some color changes,
14:42
and stuff like that, and even text. So yeah, you see, for instance here, you see that the text now has a semicolon at the end. So this is, it gives you all the changes, and as I said, you can ignore what you don't care about. So if you have it like that,
15:00
then you immediately see that the text has changed, not only the button, but also the text. And this is fairly powerful, actually, because you can now, for instance, compare those two. And if you have a look at that, it looks very different, right, from one another. As you can see, there are a lot of changes. So let's just see what happens.
15:26
We create, we create a gold master, and then we compare, gives us a new report,
15:42
we open the report, and now, as I said, we now have a lot of changes, right, because we changed very much. But we can just say, okay, ignore everything that is related to positioning. Like if something is five pixel to the left, ignore that. We can say ignore everything that is related to style.
16:02
For instance, blue versus green color, everything like that. And we can say ignore everything that is invisible, like ID, CSS class, all of that. And so what you end up with is just that. And as you can see, these are the actual changes.
16:22
So here, I changed the text from $115 to $120, and here I changed the text from 50 to 45. So this immediately drills down to what you care about. And you can use those filters as ignore rules.
16:42
So you can say my test should pass even with such substantial changes, my test should pass only if the content changes. Oh, sorry, should fail only if the content changes. And what is even more interesting,
17:00
how would you, like, there are tools, what typically people use for that is pixel diffing, right? You can, as I just said, you can use a pixel diff for the change of the, for this change already, pixel diffing doesn't work. But how do you handle moving elements? So you can pixel diff that, right?
17:21
So, I mean, it changed, so what? But with recheck, it works, because you just say, okay, create a diff animation. I prepared that because, so everything is already existing. And now I say compare.
17:41
And there's, again, there's a report. I open that with the GUI. And now, again, okay, it says the outline and the margin changed, but I can say, okay, ignore that, I don't care about outline and margin. And then I see, oh, the animation delay changed.
18:03
Now the animation started 0.3 seconds later, and the text changed. There was that part of the text that was deleted. Great. So, okay, typically, so what you use typically
18:26
for that visual regression testing is a pixel diff. And I would say the market leader there is Aplitools, which is, as far as I know, currently the largest company. And they use AI to give you, to filter non-relevant changes.
18:42
So in that case, for instance, I'm sorry, the whole website shifts five pixel. Aplitools can filter that and say, oh, this is a whole change of the whole website. And in reality, nothing changed. So Aplitools uses AI for that. But in that case, he has a false negative because the semicolon was added,
19:03
and AI thought it was an artifact and didn't mark it. And on the other hand, this one didn't really change. It only changed because the button changed in size. And so you have a false positive and a false negative. And as you can see, it only gives you pixel diff.
19:21
So you have to manually review that and have a look at that and you can't write any rules for that. So you can't say ignore whatever. So you can't handle that. What Retake Web, on the other hand, gives you is a semantic diff. So we capture every CSS element of, as every CSS attribute of every CSS element
19:42
on the rendered website, and then store that and compare that later on. So instead, we can even say the class changed. Okay, something that you don't see. If you don't have a look at the code itself, you don't see what changed. We say the class changed. And we say the border, the color changed
20:02
and stuff like that. So you can, as I just showed you, you can write rules for that. You can say, I wanna ignore class changes, but I'm interested in color changes, for instance. Yeah, these are just backups in case the demo wouldn't have worked.
20:22
So yeah, as I just said, you have rule-based, deterministic ignore. It's open source, freely available. And what I just showed you, the GUI, this is the part, and the service behind it is the part where we make money from. But the core technology, the Retake framework
20:40
and the command line interface is fully open source and fully free. You can use it offline because you don't have, like with Applitools, it's a service. So all of your systems have to send their data to them. They analyze it and send back the result. You can use all of that offline. It gives you unbreakable Selenium. To that, I will come in a minute.
21:01
And it works in principle for any technical interface. Like it even works, you can implement it for XML, for JSON, for anything like that. And the difference rules and everything still work for that. So some people like it. But why you all came here?
21:23
What I was initially talking about is how we make the test actually unbreakable. How do we achieve that feat that Selenium can still execute, the test can still identify those elements even though the ID changed?
21:41
How did we do that? Well, the idea in principle is simple. We create a golden master anyway. So this is our type of testing, right? We create a golden master, and now we can edit the golden master. We can insert a virtual ID and then refer to that virtual ID.
22:01
So you can, oh actually that slide's too early. And those changes are unaffected by the actual change. So whatever is in there, whatever additional information, we add there, is not contained on the actual site, is not reflected on the actual site, is not affected by changes.
22:21
Which means that, I'm sorry that was too fast. Your test goes and says, okay, I want Selenium, give me that. We created a wrapper. Selenium says, I wanna use, I wanna search in the golden master for that. Then we create a one-on-one assignment
22:41
and return the best match. So we say, this is the element that is most closely what you're looking for, and then return that. And then we can even, ah, I didn't show that. We can even output a warning. So when I execute the test, that just failed.
23:02
I have warnings here. It says, okay, the golden master, the test would have broken if it was a typical test. Now you have to update your test or use our retest ID.
23:22
You see that? I don't know how to make that bigger, actually. No, it doesn't work. Okay, back to the slide. So we say, we guarded your test from breaking. Now if you apply those changes to the golden master, so we say, okay, look, in the old, ah, okay, sorry.
23:42
Here, oh, that also is too small, too bad. Okay, here we say what has changed, and for instance, we say that the ID has changed. Anyway, it's also too small. Sorry about that. I don't know how to make it bigger than Eclipse.
24:02
So we say, based on the golden master, we identified that element, and we saved your test from breaking. Now if you update the golden master and remove the ID also in the golden master, then we have no way of identifying the element anymore,
24:23
and then it will break. So it breaks only if you update the golden master accordingly to that change. And how does the identification work? Well, we have redundancy. First of all, we have redundancy.
24:41
If you say, give me the element by the ID and the ID changes, then you don't know which element to refer to, right? But in the golden master, we have redundancy anyway. We know the path of the element, we know the name of the element, the label of the element, whatever. So we have multiple, much more information
25:01
about the element than the typical test has. And, additional to that, we have the complete picture. Not only do we have all the identifying attributes of that individual element, but we know all other elements, so we can make a one-on-one assignment.
25:22
We can say, this is a button what we're looking for, that button is not there anymore, but this button is that, we're sure about that. There is no any other button. We can make one-on-one assignments of all of those elements, so this button has to be that. Even if the ID changed, if the label changed,
25:41
if the XPath changed, it still has to be that button because it's the highest match. All other elements are too different. So we find that element, I did that already, and we return it to Selenium, which then executes nicely. And on top of that, what that gives you on top of that,
26:05
is that you now can use the additional attributes that we put into the gold master for identification. For instance, instead of using the ID, you can now use a retest ID. So for instance, at some points, people use CSS locators.
26:22
Sometimes you just don't have the ID of an element. You don't have a nice, simple XPath, so you have to use something else. And what we can use instead is now the retest ID. So in that, this is gold master, gold master is stored in XML, by the way.
26:42
In there, you have an attribute that's called retest ID, and you can just say, I wanna use that retest ID instead. So I changed my test, I didn't do that before, I can do it now. I can change my test and say,
27:00
okay, that ID doesn't exist anymore, I deleted it, I can use the retest ID. And now if I update the test, it won't break. If I update the gold master behind the test, it won't break, because the ID is unaffected by the changes. The ID is not on the original page,
27:21
so applying the changes won't change the retest ID, so I can now safely update my test, and it won't break. And what the technology also gives you is a possibility to now write data-independent tests.
27:46
Because now what you can do is you can create a test that just clicks on the first element, whatever that element is, of the list, for instance. Then you write the test, you load new data, for instance, every night,
28:00
you load production data into your system, some people do that. Then you create a gold master, then you load the new version that you actually want to test, and then you create the diff. And then you see what changed between the two versions of the software that you want to test. And you can do it,
28:20
you can create your test data independently. You just have to execute the test twice, right? You have to execute it once to create the gold master, and then you have to execute it with a new version again to test against the gold master. The thing is you can't change both at the same time.
28:40
So you can't update the data and the software version at the same time. You have to do it one after another. So, I was a bit faster than I expected. Next thing I want to show you is how you turn an existing test
29:01
into an unbreakable test. So, this is the test I start with. This is the typical Selenium test, right? And this one breaks if I execute it because I here changed the ID. So let's undo that. Now I execute the test.
29:21
It should be green again. Okay. So, now in order to utilize ReCheck in that test,
29:41
you just have to, peak, sorry. You just have to use the ReCheck driver instead of the remote driver, and that one is fully compatible. So you can use it even underneath other test frameworks. So if you use Cucumber, for instance, if you use whatever, there are other test frameworks,
30:03
this is fully compatible. So you can mix ReCheck with any other test framework that utilizes Selenium. So you just wrap your original driver that you would use inside of the ReCheck driver, and then there are two more things that you have to say.
30:22
Start test, and quit. Now I can remove the assertion, by the way, because if I do that, it will create a gold master after every step, so I can remove the assertion. And I just say driver.cap.test.
30:43
So I tell it that the test has ended. These are the two things that I need to do. And I already have transformed my test into a ReCheck test. That's now unbreakable. So I remove it. The first time it will execute, it will fail, because it has to create the gold master.
31:17
So as I said, it failed, and here it says, no gold master found.
31:22
First time test was run. Don't forget to commit. So now the gold master files are created. Now I re-execute the test.
31:52
Okay. Okay. So if the test isn't green the first time you execute it,
32:01
you can just use that file, and this is the ReCheck file that I showed earlier. I don't know why there are differences. Maybe because of the BIMA.
32:21
So I just ignore the differences. I say, okay, I'm not interested in positioning, in that case, it's the outline, so where elements are on the website. I don't care about that, because I only want to know whether content is correct, for instance.
32:44
So now the test is green, as I just saw. If it isn't green the first time, you just use the ignore file, like you would with Git. If you set up Git, it says that your class files are not committed. Do you want to commit them? So you have to adapt the Git ignore file,
33:00
and say ignore class files. You have to do the same thing here. As you just saw, it's fairly easy. You just say, can I make that bigger? You just say, okay, these are the attributes I'm not interested in. Please ignore them globally, and then the test is green. And now, if I change the email, sorry,
33:22
on the actual page to something else, or I remove the age, then this test will also pass. And as I just said, what will happen is, it doesn't find the original element, so it goes into the gold master,
33:40
finds the element with the ID email in the gold master, creates a one-on-one assignment of old to new elements, and returns the new element from the website. So the test now is unbreakable in that regard, or in regard to that change. I mean, you can still break your test, obviously, if you change too much. Like if the button isn't there anymore,
34:03
like if it really isn't there anymore, then the test fails. But it still executes, and now it shows you all the differences. So here it says, for instance, that the ID, it's pretty small, but I think you can maybe see it, that the ID has changed from email to something else. If you wanna ignore that change,
34:21
you can say ignore in the ignore file, and here is the console. That says, okay, here's the warning, the ID used to identify change from age, okay, that's the second one,
34:41
change from age to null, because I deleted the ID, so the ID is not here anymore, and here it says the ID changed from email to something else. And now, it even tells you the retest ID. So it says, okay, either you have to use
35:00
a new ID in your test, so you have to, when you update the golden master, for example, using the CLI or the GUI, if you update the golden master and say delete the ID in the golden master, then your test will break. So it says either you have to update your test with that new ID, or you have to, you can use the retest ID
35:23
that won't change ever. So this one is virtually completely independent from the actual page. It won't change no matter what you do to the page. So you can use that instead.
35:41
And in the case where we deleted the ID, obviously it doesn't give you an alternative ID, it just says use the retest ID. Okay, this one is open source, as I said, and also the Chrome extension that I showed you is open source.
36:02
Where we make money from is the services that are behind the open source, the Chrome extension and the GUI. Because there are two, there's a command line interface that is free and the GUI we make money with. So, this essentially was the talk.
36:20
Are there any questions? Yes? Can you? You removed the explicit assertions in your test, so how would you really delete a button
36:41
if you want to delete a button? You have no way to tell that, yes, I really want to delete it. Or you have to update the golden master manually? Okay, I didn't show that part. So you can use the CLI. We, I didn't prepare that. I don't know if it works out of the box. So we have a command line interface,
37:00
like with Git, where you can say apply change. And then it applies the changes and updates the golden master. Or we have a GUI where you can simply say I want to update the golden master accordingly and update that. So you, it's just essentially two clicks. So you click, you have a check box
37:21
where you say this change update and then you have a button where you say apply and then your golden master file is updated. That was the question? Yeah, and just another question. The golden master, you committed that in your normal Git repository, just? So the golden master is essentially XML file
37:40
that you can just commit normally in your, like you would with any other file in your Git repository, right? Any other questions? The GUI, is that like software as a service or is it something you buy and can run?
38:02
So that's something you buy and can run locally. Okay. Right. Does it work with React or Angular? It should. I haven't specifically tested it with React or Angular. Ah, right now it's implemented in Java
38:20
and right now we focused the tests also to be implemented in Java. But a version that works with, like where you write the tests in JavaScript or PHP or something should come in the next months. So we're looking to four to eight weeks, I guess.
38:40
Because we shrinked it down essentially to a JavaScript file that's executed in the browser. Like you saw with Chromium, with the extension. So it doesn't really matter what the underlying programming language is. But for now we implemented it only in Java and other languages will follow.
39:01
Okay, thanks. Any other questions? Will it be, work if the ID dynamically changed everything every time the website is loaded? That means if you,
39:22
first time you load the site, IDs were created and next time, then you will create a gold master and from that point you can change the IDs every time the website loaded?
39:41
Yes, it works. So what you would have to make sure is that you know the ID that went into the gold master. If you use get element by ID, you obviously have to use the correct ID at that point to execute the test once, right? You have a problem essentially to execute the test.
40:02
Like how would you reference the elements the first time you execute the test? You can create the gold masters even with the Chrome extension and then use the retest ID of the gold master and then reference that in the test.
40:20
But other than that, it should work because you can ignore ID, it's just an attribute as any other attributes, you can ignore it. And you just have to reference the retest ID or XPath or something that works for you. Okay, great. Any other questions?
40:41
Yep, can you pass? Are there any experiences how this is working with larger dinosaur web applications from the wild? SAP for example, which is not very nice to test. We have with web pages,
41:02
we don't have a customer that's using it on very large websites. We have the same thing implemented in Java Swing. And for instance there we have someone, so just recently I've seen a report where there were checked 800,000 elements and there were 200 differences
41:20
and it takes like two minutes to drill down because you can just ignore, so you can say ignore hide all accepted or ignore differences. So everything that's green goes away, you only focus on the differences. So out of that 800,000 elements you get the 200 and then within two minutes you know what the problem is.
41:42
So I assume it should work the same way with web but unfortunately we don't have big customers that use it as extensively as those with Swing yet. Any other questions?
42:02
Okay, if there are no other questions then thanks for coming. If any other questions come up afterwards then please don't hesitate to contact me. So you can find us on the web retest. You can contact me directly or contact the company.
42:23
Please, we try it out. Like the Chrome extension isn't available in the web store yet. It will be on Tuesday because it's all very fresh. But the retake web project is working and is being used.
42:41
So please check it out, tell your friends and give us feedback. If anything doesn't work for you, if you have problems please tell us because we, as I said, this is fairly new. Retake web is now in version 1.3. Chrome extension is 0.8. So it's not publicly available in web store yet.
43:00
So this is all very fresh and your feedback helps us improve it further. And yeah, tell people you know so this gets more attention. Thank you very much.