3.1 Regression Testing

Video in TIB AV-Portal: 3.1 Regression Testing

Formal Metadata

3.1 Regression Testing
Title of Series
Part Number
Number of Parts
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
00:09 Regression Testing 00:20 You've published it! 00:50 What's next? 02:53 You've published it again! 03:29 Cautionary Tale 06:18 Guard against those errors! 12:19 Create test cases 13:56 Label test cases 15:09 Demo: app-fermi tests 21:44 Types of errors 22:15 Assignment #10: Create tests for your tool
Simulation Rational number Linear regression Linear regression Sheaf (mathematics) Water vapor Whiteboard Statistical hypothesis testing Statistical hypothesis testing
Point (geometry) Logical constant Linear regression Multiplication sign Equaliser (mathematics) Sheaf (mathematics) Maxima and minima Insertion loss Parameter (computer programming) Mereology Computer programming Statistical hypothesis testing Mathematics Different (Kate Ryan album) Endliche Modelltheorie Nichtlineares Gleichungssystem Regulator gene Interior (topology) Physicalism Computer simulation Bit Software Video game output Right angle
Point (geometry) Wechselseitige Information Linear regression Sequel Code View (database) Multiplication sign Coroutine First-order logic Set (mathematics) Dynamic random-access memory Computer programming Declarative programming Variable (mathematics) Value-added network Revision control Pointer (computer programming) Mathematics Different (Kate Ryan album) Analogy Endliche Modelltheorie Mathematical optimization Metropolitan area network Raw image format Email Simulation Wechselseitige Information Real number Chemical equation Computer program Interior (topology) Physicalism Menu (computing) Bit Basis <Mathematik> Coroutine Variable (mathematics) Statistical hypothesis testing Degree (graph theory) Particle system Word Gravitation Right angle Game theory Resultant
Building Multiplication sign 3 (number) Likelihood-ratio test Mereology Host Identity Protocol Law of large numbers Computer programming Statistical hypothesis testing Pointer (computer programming) Personal digital assistant Physical law Series (mathematics) Gamma function Conditional-access module Traffic reporting Newton's law of universal gravitation Metropolitan area network Information management Execution unit Demo (music) Suite (music) Building Statistical hypothesis testing CAN bus Error message Personal digital assistant Uniform resource name Computer cluster Right angle Figurate number Wide area network
Point (geometry) Statistical hypothesis testing Functional (mathematics) Statistics View (database) Multiplication sign Maxima and minima Perturbation theory Function (mathematics) Infinity Mereology Arm Law of large numbers Value-added network Statistical hypothesis testing Mathematics Goodness of fit Bit rate Different (Kate Ryan album) String (computer science) Summierbarkeit Physical system Task (computing) Area Metropolitan area network Execution unit Information management Standard deviation Linear regression Closed set Interior (topology) Menu (computing) Euler angles Inclusion map Process (computing) Personal digital assistant Uniform resource name Convex hull Hill differential equation Right angle Resultant
Software engineering User interface Law of large numbers Hand fan Emulation Statistical hypothesis testing Personal digital assistant Physical law Series (mathematics) Summierbarkeit Newton's law of universal gravitation Metropolitan area network Default (computer science) Matching (graph theory) Sine Suite (music) Witt algebra Building Statistical hypothesis testing Error message Personal digital assistant Uniform resource name Lie group Computer cluster Quicksort Wide area network
Computer file Sheaf (mathematics) Motion capture Device driver Directory service Run-time system Computer programming Statistical hypothesis testing Personal digital assistant Integrated development environment Series (mathematics) Descriptive statistics Area Information management Computer program Menu (computing) Bit Directory service Statistical hypothesis testing Symbol table Statistical hypothesis testing Software development kit Integrated development environment Personal digital assistant Vertex (graph theory) Normal (geometry) Resultant Electric current
Laptop Point (geometry) Group action MUD Mapping Line (geometry) Logarithm Demo (music) 3 (number) Maxima and minima Set (mathematics) Water vapor Function (mathematics) Value-added network Variance Number Statistical hypothesis testing Goodness of fit Uniformer Raum Radio-frequency identification Insertion loss Different (Kate Ryan album) String (computer science) Physical law Integrated development environment Addressing mode Summierbarkeit Error message Area Rule of inference Metropolitan area network Curve View (database) Computer file Interior (topology) Local area network Complete metric space Statistical hypothesis testing Software development kit Uniform resource locator Personal digital assistant Uniform resource name Right angle Resultant Booting
Point (geometry) Statistical hypothesis testing MUD Code Line (geometry) Demo (music) Maxima and minima Special unitary group Law of large numbers Field (computer science) Computer programming Emulation Statistical hypothesis testing Green's function Formal verification Process (computing) Endliche Modelltheorie Error message Computer icon Execution unit Standard deviation Link (knot theory) Validity (statistics) Interior (topology) Statistical hypothesis testing Inclusion map Personal digital assistant Uniform resource name Metric system
Point (geometry) Sine Demo (music) Execution unit Function (mathematics) Statistical hypothesis testing Statistical hypothesis testing Statistical hypothesis testing CAN bus Medical imaging Mathematics Error message Type theory Personal digital assistant Function (mathematics) Personal digital assistant Chain output output Summierbarkeit Error message Maß <Mathematik>
Group action Linear regression Multiplication sign Mathematical singularity Set (mathematics) Shape (magazine) Special unitary group Proper map Arm Computer programming Thomas Kuhn Exploratory data analysis Pointer (computer programming) Uniformer Raum Different (Kate Ryan album) Semiconductor memory Personal digital assistant Row (database) Library (computing) Descriptive statistics Metropolitan area network Texture mapping View (database) Linear regression Interior (topology) Branch (computer science) Menu (computing) Hidden Markov model Statistics Statistical hypothesis testing Statistical hypothesis testing Process (computing) Uniform resource name Phase transition Hill differential equation Summierbarkeit Resultant Booting Point (geometry) MUD Computer file Ant colony optimization algorithms 3 (number) Maxima and minima Perturbation theory Law of large numbers Emulation Value-added network Statistical hypothesis testing Number Revision control Mixture model Frequency Finite element method Radius Single sign-on Computer worm MiniDisc output Newton's law of universal gravitation Supremum Context awareness Execution unit Suite (music) Sine Directory service Inclusion map CAN bus Number Error message Moment of inertia Intrusion detection system Personal digital assistant Wireless Markup Language
Point (geometry) MUD Ant colony optimization algorithms Plotter Simultaneous localization and mapping Function (mathematics) Entire function Law of large numbers Computer programming Statistical hypothesis testing 19 (number) Different (Kate Ryan album) Physical law Descriptive statistics Metropolitan area network Default (computer science) Execution unit Standard deviation Link (knot theory) Sine Interior (topology) Hecke operator Directory service Line (geometry) CAN bus Liquid Personal digital assistant Uniform resource name Right angle Resultant Wide area network
Statistical hypothesis testing Email Suite (music) Code State of matter Multiplication sign 1 (number) Set (mathematics) Function (mathematics) Likelihood-ratio test Special unitary group Computer programming Electronvolt Duality (mathematics) Mathematics Negative number Metropolitan area network Simulation Link (knot theory) Linear regression Menu (computing) Variable (mathematics) Statistics Degree (graph theory) output Right angle Imaginary number Point (geometry) Software engineering Game controller 3 (number) Maxima and minima Statistical hypothesis testing Power (physics) Arithmetic mean Term (mathematics) Energy level Maize Summierbarkeit Traffic reporting Metropolitan area network Boolean algebra Execution unit Information management Standard deviation Stapeldatei Just-in-Time-Compiler Validity (statistics) Sine Eigenvalues and eigenvectors Line (geometry) CAN bus Loop (music) Voting Hypermedia Personal digital assistant Network topology Universe (mathematics)
while I back online simulation and more Fernando technology board for
this next section wanna talk about regression testing that you build tools you can set it up so that you can test the tool and warm water only when you publish until you can test it again and again and again so we
send the that you have proven out of the rationals together to build running in your workspace it works in a minute I'll show you how you can get published and then all so congratulations you got a tool that will go through all this business about how you install and 1 should test and finally published the tools the lot market
regulations you got to move apart so what's next well
what's next is once 2 was published people on an alternate way in which this is a an unusual but which is simulated gallium-arsenide Hamilton tool that we should have more options for while the little in different ideas things that they want to see if this which listening to me that loss of life viability neutral there's like you write software for writing books at the same time always so people will find problems and they're not going to take final nano and guess what if there's a problem here told you need to take it congratulations to the right so what write software their abundance and somebody's got and if you wrote tools you need to fix it so all of which you need you know the work nearly as it happens that your advisor will come along and say I that great like I can't believe asexuals overtly hassle ideas let's try and this equation was try adding that equation right so you're advisable this is accelerated trial and this the model and added to the model and you have to add a few more inputs so you be making changes no matter what you're going to be making changes due to bump reassure on the so let me tell you a story of that of that back in the day you are told you about that's equal program that I wrote back in the days when I was rescued all I have I got to this point and status in physics in the whole section of parameters here and in my physics in Chadian in all these different constants that I had a fixed up and can I have just a little bit of physics the intellectual part I wanted to republish traits so at some point when you have that in physics you'll you'll nuclear believe that this season's always vote for testing and approval and you want to republish to each and so OK
congratulations you publish your tool all over again so here is the story but you got this mutual balance different versions through the mail on the multiple versions of a lot of you you have never really had a 1 in the very latest and that's a lot and it's not always a working on this solution to that 1 and 1 is a lot of people use it is that different models that evolve people also use it in what people were in the verge of 1 so now now we have multiple versions of the tool try and
but getting back to my story is worse than the my sequel program with that thing is that he thought that was what that future playing Tetris game on so I have this program and I want to make a change of simple change relatively simple change onto the physics it turns out that there was a little bit of right particles were actually sort routine and again using the standard bubble called for this assignment was like and rules that in the various values this size greater than that time slot that's also companies wanting to variables we need to do is itself to the words 1st and 2nd so that you introduce some very right so you can you can slot values are so unintelligible on the basis of the set of fixed my program because of values like this and I started running again all of a sudden my program got this is our business flights on all the views of the funding bill occurred in the thing in every day and my my 1st was very excited because we it's raining on this slide of but we looked more carefully the code and it turns out no was actually about the behavior of everything's slightly bizarre because you can now out when I was doing my swapping here contemporary variable there was another local variable optimize the Declaration of all have actually is that the temperature of the simulation so at 7 levels of or might have variable was overriding the temperature of the simulation and instead of being at 4 degrees Kelvin it was going all over the place it was like result 107 thousand resulted and Sierra was going all over the place because of the simple 1 like of more than that of the that were those of you program but rather will variable and all hell broke so the point is that when you go to republish program you can all per prior to testing the everything will be all perfect you publish analog and then becomes an innovative I like this and then use that would put back up and it turns out that violence this is the completely horrible to the rest your program every time we go to publish relationship program again as if you were published in the 1st half of gravity which it all right and you really should you really should nobody does that it is it's a lot of work right you thinks about it he pointed out it's about put it out and you just want be done with it because you just 6 involves but really you should be checking everything very carefully each time
so but rapture has a solution for rational can solve the problem because it has testing tool and the way it works is you could you build up a we did test cases that 1st time through when you're building a program to a few extra minutes and build up much testing and say that and then you might not even need them again but then the book report comes in which some in whatever you met children are something just before you go to publish again you can run back through these test cases but is that you doing all work and taking over time you just push a button and rational do automatic rational over all the test cases that we started with injectors yeah still working they may not be broken and when broken and you can look at each of 1000 broken and figure out why it's broken in my case the temperature was varied wildly and getting strange physics out and I wish I had known that because I had a series of test cases I would seem right away simple test cases are not working anymore and I wanna talk here right away so let me show you how how it works but maybe that's the best way right allomorph flip and show you
try a demo of what it looks like part so if I got this program here my very
function calculated that and I can write by were engraved by go ahead republished the tool right fantastic everything's working if I run rapture dashed tester it'll bring up all
these test cases that I created before so I can all these different test cases that at 1 point worked perfectly for me now what I can do is say select all and run and this little regression caterwauling Bacchanals testing testing worse lower rate all went down all the way out there what regional testcases tries to replace the old results and where there was like a red X 1 that by the checkbox the fail in right next to the room all these tests and again this is what you do do that is well by hand and look at it and for any particular task if it's working fine great but what it's not working will show you what's wrong so this 1 in this case it says the output status is that the test run failure was not expected so if I look it's giving me child process Xt exited abnormal over that has quarter or whatever reason and so that was expected exposed to be a real result there and so OK something's wrong with that case I should really look carefully at the 1 the case and 0 if he gave me different things here it tells me that the young but there's a result missing from the expected to get a certain string in the output is expected a STF but for some reason it isn't there and then here it's telling me the details of change if I click view something changed all in this case is really not about what happened was used to be called area of statistics and now the whole paragraph that you may look at that and say 0 yeah I change that is supposed to be different right and in that case that's OK don't worry about changes like that because what we can do is if I if I like that particular test case I can say this test cases good it's a new golden standard the everything's working on it was that close that that's string in the and useful to have a different way more right so if you want to test case like this and you say that's working correctly that all you have to do is click on a new golden standard like this and it says are you sure that this is right alright this is perfectly right and I click yes and so now I basically told the system or that access this part of the now what I write that has small fail anymore it's using the new output as the gold standard so sometimes the changes in the past that really means really did change the supposed change and other times their problems like this where it really wasn't supposed to corridor right so you want to go through all of these tests that failed 1 by 1 and and see what's happening and for some reason I 1 EV is killing me
with that test case and so I can bring up my tools and I can set it to 1 EV and run at that ran their maybe it doesn't match the whole maybe it's a different maybe a different done temperature to our question look at the test case and by the way that it leaves around all these run that XML files so I can check amount then there's a series of test cases here that I can look at I think it was just 3 that was fairly well so I can look into history and see what was the default value in the current value and all that encountered nail down what was going wrong but we get back to the lecture and show you
know sort of the rest of it has get a sense of how this works let me go back and explain how this
is working so from a normal rapture runtime environment we can run the tool you member it generates a driver runs programs that run the XML or and we load up the result and the normally that run that XML gets switched away into your data results directory we've seen up to now so normally that because there is no that has that around because I like to use the capture that uses a testing and the only way is to me at all 1 way of stopping it from soaring that result is to set the onset you're very what we normally recognize that looks for an hour and vertical session here and it tries to put that result in that session here if you can find an environment variable that will put the result there so if you want to keep those what file sitting around 1 way to do it is is just like section another way to do it is create a driver of our by and using that to write on when you write a program by and with the driver file run file because that is your program of rest but somehow we really want your hands are on file and then you can move around while into a test directory so that I can write it like this 1 if I moving your test directory like you keep it around for later as a test case so imagine running after doesn't different cases grabbing overrun files and moving them into the test directory
1 test face is about his little father and rational recognize now as a test case so I can edit that run that run that XML file and I can add little test label and a description and all of that and this certainly that's how basically it makes it a test case for the test so unusual about so little related stuff and I have a new test for the tested on in the label you can put a little pipe symbol and that's how you create folders there's a folder run and then out of the value called 0 we the land so if I have room temp and 0 beetles short belt in a series of folders in the tested in the description here is a little note to yourself to remember what it was like that in face of the section of the course 1 person during the and areas of interest for this year you can do negative testing with this to you can expect failure when certain things have already said that they shouldn't of 1 0 yeah that's support all in the problem so just by taking you run file and adding a little bit test stuff into it and you can create yourself a test case and so on and then
just like we saw before you run through all these has cases and see what's going wrong and for each particular 1 so we can we can run through and look at the various outputs since was failing that but we should also should also make sure my laptop paragraph of 1 other thing
that I didn't show you before groups parenteral these test cases looking for the failures number that's the 1 every we goldeneye so it's passing now you at the end of the year a right so we have a complete set of test results sometimes of you outputs that slightly different this is a case where the output curve it says differs from expected value and if I view it it will actually show me some of the differences in the output of the location where I is actually wrong my old case that it expected to get and you see the black is the expected result so expected to get this black curve and the red is the test result and this is what actually got so here's a case where I just without at my eyes I think everything's fine but yet it's actually long completely wrong here what I got was 1st published the tools so that's exactly the kind of subtle error that you want to be able to to check you notice it gives you 2 2 ways you can see the output here is the string then you can also see the output as a plot of land and you can scroll through turns out I guess there's 1 value in here that's correct so that point 5 we actually match the the same value on the front and on across right but everything else is wrong on this curve and so on so the tester catches areas like that too so you can go through case by case and examine all the results and try to figure out what's wrong on here it says is that there's an extra number in the other so here somehow this is extra number that the tool is reporting that isn't supposed to be there when you should say 0 yeah that's good that's actually supposed to be there have water sector forgot deleted that told us now I need to the 1 where the the other you go through all of these different things you notice that on it will tell you the subtle differences in labeling and and the run status and so forth and and it's up to you to look at each 1 of these and and go through point by point and try to convince yourself whether or not the difference is real and it should be there or that the difference is completely wrong and you need to fix your tool of you can go back into the tool then and edited if you do some debugging on this particular very 1 but you can see how it's working and you might notice well here it was 4 point 2 kelvin so this 4 point 2 kelvin and 1 that's actually died of a put in an explicit kill kind
of thing so if I simulate what's not that 1 by simulate 4 . 2 and I 1 I shall process exited abnormally so that's the problem that into in my test case and I can fix the code this by saying I now that's fixed doesn't do that anymore so if I if I put in 4 . 2 but then when you be then error works good that works which should have around my tester and
select well that particular case let's say in red the the the the but that was working on so little by little you can work your way through all the failures that you have and convince yourself yes this is working now working correctly and you want to make sure that you get all green check marks down the column before you republish ritual so this is the last thing you do is try to double check at at some point as we get more and more people using the tester when they start using it as a standard for all the tools and will be able to find an answer to this is test the and ask this is tool that doesn't successfully worked in the past tests last RAM so we'll using that kind of quality metric on in the field of science that policy verification and validation and verification is making sure that the programs directly like this and validation is major that model actually explain to reality of social scientists use very carefully all of this and in the future we may start to like I said require better use that on for nanoprobe right now we
don't because the test is not quite feature complete like the builder testers about halfway there of only in the last quarter of the way there but it doesn't simple things that handles most of the simple inputs like build on the output side of the union of all about what's going at 1st but for example of there may not be the images doesn't handle directly the fields for sure we have implemented that so there's still some things and the tested at another point there but the simple tools that will work great for you so so give it a try
and see what you think these are the kinds of errors that we you catch that not that means changes in Alabama missing or extra and also that the input values change mainly change units chain of missing extra and so you can you can track out all those cases and again if you convince yourself that should be like that I really didn't prevent different than when you include people seem to enforce that test cases to pass from then on
they will be used so here's the assignment for you guys so you have a nice tool working now I yeah actually might want that was used very old version of this tool you created in because of the texture of the features of the natural world of all phases of the nodes in groups of all that enable and disable and all that stuff that you put in your latest firearm much you what's test so we now have to a gene version of your police are just as remembers Andrews far 1 on you can create a test need for use firearms was so that in the main test directory you run your test cases 2 times by hand you take on the XML files have unions only that those 3 cases of what you run into all of these processes flower is already created work that itself within the test directory and then try to run the rapture regression testing on at that point if you run it arranged program cleaning past and then you and I I want you to test and the sum of the numbers of testing and then run again and at that point you're your test will be weird compared to the current result and you should see the picture here on and you'll say no no like tools working properly now and always corrupted the test so that you can read all the lights naval thought so yeah that's right back to your own version of firearm on you don't have a version of this we can know what 1 of them will get can start from an not of previously that's and that's just OK let me show you my solution for the lab assignment here with the regression gestures uh so
here I am in my directory I can run rapture is an simulate different cases and I want to build the set of aggression just here for various test cases but with different values and really when I run rapture will get the results but when I close rapture and look around I don't see the run files Jesus I get my hands on those 1 files applied at test cases that I could use so there are 2 ways to get your hands on those run files 1 way is to copy them out of your data results directory under data results there's all these other directories that you can see that all depend on all they're all different workspace sessions that I fired up you might wonder which 1 is the current 1 while there 2 ways find and I can't echo session and it shows me that my current session is 32 64 so I can go and the data results 30 to 64 L and there a bunch run so those are all real violence usually the run filed with the highest number is the 1 that I just ran so 170 78 7 3 9 0 3 9 2 is 1 3 9 2 I probably wanted 3 9 4 that's the latest 1 are right so there's a run file with active I wanna double check and see what it looks like a memory can be rapture dash alone little popping up OK so that's that run by and it's interesting 1 because the 17 10 months really is really funny looking shape I'm actually grab that 1 is a new test case on another test this time the copy that filed from that directory back into my test directory and I'll give it a name I could lead to the same ranking given name funny shape that x and right so now in my test directory I've got 4 test cases actually idea to make that into a proper test case when I do that books by adding on test in here and slashes with the label and a description book at the description should probably remind you later when it breaks like what was important about this test case from now another way to grab test cases for the run that XML is too if you do the onset period session there and then if you run rapture all the cases that I run from here on 1 and 2 OK so that's another test case and now if I look that run files sitting right there so I don't have to go over 100 might give results directory of great that many of you agree that that there were about sitting right there now I can take that run file and I can move it into my test directory and I'll I'll give it a different name again another that XML plenty and at that point and give it a test description would the prompts and with the label and questions that the description of the snippet so that's how I generate test cases and some people were asking me 0 boy always that my session de back on well 1 way is just by starting a new mixture to the best start fresh with the new xterm will have a session the back on I can start a copy and paste that value into the session b equals and then can copy and paste from the middle mouse but in regard to not get recession do that strike so now I've got all these different test cases and if I run rapture dashed tester will bring it
up and it finds them all if it doesn't find some your test cases probably because you forgot to put test with the label in there and all that anything that finds in the test directory that has a label on it can pull up as a test case and you notice there's no description for that test case but everything else has a description so I can grab 1 test case or I can grab all of the test cases and run through them now and all its telomeres wrong with that test case let's take a look so if I look at that test case and it shows me just the output is different the result differs with I click on it it'll show me it expected these values and it got a bunch of other new values over here and you can even see the difference on the plot here the black line versus the red line right there this sort of that the difference between this and this and if you look at it very carefully whenever you see a difference like that shall ask yourself what the heck is going on you look in your program you convince yourself whether or not the program is correct should ask yourself is my program right in this case my program seems to be doing the right thing the latest result that I got the red result the test result is smooth and correct in the old result is actually we're going shopping you might ask yourself at this point had that has ever been there but maybe you just never noticed that last anywhere the test case it looked OK your eyeballs but it's all so at that point you might say well right the test is wrong my program is correct the test is wrong in that case I wanna use the current output for my program as the new golden standard now raptures says are you sure are you sure you want to reserve the right result of completely correct and if so then yes so now if I run that test now it's checking the output against my my latest standard right so rapture will look at all kinds of things let's say run the rapture builder and I'm gonna open my tool that XML and suppose I had a of Boolean control in out my tool that XML and I'm going say that but so we set a
default value and save say
that on soft say that tool that XML now when I run rapture is here that the extra control of their right and now when I run my fractured dashed tester and are all these tests they're all fail all why they fail well it's telling me that the inputs change this test case doesn't specify that new boolean input value and it didn't of course because back in the day when I created these tests I didn't have that control the value so the question is is that OK and again if that's OK then I can say alright keep that is my new golden standard and unfortunately you have to do it for each 1 you has here's here's here's maybe maybe someday they'll be some kind of batch thing where you can select them all and so now I've read goldeneyes my tests again so now they expect that control and if I run the malls bill passed cleanly because I change the universe so as are building a tool and checking everything we always want ask yourself did is something and I did that in in in the other you know just make a mistake in terms of might simulator or don't make mistake I add something to rapture or what you member it's really easy to mess up your code it's not very hard at all to make what looks like an innocuous change of the new code like 2 point 1 and save your program books not that much of a change and run
test and they're all failing again and this time unlike yeah yeah yeah over the output is actually completely different now from what is supposed to be completely different and it was like 0 yeah I was debugging something and I added that stupid 2 . 1 in there but it means you so he eventually tracked down the problem in the set of all man this is wrong when you fix the problem and you run it again what's the contested In and now you can convince yourself that everything's running clean so that's the whole point of the regression testers to say I don't test cases and I wanna make sure that when the temperature is 0 the temperature is 500 million degrees Kelvin whatever the electron volts is negative pollination simulation doesn't rational nature of that it is proper value and I making states last because it's so easy right to change one's character in your code and screws everything up program all I and for all I shouldn't this example case of man of the the classic example is I have a loop variable appearance that i equals 1 right or i equals 10 from counter but wait wait i is also the imaginary number in Matlab right so now which I'm running my best case select all and run by seeing the simple and that was saved by I was wondering when there was no because we're going to power of the validity of your program and you can look at each case and convince yourself like what did I do all like gosh like gosh what'd I do right is that the US fire-ravaged what level of the wrong 1 so what so when you do stuff like that the careful to check your code carefully and make sure when you go to publish books always forget the test report that make sure we about to republish tool you always go back through the Russian test suite and have rightly in my own mind because topology is that right now the rapture regression testing doesn't work really complicated tools that we need to fix it finished so that works all the complicated appetites is that would really work for you guys and you you can use it to check everything but you know it's there at least you know you should be very scared at anyone change that you made your vote is the simplest 1 line change sometimes bring every


  814 ms - page object


AV-Portal 3.20.1 (bea96f1033d39fbe77f82542458e108105398441)