LAVA federated testing
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 561 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/44367 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Statistical hypothesis testingStatistical hypothesis testingPresentation of a groupStatistical hypothesis testingStatistical hypothesis testingComputer virusComputer animation
00:24
ArchitectureAutomationStandard deviationPhysical systemComputer hardwareStatistical hypothesis testingKernel (computing)Computer-generated imageryRootBootingAndroid (robot)Power (physics)Control flowWhiteboardTelnetDynamic Host Configuration ProtocolLoginServer (computing)Configuration spaceRAIDRepository (publishing)ZugriffskontrolleError messageBlogParsingMessage passingAddress spaceDuplex (telecommunications)Structural loadBootingSuite (music)TelnetResultantWhiteboardFunction (mathematics)RootProcess (computing)CASE <Informatik>Statistical hypothesis testingServer (computing)Latent heatScaling (geometry)Perspective (visual)Instance (computer science)Dynamic Host Configuration ProtocolAdditionTwitterKernel (computing)Type theorySerial portCellular automatonBitGame controllerDifferent (Kate Ryan album)Parallel portSoftware bugInformationPhysical systemLine (geometry)Address spaceMultiplication signSound effectSystem administratorAndroid (robot)LoginTouchscreenRight angleComputer architectureError messageTable (information)Web 2.0Uniform resource locatorStandard deviationSoftwareComputer hardwareBoss CorporationPasswordTraffic reportingPower (physics)Computer programmingMereologyRevision controlClient (computing)Key (cryptography)Configuration spaceWrapper (data mining)Scheduling (computing)Software maintenanceValidity (statistics)Computer animation
06:50
PixelExecution unitWhiteboardImage warpingMaxima and minimaSquare numberEmailConvex hullLink (knot theory)Meta elementStatistical hypothesis testingStatistical hypothesis testingCore dumpPersonal digital assistantServer (computing)Computer-generated imageryNormal (geometry)BootingScripting languageBlogControl flowPower (physics)Revision controlArchitectureConnected spaceServer (computing)Gastropod shellProcess (computing)Statistical hypothesis testingRevision controlStatistical hypothesis testingType theoryFunction (mathematics)LoginResultantUniform resource locatorDifferent (Kate Ryan album)Electronic mailing listBootingConfiguration spaceSoftwareWhiteboardBitCASE <Informatik>Latent heatCellular automatonWebsiteMultiplication signEmailMultilaterationCommitment schemeFunctional (mathematics)BuildingComputer animation
13:15
Point cloudComputer animation
Transcript: English(auto-generated)
00:06
Hello. Now, Remy will present us Lava Federated Testing. Hello, everyone. Thanks for coming. So as I said, I will today present Lava Federated Testing.
00:22
So the first thing to explain is what Lava is, because I guess you don't know what it is. So Lava stands for linear automated validation architecture. It's a CI system, so the goal of it is to test your software on specific hardware like Raspberry Pi, Juno, Panda board, or things like that.
00:42
So you take your, for instance, you compile Linux and you want to test it, or you have some, your software that you want to test on real hardware. So Lava will help you to deploy the software on the hardware, boot the hardware, and then run the test. It's more or less becoming the de facto standard for a CI system on real hardware.
01:05
So just to explain a bit more what it will do for you, if you want to test on a Raspberry Pi but without Lava, what you will usually do is to, you first power on the board, so using the power controller. Then you connect to the serial.
01:20
If you have a serial relay, you will just do telnet something. And then you will see your boot starting with, and you will wait for it any key to stop at the boot. And then you just type enter, and then you type commands like DHCP, set server IP. You load kernel DTP from TFTP into the RAM, and then you set the boot tag and you boot.
01:41
So you will look roughly for the kernel output to see if everything is going correctly. And you go to prompt, you log in, and then you run the test, and you power off the board. That's all the thing that you have to do manually if you want to run tests on a physical hardware. If you want to use Lava to do that, then instead of typing everything manually, you describe
02:03
what you want to do in a job definition. I will explain a bit more later on what you put inside it. You put the information like the kernel, the TB, root FS you want to test inside the job definition. You give that to Lava Dispatcher. So Lava Dispatcher is the part of Lava which is responsible for booting and deploying booting
02:24
the boards directly, really connecting to the board and handling the board things. You give the job configuration to Lava, and it will just do everything as you did, as I explained before, but automatically. You can grab a coffee and do something else and go back and just look at the results.
02:41
Lava will do everything for you. Obviously you can do that at a larger scale, not only with one board on a dispatcher, but five, ten boards on one dispatcher, having many, many dispatchers all connected to the same server. So you can have 100 boards in the same system. All the users will talk directly to the server, to Lava server, and they will submit
03:03
jobs, get results, and see what's happening. So with that, you can have many, many jobs running in parallel. You don't have to care about it. Everything will be done automatically for you. So for instance, you can want to test your Linux kernel on a panda board, BeagleBone,
03:21
Raspberry Pi, or at the same time, just submit three different jobs, and Lava will do that for you. As I said, for a user, on a user perspective, when you use Lava, you write down your job definition, and then you submit it. Usually just with the common line, you use Lava CLI, job submit, job definition, and that's done.
03:41
You don't have to do anything else. You can do some more interesting work. In the job definition itself, what you will put usually just some more important things that you can use. There is more things to describe obviously, but it's too large for the screen. So usually you will describe the device type you want to address.
04:00
So you will not say, I want this specific device which is in the lab, not Raspberry Pi O1 or O5. I want a Raspberry Pi 3B, and Lava will pick up the right, the board which is matching the device type, and available at the same time. You will point to the kernel, DTB root FS you want to deploy. It's a URL, so it has to be available somewhere, and Lava will fetch it for you.
04:24
Some interesting features that are not mandatory, but you can use it, is auto-logging, for instance. So if you have to type username and password, Lava will do it for you. You just have to provide the username and password, obviously. Then the tests that you want to run. Usually it's inside a Git repository, so you have versions of your tests that Lava
04:44
will record the version along with the results. And we already provide, as that address, many wrappers around well-known test suite like LTP, KSF test, Android test suite, a thing like that.
05:00
For the admin perspective, the server will do all the web UI, so you will submit jobs manually or through the API. You will see the results, the logs that have been passed. It will do the access control for you, so who is able to access a board, who is not. Some boards might be private, if there is NDAs and it's a public server, you might want
05:24
to hide some boards from public access. We'll do the scheduling with priorities and things like that. We can even do multi-node jobs, which is one job definition that is using more than
05:40
one device. For instance, you want to test a streaming program like VLC. You will have two boards. One will be the server and one will be the client, and it will be all in the same job definition. On the dispatcher side, the dispatcher, which is connected directly to a board, will regularly
06:02
check that the board is having a good health, so it will run some specific jobs to see if the board is working correctly or not. If it's not the case, the board will be put in maintenance and admin will be notified that they have to look at it. When the board is booting, we will pass automatically the kernel output to see if there is a kernel
06:24
panic. In this case, we will report it and stop the jobs, or boot the errors and we'll try to classify why there was an error. Like if it's an infrastructure error, like in this case it's a TFTP which is not responded correctly. This is not the test which has an issue, it's the infra error directly, so it has to
06:43
be reported to the users. The same if it's a bug, we will try to report it so the user can just say it's not my problem, it's lava issue, not mine. We do support a lot of different methods for deploying, booting, and testing. I will not explain everything.
07:01
We do support a lot of different boards. This is the actual list of supported boards from before Friday, because Friday wanted one more. It's not listed here. So that's a lot of different boards. As lava is a testing software, we have to prove that it's working, so we have to test it.
07:20
It's a bit strange if we provide a test software which is not tested correctly. We have to do it, to do that correctly. So either we are really lazy and rich and we just buy 142 different boards and build a really big lab, or we are not and we have to be smart. Obviously, if I'm here, it's because I prefer to be smart and to do some fun stuff.
07:43
That's why we created lava fed. So lava fed for lava federated testing. So the goal of lava fed is to prove to people for the community that the next version of lava will still work the same way it was working before for them.
08:03
So the goal is to take the Docker build that we do for every commit. Every time pushing a commit on lava, we'll do two Docker containers, one for lava server and one for lava dispatcher. And I want to prove that these containers are working correctly. I want to prove that it's working correctly on the board that the community cares about.
08:22
I'm already testing on the board that my employer is caring about. I want, for the same reason, the use case you care about, because my use cases are already tested. And I want lava fed to be almost only about jobs. It's better, I'm developing lava, it's a testing software.
08:41
I should be able to test myself with lava and with some glue around it, which is building notifications and APIs, which means that I will also test notification API at the same time. So two features that you have to understand that before I explain really how lava fed is working, the first thing that we have a lot of different device types in lava and
09:05
we have one specific device type, which is Docker, we can just create a job definition in which we restart a Docker container on the dispatcher itself. It allows to do some testing quite easily. So instead of connecting to a Raspberry Pi cellular connection, we'll just run Docker
09:22
run and take the output of Docker run and just type command in the shell that we get. The second thing that we use a lot is notification. You can ask lava to post to a specific URL, so it's a callback, when a job is finished or cancelled, so we can get results, logs, everything quite easily.
09:46
So let's finally speak about lava fed. So we use GitLab for our CI, so every morning, around four in the morning if I remember correctly, GitLab will start a container, it will keep the last version of lava server
10:03
container which is available, so this is from beginning of this week, you will see the full version here. It will start it, it's a lava server and it will be the lava fed server which is available from everywhere. In blue you will see a community lab, somewhere in the community is okay to share a lab
10:26
with us, and the server is available from everywhere, and the dispatcher is in their private lab, so it's not accessible from outside, and they do have a Raspberry Pi connected to it. So the first thing lava fed is doing is asking for the version of lava fed server,
10:42
so we are testing the same containers, same versions, and it will submit a lava job. As I said, we can submit to a specific device type which is a Docker container, so we are submitting to a lab server, we are submitting a Docker job which is in fact a lava dispatcher,
11:01
so in their dispatcher they will run my own dispatcher in the Docker container, and this dispatcher is configured to connect to my server, so I'm kind of hacking a bit their own dispatcher, so I'm sitting in their network in their own dispatcher connecting back to my server. And then I'm asking for the configuration, I'm asking the lab one server, can I have
11:25
your configuration for the Raspberry Pi, and I'm copying everything on my own server, so I know how to use their boards with my dispatcher connecting in their network, and I ask the lab server to say now the board is offline for you, you should not
11:42
use it, and my lava fed server will now see the board available and will use it. So I submit jobs on my server, and it will run using my dispatcher on their board, and I will get, when it's finished, I will get results in my own servers, so I will
12:01
be able to do some graphics, and obviously when it's finished I will do the reverse and give them back their own board and cancel my jobs. So, what's next, currently we started small, we have only three labs and five devices, which is not much, and only running five jobs there is not much, but we are adding
12:24
new device types, as you can see when I'm adding a new device type it's just failing usually, so I'm fixing it and then adding another device type a bit later on, and we want to have new jobs, all the function tests to run inside lava fed. So obviously the next step is lava federated, so the goal is to share the board with the
12:46
community to test together, so I need people that are already using lava at home to use lava to just come to talk to us and try to work together to be able to test the board you care about and the job you care about, so please send me a mail if you're interested
13:04
or just go to the website to see what we are doing with lava fed, thanks.