Hello, everyone. Can you hear me? Okay. Hello, everyone. Thanks for coming. This is my first talk in EuroPython. This is my first time attending as a EuroPython speaker, and also this is my first English talk. So yeah, it's lots of first times.

So let's see how it goes. Thank you so much for coming. I know this is a really bad hour. We already have a quite good coffee, and this time we call in Spain the siesta time, so thanks for checking in. It's a pleasure to be with you. So we are going to talk about this topic, RPA, TDD, and embedded.

Maybe you know some of those words. If not, don't worry, I'm going to explain it. My idea or my intention is this to be as much clear as possible, because I want this to be really beginner friendly, so for all of you that maybe are new to Python or even new to testing itself, I hope this is useful to you.

Some things about me. My name is Javi. I'm from Spain. I'm working currently at Geodab. Thanks to them, I have some free days to come here and give this talk. So grateful. I'm working as an embedded firmware engineer. I'm the organizer of the Python conferences from the 2022 and 2024, and also I'm the

Python Spain president, elected this year. So, yep, thank you. So what's all about? Also all those fancy images, thanks to Dali3.

What's all about? Okay. We are going to go through all the TDD thing, explaining why testing is important, and trying to introduce you into a framework that, in my opinion, it's awesome for testing, because it allows everyone to start doing any kind of test without knowledge of Python.

So first thing first, we are going to first talk about raw framework, which looks like that. This is the language itself. Then we are going to introduce some, yeah, expected, sorry. We are going to introduce then the board that we are going to play a little bit with

it. This is an ASP8266 board, and we are trying to glue everything together. How? With raw framework. Seems feasible. So let's see how. First we need to introduce some terms, those three terms that maybe some of you are not

aware of them. The first one is RPA, which stands for robotic process automation, that it's basically it assumes that the environment that you are working at is not controlled by you. So it expects that the application, or in this case the test, or the automation that you are writing, it expects that automation to prepare and set up the whole environment,

because it assumes that the computer that you are going to run the tests is not controlled by you. So you have no control of it. The next terminology is TDB, which stands for test drive and development. It's a design pattern in which you define an application and your API by writing first

the tests that are going to validate that application. Seems a little bit complicated or slightly different, but the power of TDB is that when your application is ready, you already have the tests. So you are 100% sure that you are covering all your requirements.

So that's great. And finally, the embedded world, which is suffering, and lots of suffering, there are lots of things that can go wrong. It's quite easy that you start working with your board and then you start seeing some smoke, some elsewhere, yeah, you know what happened there. So embedded world is suffering in general, from the design to also writing the finger.

So that's why it's so important to test it well and have very well defined tests. How are we going to do that with Robo Framework? Some hints about Robo Framework. It's an open source automation framework. It has lots of power behind it, but it mainly allows you to automate both tests

and tasks. It can be used also for automating tasks. For me, the most important thing about it is that it's human-friendly syntax. You write the test exactly the same way as you write, like, your text, like writing a book, the same way, and finally, it integrates fully with other automation and languages.

It works 100% compatible with Python, but it can work also with Java, with C and other frameworks and languages. It's quite powerful, and it's been used by lots of companies like Cisco or even Microsoft. First thing first, I'm going to try to explain why embedded testing is sometimes

a mess, and it's difficult. I'm going to introduce just a little bit of C code. Maybe this is the first time that you're seeing C code. It's a little bit horrendous, I know, I know, but it does the thing that it needs to do. What's the problem with unit testing? When you need testing for embedded application or for C application, mostly, the problem

is that you have to define your tests as part of the application, and you need to switch applications, so you have your main application, but then the test is another application that is checking your application. So it's such a pain. And the problem with that is that we have these pieces of code wrapped inside this if

there are instructions that are basically dynamically compiled code. This code may exist or may not exist. So it changes, and it's hard sometimes. The problem with this is that it works. It works really well, but you are not properly or 100% testing your final application.

You are changing the code of your application to test the application. So strictly speaking, it's not testing the application itself. And then we have the black box, or the integration testing, where we have our board right there, and we need to connect it with all the other nodes, devices, and things,

because basically an embedded system, by definition, it's a system that works inside other systems. So it needs to communicate, for example, with a server, it needs to gather some data and publish it to a topic in MQTT, which is a protocol, a communication protocol. So this second testing is the most useful one, where you test your whole setup with

your whole board. So that's the second one. This is the one that we are going to try to automatize with our framework. So before going into that, I'm going to briefly explain our framework with some examples that maybe are near to you, using an HTTP or STPA, which I think for sure you will know,

but I will explain it otherwise. So first, we are going to define a database. This is the database. We have a robot factory, and in that robot factory, we have to store the names of the robots and also some of their status. So you for sure may already know that, but we have some crude operations, like insert

a new robot there. We have the option to get robots and things like that, okay? This is the unit testing part. We are going to unit test this API directly from robot framework and explain the process step by step.

Those are the tests. Like a little bit small. Sorry about that. And I have to say, I have an excuse. I have just bought this clicker thing. Oh, sorry. I have just bought this clicker thing, and it's quite cool, isn't it? So I really like to try that, so sorry.

It's small on purpose. But yeah, we have the test here, but we are going to go step by step, explain how our framework test is constructed. First we have the settings section, which is like that. In the settings section, we define everything related to our suite.

In this case, we have the documentation, which is going to be generated, and everyone knows what is this suite testing. Then we have some libraries. The first library, the database library, allows us to access database information directly, make SQL queries, like, okay, give me these robots or check that this

field exists. Then the second library is not a library itself. This is our Python file, the one that we are going to try to test. Remember, we have all those functions that insert robot, check robot, delete robot, so we are going to test those functions, and in here we have some environments set up

that basically are going to create some data that we are going to later use to verify that everything is in there, okay? Next we have the test cases themselves. The test cases have data variance. We have first the test case name, in this case robot insertion, robot exist, robot deletion.

We have some metadata, in this case we have only documentation, but we can include some other things, like we can tag the tests to be able to filter them out and things like that, and the powerful thing is that you can actually read the test. This one, maybe it's quite difficult or different, because we have a SQL query directly embedded

into that, but we will get into that in a moment, and maybe you may ask, like, where is the Python function, like, you don't see any Python code there, you don't see the class that we have defined previously, there's a trick, it's there.

This insert robot first, right here, that's our code. What happens with our framework? It intends to be readable, always. So when you have a function name, the underscores are replaced by spaces, and then every first letter is capitalised, so that insert robot right there is underneath our insert robot

code, the same that we have defined in the API, but that way is way more readable. Those things right there are the first argument, the robot name that we are going to insert, and the second argument, which is the status of the robot.

We commit the changes, and then we check that the SQL query actually returns the robot name that we have just inserted, okay? The same here, the same thing before. We insert a default robot, and we verify that that robot exists. So seems readable enough for the moment.

And this is the running. You run it on the terminal, robot run test units, and this is the output. Now what about integration testing? Because we've been testing only the database, but I said previously that we are going to have a full HTTP REST API, so how do we test it?

This is the API that we have defined. For sure you will know fast API at least because it's been mentioned throughout all these past days like 100 times every day, so this is the 101. This is the fast API thing that is interfacing our database. And with robot we are going to test all of these methods here.

And this is the test. I know. It's a little bit small. But it's only for the clicker thing. So the same as before, we define our settings, the thing that we are going to test how to set up, how to create everything. In this case, we have the documentation.

It's important to have documentation of the test case of the test suite because if you are going to check it after a month, at least you know what's going on here. We have some resource thing that I'll go into it a little bit later and some setup things that are the same as before. It's creating some dummy data for the database to have everything ready for whenever we want

to test it. Then we have the tests themselves. Exactly as before, we have in this case a way to list robots. We have the one to get the robot and this is basically performing some HTTP requests underneath. Like if you have a look at the line 15, we have get on session, we have a session

object which who knows what's that and we get the robot paths and we expect the status 200. So if everything goes fine, we get a response. We have to evaluate who knows what's that robot as a JSON thing and then compare the JSON with the expected output and it should be equal.

So this is readable. I mean have you seen that thing below like get from dictionary data, get all those things, what the heck is the session object, what's that JSON thing, what's going on? I know I promise you this is going to be readable, but this is not that readable at all.

I'm not a scammer. This has of course some tricks because if not, it will be clickbait. The damned small pictures, I think this joke has gone too far.

One of the powerful things of raw framework are the resources file. What's a resource file? It's almost exactly the same as a test case file, but we define things that are common to all the test cases. In this case, as before, we have the settings with the commentations, some libraries, for example, perform request, access database, spawn processes, whatever, up to you.

And then we have some variables which define the static data that it's available for us throughout the execution of the test. But the powerful thing of the resource files are the keywords. So we can wrap all this shitty code that means that we have previously into beautiful

keywords that basically translate the test previously to this thing. So for example, to list all the robots, now we only need to API get robots as JSON and then just simply compare that the return answer is exactly the same as the one that we expect. Or for example, for verifying the robot ID matches, all those get from dictionary crappy

thing that was underneath are simply those two lines. We get the robot as JSON and then we match or we make sure that the status of the robot is the one that we expect to be. And this is way more simpler and easier to read.

But what if, what if we used 100% of our brains? What if we define our keywords with Python because remember, as said previously, robot framework is 100% compatible with Python so we can define our keywords in a Python

class and in this case here, we're using the well-known request library to perform all the heavy job underneath and we simply define Python API get with a path and expected status and Python API post. And what we have, boom, your tests simplified, written and running with Python and we have

the same keywords here, we have the Python API get, we have the Python API post and everything is simplified and that's one of the key features of robot framework. If there's no library for it, just simply use Python. You don't need to reinvent the wheel, there's already lots of libraries right there.

And then we get these fancy logs. These fancy logs basically have all the information on what has happened in the test, like all the setup things, the keywords, the logging that has gone underneath and this is quite powerful but only talking about the logs will be at least a workshop.

So I'm going to pass right quickly over it. So that's robot framework. That's like the overall look of it and now let's introduce the hardware. Let's see how can we combine all these keyword fancy things with the hardware. This is the board we are going to test, as I said before, the SP8266, it's quite

a simple board and we have some sensors connected, some sensors, we have some temperature and humidity sensor and finally a light sensor. This board was developed by me like during the COVID pandemic and it serves for the post to detect how many people are in a room.

Combining all the sensors, the lectures, it's quite possible to know if there are too many people in a room. So for the COVID thing, made a lot of sense. What are our goals? What are we trying to test? What are we trying to verify? The first thing, it's quite important. We need to load a known finger to the board because aside from Python applications,

when you want to run something in an embedded device, you have to actually write it in the device and every time you update the code, you need to fully write it again from the beginning. I know it's a pen but it's the way it works. There are some means to avoid that but yeah, it's difficult.

Okay, the next thing that we need to ensure is that we can communicate through the UART. What's the UART? UART stands for Universal Asynchronous Receiver Transmitter. Okay, I'm exactly the same as before.

What's that? It's simply communication with the device. It's the USB cable that we've seen before, that USB cable is capable of sending and receiving data from the board. But we want to make fancy names. All that is because of the telecommunications engineering. They really like the acronyms. All the acronyms are because of the telecommunications engineer.

They have acronyms of acronyms and nested acronyms inside an acronym. All because of the UART. Send commands, receive response. This is basically what we want and what we are trying to do. Finally, we are going to validate all that data against an MQTT server, which is basically

a place where one can publish messages and other boards, people there can subscribe to those messages. Like, I say, hey, this room is way too crowded and there's a subscriber that receives that message and decides to do anything or publish some data or to have some operations.

So that's basically very briefly explained an MQTT server. Then we need to catch potential exceptions, because the same way as in Python we can have exceptions, in embedded devices we can have exceptions. Because we have an invalid instruction, because we have run out of heap or of stack.

And finally, of course, we need to be able to set up and tear down the whole environment for the board to work and always start from a known state. Okay? Right. Okay. This is the setup function that we are going to use to prepare everything to load

the firmware into the board and we are going to go step by step to see what's going on underneath. We have here the find available device, as I said before, some fancy keywords to basically look for where is the device connected to our computer. And if you have a look at it underneath, it's basically some process.

In this case, it's calling Platform.io, which is a hardware handling tool, device list, we get the output as a JSON, but as I said, the keywords serve to hide all those things. We don't need to know that. We only need to find available device and load firmware into the device.

Second, we create the serial session. We are using, in this case, Telnet, which is a communication protocol, and Robo framework has native support for it. So same as before, we hide everything in a keyword and we simply have create serial session. Underneath, what's doing all that? I don't care. Simply I don't care.

I know that after that keyword succeeds, I have a fully available serial connection. We could use some other libraries like PyExpect or PySerial, sorry, but in this case we are using the native implementation of Robo framework. We create the MQTT server and also we start an access point because the only way

the board has to communicate with the internet is through a Wi-Fi access point. So in this case, we have our access point in the computers and we start MQTT server with a Docker daemon underneath. As I said before, we don't care about the implementation.

And finally, we prepare the board. All that was prepared in the environment and now we need to set up the board. What's the setup of the board? First we need the board to connect to the Wi-Fi and later we need the board to connect to the MQTT server. And this is as simple as we wait until a prompt.

We have defined what's a prompt. So we are waiting basically for a specific character appearing in the board and we send these commands. We set SSID, password and all that information. And finally we wait until the board connects to the Wi-Fi. The MQTT setup, the same.

We send some commands and we expect a message to arrive in the MQTT server so that way we know that the board is already connected and has connectivity with MQTT. Same. Remember, one of our goals was detecting errors in the board. How do we detect errors?

We detect errors with regular expressions. We know that errors that are controlled from our side have this syntax. They have an error and a message. Then we simply check for that in the output of the serial device. And then we can use that also in a test case.

We can have a test case like that where we expect this error to happen. Simply. How do we then catch exceptions? The same way. We look here for the exception and then a number. Why? Because exceptions are nothing. That's a pretty funny thing that no one is able to read but at least you need to detect

it in order to try to solve it. So as you can see here, we have the exception and the number. And we are looking for this. If that's in the output, then the board has reset and exception has occurred. And these are the tests. The tests are checking that the sensor data is the one that we expect.

We are using for that variable file which is defined here. That it's basically in this case some Python dictionary with all the sensors that we have. The ranges of that sensors. We are using all that in several frameworks to verify that the data that we are reading

first is correct. And second is the value that we are expecting. Either if we write, because this board allows writing data to verify that everything is fine. Or that it's in the range that we have defined. So we have some thresholds here to verify that if we set a value that it's above the threshold, fails.

The same. This is testing the sensor data. We verify the internals. This is writing a command, reading the data, and verifying that that data appears in the MQTT server. And this is like the COVID-related thing. Where if we write some data that it's above the threshold, we should receive some events.

Like for example, if the temperature here is above 30 degrees, then we should have an event that says, okay, this room is probably too crowded. Or if the light sensor right here is above 600 lux, then very probably there are people

in the room. So those are events that we have defined and that we are checking against the MQTT server. But as you can have a look at it here, it's quite simple, the test. We have the template, which is basically simulating an event, which means writing to the board, receiving data, and verifying that everything goes fine.

The keywords underneath. I'm not going to explain them, because maybe it's too much detail. And as a bonus, you can of course generate the docs of all the keywords that you define, so it's pretty accessible to all of them. And how does this look like? How is this a real execution of our framework?

In this case, I introduce some traces to see what's happening, because if not, it's like quite quiet. But here you can see we are loading the framework once we have found the device. Then you will see that we are starting the MQTT server. We start the Wi-Fi, we start some things, and it should start running tests in a moment.

So conclusions. What's the thing about raw framework? What's the power of raw framework? It's super easy to write tests. I know there are some crappy things underneath, but if you abstract everything to keywords, it's quite accessible.

You can write the tests as you want them to be read. And it's as simple as that. And also it supports, maybe you know that, the jerking style, so you can write tests literally as if they were data. If they were text. So simply write them, read them, and everyone in your company, in your business, can start

doing tests. Can start working on some APIs for you. And you already, you will have already some test base to test everything. So what are you waiting for? Go play with your robots. Thank you so much.

That was a fantastic talk. Give a round of applause again to Javier. So now there will be like a Q&A section. So if people have some questions, please go to the mic and you can ask.

They're like two, three minutes only now. So please be quick. Any questions? I have a question for you, if you don't have any. Was it really beginner-friendly? Or maybe it was complicated or more complicated that you may expect on some things?

How can I improve this talk? Okay. Less code. Perfect. Okay.

Thanks for the talk. It was interesting to see how others are doing this. We are also testing hardware. And we're using pytest. Did you also look into that, or did you evaluate, or was it already there?

Yeah, we work lots with pytest. But the problem with pytest is that it's not accessible to everyone, at least in terms that we're trying to make it really accessible. We are pushing really hard to have tests that can be read the same way as you read

a book. So the problem with pytest is that at some point, although it's Python, it has some initial knowledge that everyone needs to go over. So if, let's say, I want some guys or some people from the production engineering or from the hardware design team to write the tests that should verify that the hardware

is working, and maybe they don't know Python, this is a great entry point. Because they will already have lots of keywords with some behaviors, and they can focus only on writing the tests and not on understanding the language and the framework.

Quick question. Have you come across the concept, is that what you were describing, using the three Amigos style of having it where BDD has three stakeholders, which are the programmer and the product owner and something else?

One of the problems, for example, that we have in the company is that, but I think it happens everywhere, marketing people really want to sell lots of things. That's great, because it's the money income, but sometimes they're trying to sell things that are almost impossible to build, and those four, these things are our main stakeholders

and shareholders. Why? Because they already know what are the potential capabilities of the things that we can do. So if they have a great idea, they can first write that idea in a formalized way with the keywords, with the tests, and if that's feasible, then the idea is possible.

So we don't need to start fighting with them, like, why are you doing that? It's literally impossible to make this board fly. So why are you doing that? So that's our main stakeholder.