Actionaz: Automation for everyone
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 90 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/40257 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201382 / 90
2
5
8
10
12
13
14
15
17
19
21
24
25
28
29
31
32
34
36
39
40
43
44
46
50
51
52
54
55
57
58
62
65
66
67
78
79
87
88
00:00
SoftwareGame theoryScripting languageGroup actionGroup actionLecture/Conference
00:14
Group actionScripting languageSoftwareDigital rights managementGame theoryDigital rights managementRange (statistics)Group actionDifferent (Kate Ryan album)Key (cryptography)Arithmetic meanSoftwareWindowWordScripting languageGame theoryContext awarenessComputer programCASE <Informatik>Validity (statistics)Cartesian coordinate systemComputer animation
01:00
Scripting languageGroup actionSoftwareGame theoryComputer programmingDrag (physics)Scripting languageGroup actionCondition numberCASE <Informatik>Parameter (computer programming)Sheaf (mathematics)PlastikkarteEntire functionComputer filePosition operatorBitGame theoryWindowWeb pageCodeCursor (computers)Regular graphJava appletVariable (mathematics)ResultantLatin squareMultiplication signType theoryString (computer science)Medical imagingWeb browserFunctional programmingTouchscreenProgramming languageTriangleFuzzy logicSocial classExecution unitElectronic mailing listLine (geometry)NumberAlgorithmVideo game consoleError messageUniform resource locatorComputer programLecture/Conference
06:43
Computing platformComputing platformSource codeMultiplication signComputer programMixed realityFreewareComputer animation
07:25
Windows VistaComputing platformRepository (publishing)ArchitectureGroup actionPlug-in (computing)CodeText editorIndependence (probability theory)Distribution (mathematics)Office suiteNetwork topologyInterface (computing)Scripting languageWindowComputing platformSoftwareComputer programGroup actionFreewareLecture/ConferenceComputer animation
08:42
ArchitectureGroup actionPlug-in (computing)CodeText editorIndependence (probability theory)Computing platformScripting languageComputerFile archiverComputer fileGroup actionCodeComputing platformBitCross-platformPlug-in (computing)Lecture/ConferenceComputer animation
09:35
Computing platformScripting languageRow (database)PlanningOnline helpSoftware bugOperating systemInstance (computer science)Mathematical singularityCodeOperator (mathematics)Lecture/Conference
10:14
Scripting languageoutputTask (computing)DatabaseScripting languageGroup actionComputing platformoutputMultiplication signCondition numberComputer fileWindowInstance (computer science)Computer programMedical imagingKeyboard shortcutTask (computing)Focus (optics)HookingCodeRow (database)Data storage deviceComputer animation
11:36
Scripting languageWeb 2.0SoftwareGoodness of fitElectronic mailing listUsabilityDatabaseComputerGroup actionWeb pageSource codeParallel portPosition operatorDistribution (mathematics)Different (Kate Ryan album)EmailSoftware developerInformationAreaLecture/Conference
13:26
Open sourceWebsiteElectronic mailing listMultiplication signComputer animation
13:43
Lecture/Conference
Transcript: English(auto-generated)
00:01
So hello everyone So the topic of this talk will be xns3 automation for everyone So first what do we want to What is actually the meaning of automation because it's a word that's used in many Contexts in that case. It's software
00:24
Automation it's a software automation tool. That means that it's most often used in games software validation business applications So it's it's not It's not like software that you can use to test your program. You can do that, but it has a wider
00:45
broader range of users So it works as scripts and scripts contain different actions and actions can be house clicks key presses Window management actions file access etcetera and
01:03
There are 46 different Actions so for everyone because you didn't don't need to learn any new programming Language to use it. You just have to drag and drop the actions
01:24
and that allows non programmers to use that tool and automate some actions But it's still developer friendly because you can program you can enter some JavaScript code in the parameters of the actions and that JavaScript code will be evaluated when the action is
01:44
Executed so it allows you more customization and you also can write An entire file in JavaScript using Classes that are provided by
02:02
Actioners who can really write only in JavaScript if you want to So, let's see a short Example here. I have script and On the left side. We can see this is the main of course the main Window of actioners and you can see on the left as different
02:22
The different actions that are available and at the center you can see the list of actions the script That I've added here and with a line number that you can write so you can write a label if you want to Name of actions and some comments if you want to explain what this action does and on the right side
02:43
you have console where there are where the Execution errors will be printed and you can also write some text there if you want to debug So, let's see the first action it's a variable Action here. We will create a variable named FOSDEM URL and it's type
03:03
It's a string and here you can see that there is a small red Triangle it means that the text I've hunted entered here is not regular text. It's the JavaScript So here in that case, I create a simple string and I use a JavaScript function to get the current here and
03:22
that all of that builds a string that contains the URL to this Lightning talk so the goal of the whole script I don't think I've said it is to show to open a browser Firefox and to show the page of this
03:43
Lightning talk So And the next action is a detached command because you can run of course commands and in that case We run the command Firefox and we use a parameter here and you can see that here. It's not JavaScript It's regular text, but you can insert in regular text a variable
04:01
Here is FOSDEM URL and it starts with a dollar like in PHP So it allows people who don't want to or don't know how to program in JavaScript to insert some basic variables in text Then we have a window condition the idea of that is to
04:21
Wait for a window with a specific title to appear or there are of course other conditions, but in that case we have The title is FOSDEM and you can see there a star before and after because you can use some wildcards if you want to To have a title in that case, which was the title contains FOSDEM
04:45
So we wait until it exists because if if it's true, that means that we found it We do nothing that means that we go to the next action And if it doesn't exist then we wait so the script execution will block on this action until we found that window we
05:03
And then we have a pause. Of course, we make suppose that's amazing For here for one second, so that's cool The next action is more interesting. It's find image action here this allows you to search for an image either on the screen to make within a screenshot or
05:25
within another action or within a window and that uses some OpenCV Algorithm and here we search on the screen for Image called FOSDEM and it's image here
05:43
So it's the FOSDEM logo because we need to search that on the on the browser And we store the result and the position we found in a new variable called FOSDEM position Then there are four other actions. I don't detail each one of them because it would
06:05
Take some time, but the idea here is to move the cursor and the position we found then scroll the wheel to scroll to get down on the page and find the action as text here and then click on it and
06:23
That works should work So yeah a bit of history I started coding action as in 2005 because a friend wanted to play a game where he had to click often on something and
06:40
Well, it was written in C+. That's not the typo. It's really C+, because it was horrible It was some kind of strange mix between C and C++ So it's a freeware Source code is not available because it's so horrible. It used the MFC Okay, so in 2006 I rewrote the whole stuff and I used WX widgets, which was quite cool and
07:03
It was still freeware. So in 2009 I rewrote for the second time the whole program. Now it's in C++ And it used Qt. So now it also works on Linux Linux is Okay
07:20
So and platforms and license it works on GNU, Linux based Distributions there is a package in the Debian officer Repository, sorry, so it's easy to install and Debian, Ubuntu, Mint, etc It should work without any problems and other distributions, but I don't have any package for them
07:45
So if someone is interested, of course, I will be very Interested to have packages for these platforms, of course it also works on Windows XP and newer and it's free software trees and
08:02
and the Google GPL license, so now some Architectural stuff there are two programs When it's called Axionas, it's the main Program with a window where you can edit your scripts and add some actions or and also
08:22
you can Start them, but there is also a second program called ActExec, which is a small program without any interface that allows you to execute a script. It's used Also on Windows because on Windows you can make some specific script, which is actually
08:42
7-zip Archive that is automatically that is auto extractable so you can run Script on a computer where Axionas is not installed. I Haven't done that for Linux because I don't really see his interest in that and it would be a lot a bit more Complicated and I don't know if there is a really a need for that, but that could be done, of course
09:06
Actions are in packs and each pack is a plug-in. So it's easy to write new actions Each action could be a plug-in but the idea of the packs would have to To share some dependencies between the actions in one file. This was a main idea
09:24
90% of the code is platform independent because it's cute. So it's really not a problem to port that to a new platform The 10% are of course some specific stuff like emulating a click and things like that which don't exist on Qt
09:41
So that's depend of course of the operating system, but it's a code which is not It's not really separated from the rest that that's really a problem. But I have a plan to To put that somewhere which is which is which could be more easy to port that to new platforms
10:01
So of course help is needed because there are lots of new ideas lots of improvements Some bug fixes that could be done For instance For the new ideas, there is a script resources idea And the basic idea of that is to take a file which a script depends on for instance an image that you want to find
10:24
And to encode that in base 64 and include that in the XML of the script file I have already done some coding for that, but it's not finished yet. A script recording, the idea of that is to Allow to have a button when you can click and it will record the actions you do and automatically add them
10:45
mainly works for clicks and keyboard Actions. The problem with that is that there is no cross-platform way to store a key It's really complicated because each platform has its own way and own ways of storing them and defining them
11:01
So that's actually more tricky than it seems There is also an idea to wait for input the idea of that is to say we want to wait for the user to click or To press any key and should work even if Axionas of course doesn't have the focus so On Windows it works for instance. It works by making a hook
11:25
Yeah, the sound recording That's obvious. A task planner, the idea of that is to have some program running all the time and waiting for some conditions to be fulfilled and Execute one or more scripts and of course the next idea is to be to parallelize
11:44
execution of scripts Because yeah, and that that of course adds some There's some issues with that because what happens if one script want to move the mouse to one position the mouse Closer of course that's mouse itself and anyway and another one wants to do the same
12:04
What happens then? So it could be a problem that has to be solved and Other idea is to make a script database That allows users to upload scripts and download script from others. Of course, there will have to be some
12:22
Some checking for scripts because a script can potentially do Anything on your computer so it's better if it's not some evil stuff, so yeah And other things web actions idea would be to execute some actions on a web page using JavaScript. There are of course
12:44
More specific software doing that and of course probably much better, but idea is to have to some basic stuff to do that Acute 5 of course would be a good idea usability improvements Documentation there is also some there is already some reference documentation, but tutorials would be a good idea
13:04
Packaging there is a packaging for Debian, but it would be nice to have some packaging for other distributions Macintosh would be a good idea. I don't have one. So if someone wants to of course, it's really welcome If you need more information, there is a website There are three different mailing lists one for developers two for users
13:25
the source code is available on github and Yeah, that's everything. So thank you everyone. Thank you for them, of course, and if you have some questions I will be outside near the door Thank you