Automate your tasks with Python and publish with Chat Apps
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 130 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/50002 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 20201 / 130
2
4
7
8
13
16
21
23
25
26
27
30
33
36
39
46
50
53
54
56
60
61
62
65
68
73
82
85
86
95
100
101
102
106
108
109
113
118
119
120
125
00:00
AutomationTask (computing)Online chatMobile appVirtual realityPersonal digital assistantCoroutineDatabaseGraph (mathematics)Process (computing)Heat transferOperations researchAttribute grammarBlogMathematical analysisSystem programmingDependent and independent variablesGroup actionInterface (computing)Programming languageScripting languageClient (computing)Computer configurationDisintegrationCustomer relationship managementChatterbotMountain passSoftware developerImplementationIntegrated development environmentDependent and independent variablesSoftwareProxy serverMultiplication signExpert systemMessage passingPhysical systemComputer programmingLevel (video gaming)Water vaporBitBoss CorporationCASE <Informatik>Computer configurationInsertion lossReal numberPoint (geometry)Graph (mathematics)Data storage deviceGroup actionPower (physics)ImplementationEndliche ModelltheorieRule of inferenceInterface (computing)Line (geometry)Food energyTask (computing)Server (computing)TouchscreenDifferent (Kate Ryan album)Execution unitSoftware developerReal-time operating systemUser interfaceCore dumpElectric generatorSubsetINTEGRALAreaClient (computing)Customer relationship managementProgramming languageIntegrated development environmentRight angleForm (programming)MathematicsWave packetFraction (mathematics)Single-precision floating-point formatScripting languagePasswordAttribute grammarMathematical analysisPersonal digital assistantCoroutineDatabaseConfiguration spaceGastropod shellVapor barrierVisualization (computer graphics)Cartesian coordinate systemAdditionMereologyComputer animation
08:26
Process (computing)Computer configurationCustomer relationship managementDisintegrationCoroutineInterface (computing)System programmingGroup actionMountain passScripting languageSoftware developerImplementationIntegrated development environmentComputing platformServer (computing)Installation artComputer programmingProgramming languageControl flowNormed vector spaceUser profileRepository (publishing)Real numberTelecommunicationExecution unitMenu (computing)Cellular automatonInclusion mapLie groupConfiguration spaceSelectivity (electronic)Multiplication signComputer configurationPhysical systemPersonal digital assistantServer (computing)CuboidSoftwareData storage deviceComa BerenicesWorkstation <Musikinstrument>Interface (computing)ResultantAttribute grammarExpert systemSubject indexingSlide ruleVideo gameMiniDiscSystem callExtension (kinesiology)TouchscreenGroup actionMathematicsAdditionParameter (computer programming)Sinc functionNumberNetwork topologyContext awarenessWhiteboardAirfoilComputing platformINTEGRALForm (programming)Scripting languageUser profileCustomer relationship managementWeb 2.0Token ringTablet computerLocal ringHigh-level programming languageSpacetimeTelecommunicationCore dumpDependent and independent variablesProfil (magazine)ImplementationCASE <Informatik>User interfaceModeling languageProgramming languageFunctional (mathematics)Operator (mathematics)ChatterbotCodeLetterpress printingLevel (video gaming)Adaptive behaviorComputer fileComputer animation
16:43
outputTerm (mathematics)Axiom of choiceSimulated annealingComputer-generated imageryView (database)Wage labourServer (computing)Personal digital assistantInformationSource codeGroup actionGraph (mathematics)BootingOnline chatTable (information)SoftwareSystem callGoodness of fitPhysical systemSupersonic speedSpeech synthesis2 (number)SpacetimeWorkstation <Musikinstrument>Dependent and independent variablesComputing platformTask (computing)MultiplicationData structureDirected graphData dictionaryGroup actionGraph (mathematics)Greatest elementLibrary (computing)DemosceneResultantInstance (computer science)Medical imagingTraffic reportingStability theoryServer (computing)Computer fileExpert systemClassical physicsCASE <Informatik>Right angleLevel (video gaming)Customer relationship managementData storage deviceHeegaard splittingSlide ruleProcess (computing)GoogolFunctional (mathematics)Client (computing)BootingAnalytic setUniform boundedness principleScripting languageMereologyoutputInformationStructural loadLetterpress printingJava appletInternet service providerAnalytic continuationOnline chatWeb pageChemical equationXML
25:01
Message passingPlanningComputing platformMeeting/Interview
25:54
Connectivity (graph theory)ResultantTrailComputing platformScripting languageExterior algebraConfiguration spaceRevision controlInformation securityServer (computing)CASE <Informatik>File systemData managementTouchscreenCircleData storage deviceComputer configurationSoftwareWhiteboardLevel (video gaming)Meeting/Interview
Transcript: English(auto-generated)
00:06
So hello, everyone. I would like to talk about automation with Python and the way we found to share this automation within the company. After several iterations, we found the way
00:25
I will speak about. And before we start, I want to narrow I want to narrow the area we will speak about. We will consider mainly operational tasks, which are a real time in most cases, the
00:46
tasks like to support infrastructure to work with customer issues, to work with incidents, and other operational tasks. So not not the development of programming. So let's, let's start.
01:06
At first, whom we will speak about. Here on the screen, you see the employees of the of the company. And these employees experienced in some subject area and able to write the automation
01:26
scripts of their time part of work. But let's see why these experts should work with certain routine tasks. They have access to some applications for some software, they in charge
01:46
for of the software, access to some databases, some network access, access to some API's ensure they have knowledge of how to interpret data and and what to do with this data.
02:06
So let's consider the idea. What if experts create a visual assistant for themselves, they can equip his workplace as their own with the same access to applications to databases and
02:23
so on. And they can transfer their time tasks to the to the system. This will free up experts time for new not retained tasks and will save time of tasks processing. Although there are barriers, programming a virtual assistant in itself requires additional
02:47
expertise, time and costs. Let's see. Let's we will consider in my talk, how we can how we can solve this, this this barriers. Let's continue. In this slide, I want to
03:10
to discuss what are the routine of operating activities for it specialist, what is usual time getting changing attributes of an entity. The simplest example of an entity is your user,
03:29
for example, the user in database anywhere. The next point collection, analysis of locks, configuring systems, it can be very routine tasks, incident response, you need to give a
03:45
response fast, you maybe you need to reconfigure a few servers to response this incident and you need to do it fast. And here, here, let's consider first step of automation.
04:07
At the beginning, at first, a real time actions are performed manually using web interfaces of systems, client applications of systems, SQL clients, API clients, SSH clients, whatever to
04:23
access the system and to to to solve the problem to reconfigure the system to get the data from database to change the configuration and so on. Then, to save time, employees
04:41
automate their tasks using a high level, let's say automation friendly language like Python. Another example can be shell script, for example, but Python is very popular for such tasks for for for task to automate some time work.
05:10
Sorry. However, it is still a specific employee who runs and maintains the automation scripts, because probably you you you can guess it's I think you know that it's hard to share the
05:26
automation scripts, because you cannot share your network accesses, you cannot share your your logins, your passwords, you even cannot share the scripts with somebody who not experienced with the Python with with with the systems. And you also need to share environment for the
05:46
scripts. It's hard to share such scripts. And before I will tell you our solution, let's let's check this options. What are the usual options to exclude our experts from
06:06
a routine processes? The first, the first, let's say it's it's usual, it's it's it's right way, but this way is expensive way. Integrate a routine into interfaces available to other
06:23
employees. For example, client relationship management system, or some system to orchestrate your servers, your containers, services, whatever. But but this, this interface,
06:41
there are many such interfaces, usually in a in a company, and one interface will be not enough, because there are many tasks and and there are many different groups in company teams in company with the different tasks with the different software available in this company with the
07:03
different interfaces. And in the you will need time for sure money to for this development, you you will need analysts and developers and the speed of implementation of change will be
07:23
usually not not the best not not the very fast for this case. Another options give access to your interfaces. Sure, it's dangerous. I think we will not stop here in details why it's dangerous. And it's requires training. And a role time did not go away,
07:46
we just shifted to others, we just relaxed our experts, but other still will will do this through time. Next options, plus your automation scripts to others. Again, it's dangerous,
08:02
accesses logins, password environments, and so on. And the other people need the competence. So these two options is it's unacceptable. The last options, the last options, develop a single
08:23
interface for systems that are managed by a group of experts. For example, the group of experts who is in charge of all the databases, they can develop or some API's they can develop the web interface or chatbot or even API to access to their systems for for their
08:50
colleagues. And through this interface, the colleagues can run some automation scripts, get the results, apply some changes, rewrote traffic, for example, the, the get the
09:03
attributes of user and so on. But this requires additional expertise from experts. They, they usually an expert in their systems, and they don't know the most of them maybe don't know
09:22
how to use chatbots, how to implement web interface, how to implement API's, and so on. So they need to spend the time for this, or they need to ask other colleagues to do it. But we choose these options, this option, and we removed the cons. And I will explain you our
09:51
solution. So we developed a platform, a platform for publishing automation scripts within a company,
10:06
and we published that platform in Python package index. So it's available for all of you. Let's see how to use it. We can take a server, make the necessary network accesses
10:23
create credentials in the necessary systems. So it's like on that slide, the virtual assistant, we equip the workplace similar to our workplace for this virtual virtual assistant.
10:40
Install the platform, we call it skill pub, skill publishing on the server, create a user board in the corporate messenger slack. For example, I, as I know, we, this slack is the most popular corporate messenger and give it to the platform. We will see how to give it
11:07
on next slides. Then the magic happens. Any, the star, any we will, we will talk about the star letter, any script, we can call it skill in a high level language. Python can be used
11:26
in the platform without adaptation. Without adaptation, it's like a, like a magic. Yes. The, the platform itself will integrate the script with the messenger and JSON API,
11:42
uh, provide access control, uh, monitoring, logging, uh, the outer, um, of the script only, uh, determines who can use the skill. So he, uh, who can access the skill. So let's, uh, let's see how it works with, uh, with the examples on this slide. Uh, you see
12:05
the, uh, the, uh, con configuration con, uh, just song, uh, that's only one, uh, config file for the platform. And here you see the channels, uh, for, for our example, I used two channels,
12:24
uh, telegram, uh, the other popular messenger and slack. And, and we only put the tokens here and the tokens is enough, uh, for platform to communicate with, uh, uh, with the messenger. And we, uh, add one user, James James use channels, channels, slack, and, uh, telegram.
12:50
And here is a username, uh, in slack and the ID in telegram. So James has access, uh, to the platform, uh, through, uh, slack and telegram. And here is a, uh, hello world, uh, script,
13:09
uh, for the platform just print. Uh, we use a hello from, from outer space. Uh, and you see the script, uh, is a script has, uh, nothing about the communication with the slack
13:24
or with the telegram. It's just a print it's usual item, uh, script. And you see the James can run the script, uh, from the slack, uh, and, uh, receive the result. Uh, he can see the
13:40
print, uh, the same in, uh, telegram, uh, all these, these scripts, uh, in folder skills can be run such way. Uh, let's see, uh, another example, uh, from real, uh, from real life. Uh, I am, um, I am solution, uh, architect, uh,
14:04
in a Russian, uh, telecom operator. Uh, and this is a very, very user, uh, usual, uh, case when I need to get some, um, uh, some user profiles, uh, subscriber, our, our subscriber profile from them, uh, from the, uh, repository, uh, one of our, uh, software model. Uh,
14:29
and there was a, uh, uh, there were many cases, uh, for me at the beginning when we started, uh, uh, our network, when we started our servers, uh, service for me, please check the,
14:42
the subscriber, uh, what's wrong with, uh, with, uh, his, um, uh, profile, uh, please update the profile and so on. And I cannot give access to support engineers, uh, to the server, uh, and, uh, uh, to move, uh, this data to other systems like, uh, billing, uh, like, uh,
15:07
client relationship management. Uh, we need time, we need integration and so on. Of course we do it, but, uh, we do it step by step. Uh, but I need response, uh, uh, I need fast response, uh, for, uh, for big, uh, number of, uh, uh, support requests. So I did, uh, the, the,
15:29
the one of the first automation it's, uh, quote like this, the pure Python code where, uh, we, uh, get them, uh, parameters. You see this, uh, uh, parameters, uh, it's me in Russian,
15:44
sorry. Uh, I call the script and I gave, uh, the parameters to, uh, to the scripts, the phone number. And, uh, I used, uh, these, uh, parameters in script, the usual way, uh, the same, uh, when we run the script, uh, in shell, uh, then I use these, uh, uh, number
16:07
in the script. I connect to the subscriber profile, a repository, get the data and print the data. So it's, it's simple, but it was useful in the life, uh, uh, in our work.
16:21
Of course, uh, this script, uh, now it's, uh, uh, it's, uh, has, uh, this script has, uh, many other functions like update the profile and, uh, uh, other things to do with, uh, with the subscriber. Uh, let's see other examples. Uh, do you remember the star, um,
16:44
near, uh, to any script? So not, uh, any, some scripts, some scripts can be adopted for the platform. And, uh, uh, here we see the example how script can be adopted for, uh, for platform. You can use, uh, the structures like this print and the Python dictionary. Uh, it's,
17:07
uh, uh, usual structure, uh, dictionary, uh, just the print buttons and the names of the buttons. Uh, and you will see in Slack, the buttons and the, in telegrams in telegram, you,
17:20
you'll see the, the same buttons, and then you can input the data, uh, the usual for Python way, input the data, uh, from the user, uh, and the user will print one of the, one of the bad, bad button, and you will get the text from the battle. Uh, okay. Let's, let's continue,
17:45
uh, more features, uh, displaying images. Uh, this script, uh, uh, is, uh, isn't, uh, adopted for, for the platform. It's you, uh, it's a common script. Uh, you see that it's a library,
18:01
uh, it's Python library, uh, for, uh, for showing the image. And, uh, uh, here we get the image, uh, from, uh, Wikipedia, uh, get the image of the space station and just to show this image and you will see the image in Slack. Uh, this, uh, simple example, uh, maybe, uh,
18:26
looks like we were playing, uh, we just have fun. Uh, but it's very usual in, uh, operational, it's very, uh, useful, uh, in operational, uh, work. For example, uh, I am, um, I am
18:42
responsible for some part of the network. I am at home, uh, at night, somebody from, uh, from support team, call me, uh, ask me, uh, the reason, uh, there is an, uh, uh, incident, uh, we need to solve it. Uh, we need your support. Uh, you are responsible for the part
19:02
of the network. We already, uh, did all we can. So I just, uh, before I, um, uh, wake up and get my notebook, uh, I can, uh, push the buttons in Slack and I can get all the important images from our, um, monitoring system like a Grafana or Zadix. And, uh, you can imagine
19:26
that, uh, on this picture, not the space station, but, uh, the, your traffic, uh, on your servers, your users requests, and they goes down and you need to do something. And, uh,
19:41
in, in 10 seconds, you already see the most important, uh, pictures and, and, uh, probably you already know what to do. Maybe you will do it. You will run another script you have on the platform and you will reroll the traffic. You will, uh, reload, uh, reboot the servers
20:00
and so on. You will, uh, make an action. Uh, so more features, uh, more features you can find, uh, all internet, uh, uh, on the GitHub. Uh, you can search in Google skill pub GitHub,
20:20
and you will find this page. Uh, for example, the, you can print the file, uh, some other features. Uh, I will only tell you maybe that, um, uh, not all the features, uh, are described, uh, here, uh, uh, I'm working on this. Uh, uh, but now the platform, uh, is,
20:45
uh, uh, now platform, uh, has, uh, um, um, many functions, uh, many new functions. I will add the possibility to share, uh, the requests from Slack, uh, uh, for multi instance, uh, of the
21:09
platform. So you can, uh, uh, you can launch, uh, the multi instance, uh, uh, reliable, uh, platform and, uh, balance the load for this platform. Uh, but your letter, you, uh, if you,
21:26
uh, if you will, uh, in interest with the platform, you will find it here. So let's, uh, continue. I think I have just a few, uh, this, uh, uh, one more, um, slide how we use it in customer service. So, uh, customer service is a business process.
21:45
And on this slide, you see the, um, uh, the usual, I think the, uh, uh, process, uh, how to solve customers, uh, issues, the first level support, uh, well, the guys, uh, who work,
22:00
uh, uh, in first level support use a knowledge base, uh, and use a client relationship management. Usually this, uh, two, uh, two systems, uh, and if they cannot solve the problem, they are send the ticket to second level support. Second level support use, uh, various
22:22
tools, additional to client relationship management, uh, and these tools are connected to the network, to, uh, to, to our, uh, servers, uh, services, databases, APIs, and so on. And if the second level support cannot solve the problem, uh, they send, um, the ticket
22:44
to SEB, uh, subject matter experts. And these guys, uh, I am one of these guys, uh, use the platform skill for up to automate the, um, uh, uh, erotine tasks, uh, the tasks, uh, um,
23:02
related to the support. For example, uh, it is just one example of such tasks and they can, uh, give them, uh, skills, uh, give the, uh, automated, uh, skills to the second level support and, uh, uh, uh, second level support, uh, in, uh, uh, when they will have, uh, the same
23:25
ticket, uh, with the similar pro pro problem, they will just run the, uh, the script through the Slack and, uh, ever, uh, the result. So that's probably all for this. And, uh,
23:41
I think that this one is lost, uh, more use cases, uh, just to note, uh, some other cases, uh, maybe repeat some cases, uh, we use it, uh, the, the platform and this way to, to share automated scripts, uh, we use to fetch some diagnostic information from multiple data
24:04
sources, uh, into chat, uh, up into Slack, uh, gather metrics, logs, and analyze them, uh, to take action in case of incident, right from chat up, uh, eroding user requests, server rebooting, launching new instances, and many other actions, uh, to give easy
24:23
access to API's, uh, via Java for teams, uh, members and other colleagues, uh, to provide analytical re reports. This one is very, is very popular, uh, by the request from chat up, rewrote, uh, reports with graphs, images, tables, files, et cetera. So, uh, I think,
24:46
uh, I'm done. Uh, and this one, uh, this just, uh, uh, thank you Python community. So thank you for everything. So we got a few minutes, so we can read two questions.
25:15
So let me check them. Um, I can see the questions can select Skype, uh,
25:22
be, uh, replace it with the messenger, such as Cisco Jagger. Uh, it, uh, now we support, uh, two messengers, uh, telegram and Slack, but, uh, we plan to add, uh, other messengers in, in company. I work, uh, in, uh, we use only two, we use Slack and telegram, but, uh, to,
25:43
uh, to, uh, to develop the, the, the platform we have plans to add some other popular messengers. Uh, is there a chance, uh, to self host us? Sorry. The question was, ah, I see. Uh, is there a chance to, uh, self host a Slack alternative that works
26:02
with skill pop? Uh, so now, um, uh, the alternative, uh, for slack slack is not, is not self hosted. It's impossible for slack. Uh, so, uh, we need, uh, uh, to integrate other messenger, uh, but probably we will do it, uh, in the future. So in future,
26:21
you will have such chance, uh, but now, uh, you have other option, uh, to use, uh, the just on API, the skill pop will build, uh, the just on API for the, uh, scripts and, uh, you can run it instead of, uh, uh, messengers. The, this is a one, one more option. Uh, the next one are their best practice for managing, uh, credentials with skill pop
26:46
hard coded in the scripts. Uh, it's hard coded in the scripts, but you can, uh, access this, uh, con config file. And we, we already do it. Uh, we, uh, now we're, uh, on the, our way to develop, to, to integrate our platform with, uh, uh, company, uh, service desk,
27:06
uh, the software. Uh, so the, the people will, uh, request access in service desk service desk will not notify us and we will add, if the, the, the request was approved, uh, we will add the
27:20
user, uh, to our, uh, platform. Uh, we will do it and, uh, it's already, uh, easy to do if we, if you will just, uh, integrate with the conflict file, it's simple, just so and so you can access it and you can add the users. What, uh, what to do, uh, in case of an outage with
27:43
telegram, slack, uh, or board, uh, so, uh, in case of outage, you will not able to access, uh, the platform. So if you will not have the, uh, other, uh, for example, API, if you use the API, it will still work. Uh, for example, for us, we have, uh, we, um,
28:06
our, uh, security engineers, uh, security team, uh, approve approved only slack for, uh, uh, for our usual work, but for the case when slack, uh, in outage, uh, we, we can switch on
28:27
telegram. So this is our, uh, the, uh, how to say the second option, uh, where do the scripts leave? Uh, uh, and how are they connected to our slack? What, what, what's the scripts
28:43
leave on the, on your server on, on, uh, they, they're just, uh, uh, uh, the, like in file system in your server and the connection with the slack, uh, only through the platform platform connect to slack, get the request from the slack. They, uh, then check the access of this person.
29:02
If this person, uh, uh, uh, have access to the, to the platform and to the script, uh, so the platform around the script and, uh, uh, run it locally, short locally on your server, uh, run it locally, uh, get the result of the script and send it back to the slack.
29:23
Okay, cool. Thank you very much. If anyone wants to continue the discussion with Anton, you can do that in this corner. You can find him there. There is a channel for this talk in particular, and you can also find him in the, in the Brian track. So thank you very much. Have a good day. Thank you. Thank you. Bye.