Unlock the Power of Dev Containers: Consistent Environments in Seconds!
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 131 | |
| Author | ||
| Contributors | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/69516 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 20248 / 131
1
10
12
13
16
19
22
33
48
51
54
56
70
71
84
92
93
95
99
107
111
117
123
00:00
Power (physics)ConsistencyIntegrated development environmentPower (physics)Point (geometry)Video gameDemoscene2 (number)Programmer (hardware)Computer animationLecture/Conference
00:34
CybersexInformation securityIntegrated development environmentBuildingConsistencyPower (physics)ConsistencyPoint (geometry)View (database)Key (cryptography)Integrated development environmentSoftware developerInstallation artComputer animation
01:22
Integrated development environmentComputer programmingPoint (geometry)Software developerCodeWebsiteFront and back endsServer (computing)Video gameRevision controlView (database)Library (computing)Virtual realityWeb 2.0Installation artImplementationProjective planeCASE <Informatik>Metropolitan area networkProduct (business)Web serviceMultiplication signSoftware frameworkVideo game consoleMobile appType theoryDataflowTensorProcess (computing)Boss CorporationGoodness of fitSingle-precision floating-point formatRepository (publishing)Computer animation
05:21
Video game consoleInstallation artSynchronizationRootMessage passingSoftware frameworkHand fanInterpreter (computing)Virtual realityChaos (cosmogony)Local ringRevision controlDependent and independent variablesPoint (geometry)PiNeuroinformatikComputer configurationLevel (video gaming)BuildingBinary codeSoftware maintenanceVirtual machineCodeExistential quantificationOrder (biology)Integrated development environmentShared memoryComputer programmingPower (physics)Computer hardwareConsistencyHand fanInstallation artPhysical systemInterpreter (computing)Operating systemLibrary (computing)Moore's lawProjective planeExtension (kinesiology)Game controllerDataflowTensorData miningGradient descentPlanningComputer animation
11:10
Interpreter (computing)Video game consoleInstallation artPhysical systemPoint (geometry)BuildingComputer configurationComputer fileConstructor (object-oriented programming)Cartesian coordinate systemLibrary (computing)Integrated development environmentSoftwareSoftware maintenanceProgrammer (hardware)Repository (publishing)System administratorJava appletMatching (graph theory)Mobile appDatabaseType theorySoftware developerConsistencyServer (computing)FreezingLine (geometry)Symbol tableWeb 2.0Directory serviceMedical imagingQuicksortSet (mathematics)Basis <Mathematik>Computer programmingExpected valueBookmark (World Wide Web)WordPoint cloudFeedbackContext awarenessProjective planePhysical systemVideo game consoleOrder (biology)Level (video gaming)Installation artMathematicianMultiplication signWeb-DesignerData managementOperating systemPlanningTouchscreenInterpreter (computing)Coefficient of determinationMultiplicationLocal ringData Encryption StandardRevision controlComputer animation
20:15
Video game consoleInstallation artInterpreter (computing)Object-oriented analysis and designComputer-generated imageryOpen sourceCone penetration testCASE <Informatik>Repository (publishing)Matching (graph theory)Vulnerability (computing)Computer fileService (economics)PiComputing platformLatent heatForm (programming)Multiplication signProjective planeKey (cryptography)Virtual machineLibrary (computing)Point (geometry)SpacetimeCodeOpen sourceIntegrated development environmentSingle-precision floating-point formatType theoryComputer configurationStaff (military)Level (video gaming)Physical systemInformation securityCybersexPlastikkarteLine (geometry)Product (business)ConsistencyRevision controlComputer programmingStress (mechanics)Software developerInterpreter (computing)Selectivity (electronic)Reading (process)Descriptive statisticsRemote procedure callMedical imagingArc (geometry)Parameter (computer programming)Local ringLaptopNeuroinformatikComputer animation
29:21
Information securityInterpreter (computing)Total S.A.CybersexPlastikkarteBuildingConsistencyPower (physics)BuildingRevision controlCASE <Informatik>View (database)Client (computing)Connected spaceVirtual machineServer (computing)LaptopInstallation artDirectory serviceComputer fileGraphics processing unitPoint (geometry)Medical imagingShared memoryMathematicsOpen sourcePhysical systemYouTubeEvent horizonSpeech synthesisInformationArc (geometry)Interpreter (computing)Remote procedure callRepository (publishing)Projective planeComputer hardwareMultiplication signService (economics)VirtualizationCloud computingDemonRight angleOrder (biology)NeuroinformatikContext awarenessVideo gamePlanningMobile appSoftware development kitCodeComputer animationDiagramProgram flowchart
38:26
CybersexInformation securityPlastikkarteConsistencyBuildingPower (physics)Object-oriented analysis and designRouter (computing)Computer fontQuicksortAiry functionSystem on a chipService (economics)Speech synthesisVoltmeterCloud computingData conversionComputer programmingSimilarity (geometry)Roundness (object)Goodness of fitExterior algebraLevel (video gaming)Point (geometry)Latent heatKernel (computing)BitGroup actionRepository (publishing)Presentation of a groupSynchronizationInformation securityPhysical systemLine (geometry)Computer fileSampling (statistics)DemosceneCodeIntegrated development environmentGastropod shellSoftware developerPoint cloudMathematicsVapor barrierSoftware repositoryMoment (mathematics)Online chatKey (cryptography)ImplementationOpen sourceDirected graphGreatest elementRight angleComputer animationLecture/Conference
Transcript: English(auto-generated)
00:04
So Unlocking the power of deaf containers. What can you expect from the next 45 minutes? So I want to get two points. The first point is that I want to present you the basic concepts of deaf containers in the sense that you
00:25
Know what's going behind the scenes when you're using deaf containers and how it will support you in your everyday life as a programmer and the second point I perhaps Just come back to the keynote
00:40
yesterday from Carol so in the past So in the 90s 80s, it was hard to answer Python nowadays it might be easier to answer Python, but It's my point of view. We still got a problem in getting consistent Python development environments and
01:03
My point of view is my opinion that deaf containers are one way to tackle down this problem so and When I speak of consistent Python development consistency, why is consistency so important? because
01:21
Consistency is the key for reproduction So When do you have to reproduce a Python development environment quite often? So for example when you start programming You spin off your Python development environment You leave the project come back three months later, and you have to reproduce it
01:44
Such that the code works and runs as it did three months earlier We have to reproduce Python development environments. For example, when we do local development want to ship our code to production Then on the server it should work as it did in development and
02:04
Last but not least and that's I think the most important Thing my point of view. So whenever we want to share our code to the community or to France So when you push it to a repository We put perhaps also some documentation inside how you can
02:22
Well check out the repo and get started developing And Let us be honest. How often did we fail in reproducing in development environment based on documentation? So that container technology is a promise a
02:42
promise that You get a working development environment with only a single click Yes, and to fill the thing with life. Let's have an example examples are always good. So Let's say you're a fresh backend developer. You want to use Python and you get a job to
03:04
well program a web server an API and Most probably you start searching which Python Web servers are available and you see okay fast API is the best one Okay, who's using fast API who knows fast API?
03:22
Okay quite a lot the others please think of flask as a web source, etc So I think we got then a quite a big coverage of this use case so whenever you want to implement a web server using Python you go to your Preferred framework to the documentation site you copy the example code
03:41
So here's a simple hello world program of fast API from the documentation side and you read on the documentation It tells you okay going to go to the console You have first to install the fast API package and the you recall package to a pip install so the packages get installed in your environment and
04:02
Next step is to Execute the web server to run the web server. So you type in you recall main app There are other possibilities I heard yesterday, but let's say we are using this command and hooray hooray. You get a working web server well, maybe So if you're really a fresh man fresh girl you're starting this
04:24
Yes, it will work for sure because you've not yet installed other packages, but time evolves you're doing more project you install more packages and Time will come for sure when a package says no, I don't want to install the dependencies wrong
04:40
I need another version of this package and then you google again or search again and see okay I have to start with virtual environments You spin off a new virtual environment you go on programming as a backend developer until your Chef says the boss says okay. You are now an ML engineer. You're now doing
05:01
stuff with tensorflow or Other things and you come to a point where you have to install system libraries and at least at this point you are leaving the Python ecosystem where it comes to a problem and That's also the point where
05:21
Your local development environment Will perhaps look like this so where it descends into chaos because you have to so much projects know much virtual environments so much interpreters because perhaps also you cannot stay in Python 2.7 you have to go to Python 3 and
05:41
Well, I love this picture from Randall Munroe because I Know if he did it by heart or if really a plan, but he captured really very well were Development environments decent into chaos and what you need to fix in order to get a consistent development environment
06:01
so Let's analyze this picture If you want a consistent Python development environment Which is working now and also in three months you have to keep care about the packages So that's the first level
06:21
You saw me earlier when I wrote down this pip install fast API Your code won't work in three months or one year later if you're not fixing to package version So you have to mention? Okay, I already fast API 0.1 hundred eleven point zero. Yes, they are at 0.1 hundred eleven
06:42
I don't know when they reach one point zero ask a maintainer, but they never get response still waiting so You need to fix the package version definitely and to be honest you even need to fix the building wheel and That's no interesting question for me who heard about building wheels in this room
07:04
Okay 50% For those who don't know about building with let's make a small extent to this. So whenever you do a pip install Your machine is most probably looking for a building wheel which matched to your Python interpreter and
07:21
your system If it does not for example if you want to insert mumpy it will compile C on your machine so for example When I did programming on a Jetson nanos on what you call it perhaps embedded device, but it's not really embedded
07:41
Perhaps ml embedded There was no building wheel and when you do a pip install numpy you're waiting hours Until it's compiled because you don't have so much in computational power. So in order to do a faster install of packages Packages are from some maintainers shipped with pre compiled binaries matching to your Python interpreter and your system
08:05
So If you really want a hundred percent pure fix of your package that you use you should also fix the building wheel So Let's come back the second level the interpreter level Because not every package work works with every interpreter
08:25
so some packages have restrictions working just with Python 3.11 and when Python 3.11 which is required doesn't reflect your system Python interpreter. You have to keep care about it So previously the package thing when you want to fix it
08:44
There's a really big ecosystem. You can use rye poetry Anaconda whatever. So there are really a lot of possibilities to do so and you can use your preferred way to manage your packages minus for example poetry because also Fix the building with but when you come to the interpreter level we get not as much options
09:05
There are some for example pie and but there are drawbacks. So I've just been to Python Germany Python Berlin and I met you can mark. He's one of the maintainers of Ubuntu and
09:20
he mentioned in his talk when you're doing pie and on Ubuntu system, you might crash the system because Yeah, they're expecting that Python version is Fixed at somehow and they're just built the whole operating system around this Python interpreter That everything works fine
09:40
So if you're using pie and you might break at least your Ubuntu system our system might behave differently and then we come really to the point where We have to leave the Python ecosystem because I also mentioned the system level so This is really just level down there. You can't control from the Python ecosystem
10:04
Whenever you install for example TensorFlow as a Python package you have to also install system libraries CUDA libraries or something like this in order to get the full power of hardware acceleration so
10:22
What's the consequence The consequence is binary you can now you say, okay Happens there's no way out. I Can't share my code with my friend But I don't like that option. Not really. I'm more a Star Trek Captain Kirk fan. I don't accept
10:43
Something's not working. No If it's only running on my machine, then we'll ship the machine. Yes why not and There's a golden rule in informatics Copy as much as possible and look around well and anybody else had the same problem and in fact, yes
11:03
There were same problems around them for years and the answer is Use containers Now next interesting question will be it's a beginner's talk so it's completely okay if you've never used containers But I just want to get feedback who ever used containers
11:23
Wow, that's pretty who did not Okay, one two three, I don't want to lose you and I can also tell some anecdotes so So containers are there for some years and there's a reason why there are containers so I grew up in 90s and I had a bare metal server in my room and I
11:42
were running a web server there and When you need it next to the web server or a database So you had to be lucky that the system libraries the web server and the database needed Matched Otherwise you get problems and a lot of administrators of that time had these problems and I talked about okay
12:04
Is there a possibility that I can just put my application in splendid isolation? Such that there's a sandbox around and yes, they got the idea of Container rising software, which means that you ship your software with its own
12:21
Operating system somehow such that every application can have its own libraries such that the libraries match to the application and That's basically the thing what containers are about and nowadays as time gone by there's one container technology
12:40
Which is really well known and widespread. It's darker. So darker really popularized a container technology But I have to admit there are other options container. I owe Just give me some names after but so they're bland there are multiple possibilities to containerize things But let's go to dog away and
13:04
Let's come back to the example how To dockerize our web server from the beginning so We go to our console and So we remember all three levels we have to fix in order to get a consistent development environment
13:25
So the first level is the package level and I can fix the package level quite easy I Can do a pip freeze which? Is piped into a file and I get all the packages installed and the environment including diversion
13:43
Well, not the building rules, but it's fine. It's it's it's okay It's a it's a really good starting point and it's short so I can put it on the screen here But there as I mentioned other options you can use poetry rye whatever multiple options, so Use the package manager you love most and
14:03
Now You have to dockerize it so those who never use containers When you have a container and you start a container you run a container someone on the server every container has a Blueprint and this blueprint is called image. So I'm now giving you some definitions and mathematicians. I love definitions
14:28
so and these images are Constructed by building plans, which are Docker files text files and These Docker files are the construction plan, so
14:43
afterwards when you Constructing it then it's getting an image where which you can put to a repository and there are a lot of images You can just take as another basis of your building So for example, there are might be images Serving as a seller of your house and you're just building on top something else
15:00
And this is what we are doing now. So we are writing a Docker file and the very first line we say, okay We use another image that the Python community provides us the Python maintainers. So we say okay from from this Python Repository let's say We are taking now an image with attack
15:23
3.11 minus bullseye and what's behind that? well The Python maintainers or the maintenance of this repository have taken a DBM bullseye Image which were provided by the DBM maintainers. So first step then they did in best practice manner
15:45
Install Python 3.11 inside this image and Afterwards they're pushing it such that we can use it again So now is we're taking this image and we are copying all our files and our
16:03
directory into This image and say afterwards. Okay in this image, please install my requirements from this requirements TCC so we just pip freeze so and afterwards we say, okay when you ever start when you were ever Well served in some way. So when you ever start up as a container
16:25
Then execute the command you recall main app as we've seen before in the console, so We saved it file go back to the console And now we're doing the thing We are now using this construction plan this docker file in order to build up a docker image and we take it
16:41
So we give it a name my image and say okay just stay in this Folder as context such that everything gets copied as we said, okay Everything's fine. And when we execute it you see that the first three lines are executed So the image Python 311 Bose is downloaded the files are copied and the pip install is done
17:01
But not yet the command executed this will follow in the next line of the console You say okay now docker run my image Which I am just built and now the command gets executed and your web server is starting up in your container So that's the docker word basically how we containerize
17:22
But Is this already a Development environment because this was our goal. We wanted a Python development environment. Well In fact, you're almost there Because what you can do now when this container is running you can go inside the container use your favorite IDE
17:42
whim and start programming Yeah, it will work but it's not the way as you would expect Python Programming to be because when you change some files inside the container nothing get changed on your file text because everything is
18:01
isolated in the container But no problem So if you want to have something like this setting so you get your host system where you're working in my project folder And you just want to get this into some sort of place in the container you can use native docker commands
18:22
And this it is so for now. We are skipping the fingers the docker file. We make it more interactive So we go to the console and say okay just docker run directly an interactive way this image Python 3.11 bullseye and We'll mount our local directory occurring in in the workspace folder inside the container
18:45
So in doing so we again fixed our system In using this DBM bullseye and the Python interpreter 3.11 so what's missing is the pip install we did before in the docker file
19:02
But as we are now inside the container as you see in the sharp symbol We can do it by hands. So we do now a pip install means our requirements The packages get installed again and we run ourselves. So we don't do this Rely on this command thing in the docker file We do it now by hand type in again you recommend app as we did in the start and now
19:25
Congratulations, you got your first Dockerized or containerized Python development environment and you're not really far away from a dev container And in fact, that's the way how I programmed for quite a long time
19:42
I started you use containerized Python development environments, I think six years ago because I was in the team of Java Kotlin programmers Pushing applications to the cloud and containers, so I
20:03
Got used in using containers. I even did front-end development that time just to get some experience again in web development and At that time I also used pycharm and Pycharm you can select an interpreter and at that time I saw that was an option to select
20:27
docker or docker compose interpreter and Then I just thought about it. I just read the documentation. I started to use it Quite in the same manner as you seen it here and I learned to love it
20:40
Because finally when I was the fast API development, I could be sure that what I program Will later also work in production because there I'm using the same Docker images To serve it as a container So this was really quite a big benefit
21:01
Yes so but The talk is about deaf containers. So we have to remember it. So what's between These lines and a deaf container Not much in fact, so if you really want to transfer in a deaf container solution you
21:21
Well at a deaf container dot chase and file into your project and it's a JSON file So and you need three things to write down this chasen. So first of all a name to give the baby a name as we are at a Python conference, we name it Python deaf container and
21:41
Next you have to define the image which run on it's What Python 311 pussy as we did before in our docker run command and then we say it Okay, I don't want to do this post create command. We did Interactively on my own. No, I want you to do it
22:01
Like this so pip install Minos our requirements. So hooray hooray, and there's also type of point xt and You get your first deaf container great now Who thinks the deaf containers is we as code github thing
22:24
Hands up. Oh one person two person Okay, so you're quite right at some point because it got popular with we as code and github code spaces yes, definitely, so this was really a driving thing for this technology, but
22:41
My point is that in fact It's the deaf container technology or deaf container tooling is 90% docker and the 10% service layer on top of it and that's why I think it's a great thing to have and the deaf container Specification is now even open source. So
23:04
It's getting more and more popular and there are platforms supporting a deaf containers where you can just go to and Spin off your development environment remotely or Pi charm is supporting it now spinning off your development around and using deaf container, so it's get more and more popular and it's because
23:26
This service layer on top is really quite nice to have for example it's really taking care about when you're developing with a deaf container that the file permission inside the container and outside the container are in match to the according system and
23:41
It's really fun to use and the cool thing is when you get a deaf container chase and inside your repository You check out the repository you open. Well in my case. I just went back to yes code because of somehow has Or had two years ago better remote computation support and
24:01
support for jupyter notebooks then There's a pop-up and asking you. Oh, hey, I found a deaf container chase and lying inside this project May I open it for you and you say yes, please because then I get a working local development environment Hooray, hooray one click and you're finished
24:21
Well, at least when you got one developer, which is also working using this deaf container so and that's another thing I really laugh about that container, so When you get one developer using a deaf container You get a living documentation
24:42
because the problem with documentation readme's and repositories is that as the Project goes on Nobody really loves to do documentation who loves to do documentation. Who are the documentation guys to okay. Okay, you're hired
25:00
No, so there are always points where the description of how you set up your Development environment Diverges from the thing you write down in the readme But when there's one developer actively developing with a deaf container, it's a living documentation You can rely on when you look on the deaf container chasing that there's a source of truth how to get the project
25:23
Running even if you're not Using a deaf container because you can basically also do the things in the deaf container outside of container so that's the point and And Just to also Stress out that deaf containers are 90% darker or any other container technology. So
25:45
For example when you want to add an environment variable to your dev container For example, you want to add an API key Perhaps the most popular API key. We know one two, three, four five six. I'm pretty sure a mobile hotspot. Here's the same key and
26:03
You're trying to say, okay let's put another argument to our container and So we add the run arcs line here in our dev container chasing where it says Okay Use the end file dot end. So and this is the same thing you would put behind a docker run command
26:27
Good So just let's recapitulate some points Why should you use deaf containers?
26:40
first of all They're reproducible because you're shipping the packages the interpreter and the system as a whole package or a whole package but It's not a fire and forget solution. So you really have to keep care about the three levels
27:01
If you said you have to fix the package version Otherwise the container is useless in the sense of consistency You have to fix the still fixed interpreter level and you have to fix the system level but with staff containers or containerization You got the options to do so so The second thing I mentioned was documentation. That's the thing. I love about you get a
27:25
single source of truth to look at to get the project running and Quite obvious is security I'm now working for a cyber security company smart cyber security GmbH and I just got more and more into cyber security thing
27:44
Smart cyber security is also providing my attend here. So thank you at this point. So Security we perhaps read about That pi pi stopped taking up new packages for some time one day or two days because of vulnerabilities
28:04
software vulnerabilities are a thing which We are much more sensitive now than two or three years before so Even if a package is maintained by Open-source community. They might be vulnerabilities inside which you are not aware of
28:24
because They're also using some other libraries which are installed When using pip install which resolves the dependencies so In fact, you can't be hundred percent sure that you won't
28:42
give access to your machine in Doing a pip install it can't be there's no hundred percent issues and On your machines there is also sensitive data at least when you look inside your dot SSH folder There are some SSH keys which provide access to company resources
29:02
so just think about containerous Python development as a another form of isolation to Be at least somehow more safe that nobody gets access to sensitive data on the host system So and
29:21
last but not least what I life like most about Dev containers is the capabilities of remote computing because You get everything you need inside your dev container and you can program in fact on any
29:41
Computer virtual machine which provides Docker or container infrastructure And that's what I get used to most when I needed GPU resources, so In a perfect company, we got a bare-metal GPU server now we get a virtual GPU server and on this GPU service we get docker installed and
30:05
Now I can use One feature I really like and was which from in two years ago from pycharm to vs code the remote connection capabilities To connect to this remote host and just say okay check out the repository. He's asking me again. Shall I
30:24
Open the dev container from here just up and running those clicks and This is what I wanted to show you in the last 10 minutes Because when I wrote the talk there were not so much information about how to get really
30:40
GPUs up and running remote GPUs using dev containers so when you plan to use GPUs which are not in your laptop because A100 is quite too big for a laptop You just need to connect a remote host which were bare-metal using for example SSH or HTTP so either there is already living a vs code server on the machine or you just do using vs code a remote
31:06
SSH connection and The thing is You can't just run a GPU or a container with GPU
31:21
Capability on this well remote host with the GPU and Docker client Docker installed Demon installed without installing the container toolkit. So this you have to keep care about so there's in order to get access to the hardware acceleration, you need something on top of the native docker system, but
31:44
There are cloud providers which provide Which know about it and which know also about the data scientists for example and not so in common with setting up Machines and which just provide virtual machines with everything installed you need but you have to keep care so it's not so you need
32:03
Docker demon and the container toolkit installed on your bare-metal server your virtual machine which has the GPUs and When you get this You are ready to do some magic which is not rocket science so
32:20
again We want a container with fixed packages fixed interpreter fixed system And let's see how we can write this down Such that it works with hardware acceleration accessing a GPU. So For this we now have again to write a docker file, but we've seen it before and our docker file is really quite small
32:42
So We again go to the Image repositories docker hub for example and Nvidia just provides us an image called CUDA and was the tech 11 point seven point one which reflects the CUDA version and
33:06
They have installed CUDA on well base open to 2020 point oh four so that's our starting point but when you look inside this image when you're running it when you're looking inside, there's no Python installed, but
33:23
Thanks to you mark We got a Python on this machine And we just have to call apt install Python 3 Python 3 pip and we got at least a fixed Python version Which is shipped with Ubuntu 2020 point oh four and we won't Install another one because you already heard it
33:40
May crash the Ubuntu system So and Then we go come to the point where we write down the dev container chase and so we cannot say that in the previous dev container chasing you've seen the reference to An image existing in Docker Hub, but now we need to refer to a Docker file lying inside our working directory
34:06
so we say okay, you need to build something but the building plan is the Docker file and The context is just our workspace again and As post create command you do again a pip install minutes are requirement dot txt. I'm missing the txt again and
34:27
you might ask yourself when we've done the Pip installed before we did it in our Docker file Why don't we do it now inside our Docker file? but as a post create command because it will do it every time we spin off the container and
34:45
The reason is that we want to save startup time Who knows? Uber Ludwig or Ludwig the open source project. No, no one. Yeah. Okay They're also providing a dev container and they are basing the dev container on right and
35:03
The right image installs the the pytorch Versions during build but then that they're installing it in the user directory and What I said dev containers has a service layer. So they're
35:20
matching your file permissions and The user ID of the files out in your host system to the files in your container system And there are a lot of files when you install pytorch and this reads half an hour answers to files got matched to your UID That's why you should Install it later. Then it got installed quite fast than rematching the UID
35:43
So it's just some detail and then again as I would execute this Docker GPU enabled Container I would do add some run arcs. I would say okay take all GPUs because my colleagues don't need any
36:02
And I'm setting the SHM size but that's well documented when you're googling around and In doing so and you just perhaps also mentioned in the requirements txt installing the pytorch Version or pytorch package you will see that there's a problem because you when you're really checking it
36:26
Whether you get access to the GPUs you get a no I don't see any GPUs and that's a small trick. You have to say him. Okay, please install the torch But use this building wheel which is really
36:43
compatible to your CUDA version so in this case You just refer to installing the building view of pytorch Reflecting the CUDA 11.8 version which is compatible to the CUDA 11.7.1 I've seen one guy laughing so you got you know this sir
37:03
Yes, so even so always when there are major changes in CUDA You just have to test which version is compatible to another so it's a common So when you get when you're experienced and that's why my team always says oh We need to get CUDA running or give it Thomas he's an embedded guy he will sniff it and You just need some experience and what's the right version they got better
37:24
I have to really admit they got better in the last two years, but before it was really horrible Good, then we really come to an end to this talk. So When you still have a question, why? Ubuntu serving a certain Python version just I refer to your mark just post him text him
37:43
He'll be happy or watch his talk. It will be on YouTube soon and Well, that's it so One last question who already used def containers Okay, who will give it a shot?
38:03
cool, so my mission is accomplished and I'm happy to hear your questions and you can also connect me on LinkedIn Well, and one final thing because oh no, come on come on but you're a volunteer because I went this is a Conference which is driven by volunteers and professionals and if to really say thank you for organizing this event
38:25
Which making it possible to give a speech here a talk here and you're the possibility to hear talk So thank you to the world is the professionals providing the service And also thank you for Thomas to this amazing speech
38:49
Okay, so this eases us into the Q&A session if you have any questions There's a mic that you can go and just line up pretty please
39:06
Please don't yeah Thank you for the for the presentation and as curious you're mentioned. This is 90% occur Yes, and and well doctor has had some bumpy things regarding licensing recently So some people are exploring alternatives. So can you discuss a little bit?
39:25
if this works with alternatives either similar like bottom and or more different like bubble wrap and Yeah Experience you you have with us Okay So when you just want to have an out-of-the-box solution you are most properly stuck to using darker because
39:46
The we as code implementation is referring to darker. I think I know that there are some guys from From chat brains here Perhaps they can answer the question also whether they're supporting other container technologies using deaf containers
40:02
But yes, you're right. It's at the moment very specific for darker, but in fact And that's what I want to say. It's 90% darker So when you just don't use the 10% service layer You can still containerize your Python development environment. And that's I think the most important thing that you
40:22
So what was the idea of this talk so the idea was that I'm coming from From from a team which was cloud native providing cloud services So I got some other experience at some developers here coming from data science using anaconda, etc So I just want to say that there's a world outside of Python ecosystem, which is able to solve the problems. We all experience
40:50
What I was wondering is when I am running like a shell in this dev container Do I still have access to my system packages or or do they get kind of isolated away as well?
41:02
So I can't speak for Docker technology because this is what I'm using most but there are also container technology so what's going on behind the scenes of Docker so as long as you don't mount Something into the container. It's lives in a really splendid isolation because on a Linux system. It's sharing
41:22
C groups on the kernel level So this is the isolation level as long as the C groups and kernel level are safe You're safe that it cannot go inside to your host system. That's the security layer other So there are some I heard when I gave a talk somewhere else There are some issues with Docker that people asking himself whether the locker is really as secure but in fact
41:46
You have at least one more barrier and C groups are quite a good barrier. So it's just somehow an answer to you Yeah, not entirely, but I'll come back to you later. Okay. Thank you. Next question, please Good question whenever you run that container you need to and you may change this to your code
42:03
You need to read and rebuild the dev container and then rerun it or just rerun it and it will automatically rebuild itself. Okay So As long as you don't touch the dev container chasing we as code for various code won't
42:22
rebuild it so And that's also why I like to have this post create command inside with the requirements txt. So in fact, I'm So when I really do a daily programming I spin off my dev container and I'm working inside the dev container perhaps for one month or something like this and I
42:44
only rebuild it when I really experience breaking changes or if I want to retest it if I'm still in sync was the specification but So it's quite a good feeling You should test it. You should give it a try So it won't it won't stop your programming every 30 minutes and want to rebuild something definitely
43:02
Thank you and one more question, please Just wanted to bring up the docker conversation again, so the licensing issues are like with docker desktop So is this dependent on docker desktop or is it dependent on the docker engine? Like can I use like kolima or something that uses the underlying docker engine?
43:23
With these dev containers. Oh, did that make sense? I Was not aware about the docker licensing issues. Perhaps we should speak afterwards about it Thank you Okay, I can pick really quick one last one from online sources if you don't wanna your API keys in your repo
43:41
How would you easily share these secrets? Okay, I Would definitely never put in dot and file in a repository. So Just make dot amp dot sample file so that somebody else can see what and Really have to be set such that the fingers up and running and that's the way we are normally programming locally
44:05
so we provide a dot and dot sample which is pushed a repository and when you're really Putting your own dot and file inside. You have to just fill out it manually as it is on the dot get ignore your
44:20
Your most probably won't push it your repository so we put the point point and in the dot the dot and the dot get ignore and Then we do it manually Brilliant. Thank you very much. We will try to answer the rest of the questions online online a sink so one last round of applause for