Conda - Easier Installs and Simpler Builds
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 42 | |
| Number of Parts | 169 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/21194 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201642 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
BuildingGradientSymbol tableComputer animationLecture/Conference
00:26
Similarity (geometry)Integrated development environmentComputing platformThermische ZustandsgleichungRevision controlDifferent (Kate Ryan album)Library (computing)Programmer (hardware)Cross-platformFormal languageData managementRevision controlBitIntegrated development environmentProcess (computing)WindowComputer programmingEndliche ModelltheorieBootstrap aggregatingComputer animation
01:40
Focus (optics)Distribution (mathematics)MiniDiscSpacetimeThermische ZustandsgleichungDifferent (Kate Ryan album)Distribution (mathematics)Hard disk driveFlow separationCartesian coordinate systemBitThermal conductivityLecture/ConferenceComputer animation
02:09
Server (computing)Default (computer science)Thermische ZustandsgleichungTask (computing)Integrated development environmentWebsitePoint cloudMultiplication signSpacetimeComputer programmingMiniDiscProgrammer (hardware)BitRight angleDefault (computer science)Repository (publishing)Virtual machineComputer configurationRevision controlInfinityOperator (mathematics)Disk read-and-write headOpen sourcePlanningPoint (geometry)Service (economics)Configuration spaceCoefficient of determinationSuite (music)Library (computing)Integrated development environmentDomain nameThermal conductivityElectronic visual displayVideo gameMoment (mathematics)Computer fileDot productServer (computing)InternetworkingType theoryTask (computing)Computer animation
04:33
Default (computer science)WebsitePoint cloudArmRevision controlComputing platformIntegrated development environmentMetropolitan area networkComplete metric spaceNumberLibrary (computing)Focus (optics)Revision controlWindowAuditory maskingComputer configurationStatisticsDot productField (computer science)BitCartesian coordinate systemElectronic mailing listExtension (kinesiology)Default (computer science)Web pageMultiplication signVideo gameGenderContent (media)Process (computing)Computer programmingForcing (mathematics)outputService (economics)Standard deviationComputer animation
07:27
BuildingPointer (computer programming)Mountain passIntegrated development environmentPoint cloudEmulationThermische ZustandsgleichungWindowIntegrated development environmentBinary codeGoodness of fitRootRevision controlThermal conductivityComputer animation
08:06
Point cloudIntegrated development environmentPointer (computer programming)Mountain passBuildingPublic key certificateComa BerenicesAreaCache (computing)Complete metric spaceThermische ZustandsgleichungMenu (computing)TensorRootElectronic mailing listCone penetration testMaxima and minimaFrequencyIntegrated development environmentElectronic mailing listInstance (computer science)RootSelectivity (electronic)Right angleInformationRevision controlDirectory serviceThermal conductivityValidity (statistics)Game theoryCovering spaceParameter (computer programming)NumberAssociative propertyLecture/ConferenceComputer animation
09:40
Integrated development environmentElectronic mailing listPublic key certificateBuildingSkeleton (computer programming)Computer fileRevision controlComputing platformPoint cloudMeta elementLibrary (computing)Metropolitan area networkLevel (video gaming)Error messageRevision controlDifferent (Kate Ryan album)Default (computer science)Computer fileProcess (computing)AliasingNormal (geometry)Computing platformUniform resource locatorMeta elementLibrary (computing)InformationBuildingGastropod shellExtension (kinesiology)StapeldateiWindowDirectory serviceMereologySkeleton (computer programming)Sheaf (mathematics)Integrated development environmentFunction (mathematics)Link (knot theory)ParsingElectronic mailing listBitType theoryField (computer science)DampingDirection (geometry)MathematicsClient (computing)Operating systemPredictabilityThermal conductivityCompilerSet (mathematics)Instance (computer science)Category of beingSoftware testingNumberPrice indexOperator (mathematics)Core dumpPlanningProgrammable read-only memoryAreaOpen sourceAnalytic continuationPhysical systemRhombusComputer iconQuicksortData storage deviceCASE <Informatik>GradientRule of inference1 (number)WebsiteIncidence algebraComputer configurationLine (geometry)Computer animation
17:31
BuildingSkeleton (computer programming)Integrated development environmentLibrary (computing)Computer fileIntegrated development environmentGroup actionOperator (mathematics)Skeleton (computer programming)Virtual machineComputer programmingType theoryGoodness of fitVideo gameData managementRadiusField (computer science)BitData dictionaryLecture/ConferenceComputer animation
19:00
Cross-platformDifferent (Kate Ryan album)Distribution (mathematics)Software maintenanceSelectivity (electronic)Group actionLecture/ConferenceMeeting/Interview
20:02
Meta elementScripting languageEvent horizonComputer fontField (computer science)Degree (graph theory)NumberStandard deviationLecture/Conference
20:53
Student's t-testScripting languageMeta elementBuildingPoint cloudSymmetry (physics)Arithmetic meanRevision controlLecture/Conference
21:33
Interface (computing)Point (geometry)Formal languageCategory of beingLibrary (computing)Particle systemLevel (video gaming)Arithmetic meanTable (information)
22:13
Revision controlVideo gameMereologyLecture/Conference
22:43
Multiplication signPoint (geometry)Goodness of fitDivisorData managementSingle-precision floating-point formatIntegrated development environmentVariable (mathematics)Binary codeVirtualization
23:19
Traverse (surveying)MereologyThermal conductivityCellular automatonComputing platformDerivation (linguistics)Figurate numberDegree (graph theory)Revision controlRight angleBitFile formatWordObservational studyOpen setLecture/Conference
24:43
AuthorizationStandard deviation
25:17
CodeCASE <Informatik>Forcing (mathematics)Open sourceBitLattice (order)Lecture/Conference
25:56
Physical systemRevision controlOperating systemProcess (computing)Operator (mathematics)
26:42
Integrated development environmentComputer filePhysical systemError messageScripting languageRevision controlElectronic mailing listRootLevel (video gaming)Online helpRule of inferenceStudent's t-testDirectory serviceIdentity managementCASE <Informatik>Lecture/Conference
27:59
Integrated development environmentData managementVirtualizationPhysical systemLecture/Conference
28:30
Electronic mailing listIntegrated development environmentRevision controlSet (mathematics)NeuroinformatikLecture/Conference
29:08
WebsiteCone penetration testDefault (computer science)Execution unitIntegrated development environmentCache (computing)Complete metric spaceElectronic mailing listMetropolitan area networkPublic key certificateReading (process)Integrated development environmentRevision controlComputer fileOperating systemDirectory serviceDifferent (Kate Ryan album)Computer configurationRepository (publishing)Standard deviationMiniDiscPersonal digital assistantMathematical analysisoutputFreezingComputer animation
30:04
WindowSymbol tableRevision controlText editorComputer fileService (economics)Operating systemLecture/Conference
30:38
Library (computing)Repository (publishing)
31:15
Library (computing)Inversion (music)Crash (computing)Integrated development environmentDifferent (Kate Ryan album)Revision controlQuicksortLecture/Conference
32:05
Control flowRight angle
Transcript: English(auto-generated)
00:01
Okay everyone, give Mike Muller a big hand, he's going to talk to us about Conda. Thank you very much for the great introduction. I would like to talk about Conda, easy installs and simple builds. Just a quick raise of hands, who of you knows what Conda is?
00:24
A third. And who is actually using Conda? Just about 20%. Thank you very much. Conda is quite a few things actually. It's an installer similar to PIP but I think a bit better.
00:42
It's an environment manager similar to virtualenv but still a bit better. It's a cross platform that works on Windows, Linux and Mac. It's not limited to Python, so you can use it for different languages. Actually there are people using it for Ruby and for other languages. Actually it comes from the scientific community but I think it can be useful for every Python programmer.
01:05
So it's BSE licensed and typically you install, you either install Miniconda or Anaconda. I will talk about these different Condas here. So there's what's called Miniconda and it's a kind of a small bootstrap like version. It includes Python itself, Conda, and a bunch of dependencies.
01:25
Like PIP and Wheel and some other tools. And it provides access to many hundred libraries. You can install a lot of libraries with Conda, maybe even thousands. And you just need to say Conda install. I will talk about this process a little bit.
01:41
And there's something called Anaconda which is a different thing. It's a distribution similar to like a Linux distribution. So you get Python packaged up with a bunch of scientific applications. And this includes of course Python, Conda, Conda install, and it's about more or less like 200 scientific packages. It could come off here.
02:01
So it's supposed to be there. But scientific packages are about 200 or so. And if you install it, it takes about two gigs of your hard drive. But then I think a lot of scientists need a lot of these tools and you don't have to fiddle around installing them all separately. They all come in one thing and then there you pay a little bit of price with some disk space. But disk space is cheap and programmer's time is expensive.
02:22
And there's something called channels. So when you install something, Conda looks for a channel. And channel is a place somewhere, somewhere on the internet. Or maybe even on your own machine. It's also possible that it looks for packages. And the default is an Anaconda server so we don't see anything that typically looks there.
02:41
But there's also something called Conda Forge which is a channel with a lot of open source packages by the Python community. But you can also have private channels. So you can have a channel with your own name, company name, whatever. And then you can install from this channel by saying install-c my-channel. My-channel is the name of the channel.
03:02
And then it will grab the package from this channel. There's also a lot of configuration options. Which channel has precedence and which channel looks first. You can configure this in a configuration file. Okay, a few basic tasks. I want to introduce you to what you can do with it. It's by no means a comprehensive introduction but gives you some feeling what you can do.
03:22
So you can install packages. You can create and administer environments. And you also can create packages yourself. So writing a setup.py as you know. And doing a lot of steps to create a Conda package. The first thing. You can search for something.
03:42
So you say Conda search and you type in the name of the package you're looking for. I use pandas. Pandas is very much used package in scientific library. And then Conda goes out there and is searching places. So I added a few channels. You see every dot, you see they're fetching packages, dot dot dot dot. And every dot means looking for a channel.
04:02
So I have a bunch of channels it's looking at. And then it gives you a long display. I just cut out a few things and you see the three dots there. It's cut out. And it lists you the package. So everything, this search looks like everything has pandas in its name. And shows you the version that's available. So all these versions are available. One dot.
04:21
A dot here means it is available. And it is downloaded. So when you install something it puts it in a local repository. And you install from there. You don't have to download it every time. And the one with the star here with the asterisk. That's the one I have installed right now here. And you see if it has a dependency on NumPy. Because the scientific focus tells you I have NumPy 1.10.
04:44
And Python 3.5. And build 0. And it comes from default which is from Anaconda. But there's also a version that comes from Conda 4. So it gives you the channel here that it comes from. So this is pretty interesting. It can be a very long list. But it shows you what you have. So it's usually a bit longer here. And by the cut out quite a few.
05:00
It can be several pages long easily. Okay. The search can be a bit more fancy. So that's a problem I have with Pitman. You search for something. If you search for jungle it just gives you. I don't know how much. Very, very, very long listing. Everything has jungle inside. Here you can say for the full name. And now it displays only the package that's called pandas.
05:21
Not all the packages have pandas in their name. And now I get only pandas. It's still a pretty long list. But it's much shorter already. And I know that it's pandas. And that's a good feature. And there's more. So you can also limit the search. You'll say I want to have only. Even though I did it on my Mac. I can search for all the available packages for Windows.
05:41
Win32. And I also can specify the version I want. Now it shows me all the available packages for version 0.18.1. And it tells me okay. Here now it offers everything starting. This is numpy 110. Python 2.7. Up to numpy 111.
06:00
Python 3.5. So there's no dots and no asterisk. Because obviously if I'm on Mac. There's no need to download it here. So that's not there. But it would be available. It would download it. And they all come from default. And there's more options if you want. There's quite a few switches you can use. And can make the search even more specific. Pretty interesting feature.
06:22
Installing a package is very easy. You just say conda install. And you name the package. And it goes through your channels. And you can have a contact. Find the channel that comes through. So it takes you on this channel. And see now it provides me with a name. I want to install this package. And typically it tells I'm going to download. Here it doesn't say because I've downloaded already.
06:41
Typically it's okay if I need to download these packages. And then I install these ones. Like MKL, mask on the library. All the dependencies. Typically if I have a properly packaged application. And all the dependencies are there. Installed all the dependencies. And everything is binary. So everything. Especially a scientific field.
07:01
There's a lot of C and Fortran extensions. And compiling these things. Typically works on Linux. It might work on OSX. And it's very likely it doesn't work on Windows. So that's why it's. And still I think most people still Windows users. So the most users of Python I think are on Windows. According to all the statistics I've heard about. And a lot of scientists.
07:21
And a lot of engineers. So I teach a lot of engineers. And most of the engineering companies are mostly actually using Windows. Some of them have used Linux. But by far the biggest majority are using Windows. So you need to support Windows users. And it's a pretty good way. Everything is binary. And you can use this. Okay. Create an environment. So this is the first thing installing something.
07:42
But very often you don't want to install in what's called the root environment. You want to create an environment. And that's what this command up here. You can see conda create dash n or dash dash name. So I create a new environment which I call mypy335. And I can specify the Python version. And I say I want to have Python 3.5.
08:02
And you can also have a, if you wanted to, could have a 3.4 or 2.7 or even 2.6. So far it supports 2.6, 2.7. I'm not sure if it's 3.3, but 3.4 or 3.5. And we are pretty quick. As soon as 3.5 is out, it won't take long. 3.6, sorry, 3.6 will be supported.
08:21
Typically a new NumPy version is supported within 24 hours or something. Pretty fast. And it tells me what it's going to do. It never does something without asking. And it tells me, okay, I need to download these packages. And I need to install these packages or the dependencies. And then if you say yes, which I did, then it starts creating this package.
08:41
And it gives you this cutoff here. But it shows you how long it takes and how fast it's doing stuff. Okay, now I created an environment to actually activate. Now I can actually list environments. I create environments. I don't have any one.
09:00
I have just a small selection of the environments I have. You say conda-env-list, for instance. It's one of the commands that shows you what environments you have. And you see an environment is nothing but a directory in your home folder, in my user folder under conda-env. So if you install miniconda, it will be miniconda-env. And here's anaconda-env. It's just a different directory.
09:21
And these directories contain all the information about the environment. And you see now the root environment is the one that's active right now. Okay, you see I gave a tutorial at PyData Berlin here. Then I made a new tutorial. I made a new environment. And I don't mess up with my root.
09:41
I can install whatever I like, different versions. And everything is isolated. Okay, activate an environment. So depending if you are on a Unix kind of system, Linux OS X, then you have to say source activate name of my environment. On Windows it's just activate. So typing source activate is pretty long. So I just had to find a short alias.
10:02
But that's typically how you activate an environment and just type deactivated deactivated again. And then the prompt changes and then you see the prompt. And now when you list it again, you see now the environment will be activated. You always can see which one is active. Okay, and then you can say list.
10:22
And then it gives you a list of all the packages you have installed. You see it tells me again the name of the package, the version. Here there's dependencies, NumPy or not. The channel. And this one is red because this is when you see it's installed this pip. So you will find maybe a few hundred packages there.
10:42
You can install this conda directly. If not, you can always install this pip. And conda will recognize this one. So it's not a conda dictionary to work either with conda or pip, you can use both. And that's what I typically do. If it's not available with conda, I use pip and install with pip. But conda sees these packages and that's why it says install this pip.
11:01
So I have the version to install this pip. And you can also install fields, there's no problem. So there's no contradiction in this. And you have this pip installed there also. Okay. This is, if you're kind of a consumer of packages, you can install it in environments. There are way more options. I'll just show a few.
11:21
But now the next thing is building a package. So you want to build a package. And there are essentially two ways you can build a package. You can just take a package from PyPI and convert it into a conda package with a few commands. Or you can start from scratch doing it with your own package. Okay.
11:40
Building from PyPI is pretty easy. There's a skeleton. First you need to install conda build. So conda build, if you have anaconda, it's there already. If you have miniconda, you need to install conda build. Which is a command line tool that helps you to build the package. And then you say conda skeleton pypi, my package. So this one goes to PyPI, fetches the package you have there.
12:04
And then you say conda build, my package, and that's pretty much it. So it's not very difficult. And it builds a package. And the package is a tar ball. It's a tar bc2 in this case. It's just one file with everything inside. And that's it. And now we can install it.
12:20
And, by default, it creates it for the platform you're on. So if I do this, as you can see here, now it's for osx. And you can now install it. And there are actually two different ways installing it right now. I can say conda install, use local my package. And it just will go to the place because it knows the default place where I put it and installs it.
12:41
But you can also, if you wanted to, if you had somewhere else, you could just specify the full path. It wouldn't be a problem. Just the file name. This is exactly the same here, but it could be a different way. If you download one of these, you can always install it as conda install from the command line. You can also specify a python version. So even if you have a different python, you can say, okay, build this one for python 3.4 and it builds it for 3.4.
13:06
So even if it's pure python, it always has a python version inside. And this is especially useful if you build extensions in C or Fortran, which is very common in scientific field. And this one builds everything for this version.
13:22
You can also convert to other platforms. You see, I can say conda convert, platform all, for instance, and just specify my file and say output directory. And if it's a plain python version, it will create a version for each platform for Windows, Linux, and Mac.
13:41
If it's with extensions, it's not that easy, then you actually need the operating system because you need to compile it. So then, I don't think there's a cross compile. At least I didn't try to know. So far, I compiled in Windows and on Mac, and I think in Linux it's the same. Typically, Linux is easiest to compile anyway. So if you compile extensions, it works. And I even compiled Fortran extensions, which is typically a bit more involved than C extension, and it works nicely so far.
14:05
So you can create these extensions also. And then, if you like to, you can upload. And you need the Anaconda client, and I uploaded to Anaconda. And you can also make your own channel if you like. So I don't have it here, but you can also create a channel.
14:22
And you just say Anaconda upload, and just give the full parse of this file, and it uploads this link to Anaconda. And other people can install it just by saying conda install, name of your package. And if you put it in different channels, they have to specify the channel when they install it.
14:43
Okay. This is building from a PyPI package. You can also do it from scratch, so if you want to, you can customize things. And you need a few things. You need a meta.yaml, so a yaml file that declaratively specifies what you want to do. And then you need a build file, and there are two different ones.
15:03
If you're in Windows, you have a build.bat file, a batch file, and in Linux and Mac you have a shell file. And you just use a normal setup.py. So you're probably familiar with it, so you just use a normal setup.py. And then you have additional information in the meta.yaml which also has dependencies in it.
15:22
Let's have a look at this file. So the meta.yaml looks like this. It's just a yaml file and has a lot of different sections in there. You don't need all of them. So you have something about meta.yaml about the package. Then you have the sources, so you can say it comes from some git URL, something like this.
15:41
You can have requirements, and there are different requirements. You have requirements for building things and requirements for running. For instance, if you use Syson, you might need Syson to build it, but you might need to install it on the client machine because you just have the extensions. You have different dependencies for the tests, what you do with the tests, and then you have normal information about who the license is and all that kind of stuff.
16:04
Okay, this is just a short part of a meta.yaml I used. So I used just my package name here. And you say, okay, the sources just depends where they are. Tool directories up, so it can be a relative path. For instance, it doesn't have to be a URL.
16:23
Requirements of Python and setup tools, and then for running, I need Jupyter. And see here, this is something you can specify. This one, I need it only for Windows. And the same way you can specify for Linux and Mac, so this will be only installed for the Windows version, but not for the version for other operating systems as well.
16:43
Very important, and you can use the same syntax at other places, so you make it dependent on the platform with one file. Of course, then you have this MIT license, whatever you want to put in there. Then you have these build files, and they're typically, if there's a skeleton, you set it up and you create files that look like this.
17:02
And you see, you just call Python and setup.py install, and they just call your setup.py. The setup.py is just a normal one, as you know from the normal installation process. But you can also add other commands in here, which is very interesting. So if something is not available as a contact package,
17:21
in here you can do a pip install something, and this will install the package, and this will go into this tarball. So everything you pip install, you will deliver with your own package. I had this problem, I had the library that wasn't available as a conda, and it's difficult to install it, and you just distribute it alongside with your package. There's quite a few things you can do here in this build file.
17:41
You can customize things, so something doesn't work. Typically, you have a way to do it in the build file, some way or the other. Okay, and then you just build it. You say, conda build my package, and that's it. And all the other steps are exactly the same as with the skeleton. So you can do upload and all kind of stuff, install from the local machine,
18:03
from these kind of operations. Good. That's it already, so conclusions. I think conda is a very interesting package, and it's very much used in scientific field, but can be really useful for all Python programmers, because it's a pretty good installer, and a very nice package manager.
18:21
You can manage environments, and you can also build conda packages. It works together with pip, so it's not a conda dictionary. You can still use pip if conda doesn't supply the package, but as you can see, you can always convert a pip installer package into a conda package if you like. That might make the installation more smoother for your users.
18:40
Then you might actually just convert the pip package to a conda package. Very well known in the scientific community. If you go to a sci-fi, your sci-fi conference, pretty much everybody will know a bit about it, I think. But it can be really useful for all Python programmers, and I encourage you to give it a try.
19:02
Thank you very much, and I'm here for questions. I've got the same questions as I have in my name. And ask them for me, right? First hand went up here.
19:21
Thanks. I understand I can use conda for distribution and platform independent package installation, which is great. Now, if I have, I am a maintainer of a tool, and I have lots of different dependencies, outside PyPI and inside PyPI and with NPM and whatever,
19:41
and it would be nice to be able to just use conda install whatever for me, would I have to package all those NPM, PyPI packages to conda manually, or is it possible to, I don't know, say conda just install this and install it from pip because you don't have the package yet?
20:03
Depends. There are some things you can do. So if you install a package that is conda, then this package will install its dependencies. So if I say I depend on pandas, pandas has a bunch of dependencies, it will install numpy for me and so on. To some degree, you can do pip. So far, I think you have to write it in this build script.
20:25
You have to say pip install, pip install, pip install, but probably this meta YAML will be improved. That's why I just saw a ticket that you can then even say, okay, this is supposed to come from conda, this is supposed to be pip, and this might be a different channel. So it's still evolving, and not everything works perfectly.
20:43
There are some problems. There are no problems, those are those problems. But so far, you can always have a workaround by putting it in the build script, I think. So what would be your suggestion for this kind of situation? I'd put it in the build script. I had the same dependency thing, and I put a pip install in the build script,
21:02
in the build SH script, and it worked for me, so that was the easiest one. The other thing is just you can always convert the pip packages into conda packages, and then you can list them in the meta YAML and just upload them. As long as they are BSD, MIT, that's not showing me a problem in anything. Just upload them to conda crowd, and that's it.
21:22
You can automate it, you just need to list all the packages, and then you can update it. If it's a new version, just check if it's a new version, and just build new packages. Okay, thanks.
21:41
Thanks for the talk. So one of the selling points at the beginning is that not only to Python, but all the examples have seen all the Python packages. So what do you mean by not limited to Python? So you can use this, everything is declarative, so people use it for Ruby and other languages, so you can use the same thing,
22:01
and you can also have dependencies that are not necessarily Python. For instance, one package is called PyTables, it's an interface to hdf5, and this depends not only on NumPy, but also in the hdf5 library, there has to be a different version, and all the things can be packaged in there, so you can have DLLs, shared libraries, you can include them in your thing,
22:22
and that makes it much easier. If you have a package at the end, then the user doesn't have to do anything about fiddling around with include parts and all this kind of stuff, I suppose. Compiling hdf5 is not the simplest thing, so I tried this once before Conda, and you can easily take two days out of your life
22:40
and you still haven't gotten anywhere. Alright, thanks. Hello, thanks for your talk. Very to the point. Given that recently we have been adopting on PyPI binary wheels, they seem to solve most of the issues
23:02
that Conda seems to solve, other than the fact that you have a single command to manage your installed packages in virtual environments, and that's good, it's nice, but what's the big advantage now that binary wheels are, well, taking off? You ride wheels, it's a great step forward, I would say.
23:22
Not everything is in the wheel format yet, and you might not have the right version and the right platform in the wheel, and Conda can install wheels, no problem. So you say pip install and install the wheel, so that's not a contradiction this way. As far as I know, the story goes like this. Travis Olufand, he's the CEO of Continium,
23:42
which is behind this. Everything's open source, but there's a company behind it, and he asked Guido van Rossum a while, a few years ago, about this packaging, and Guido said, just go ahead and do it, and that's what I did. So it's kind of a little bit parallel, to some degree a little bit parallel to pip and wheels, and something like this, but I think it's more comprehensive,
24:03
and it's better, and it has a lot of nicer features than pip and wheels. For instance, the search for a name. I don't know, maybe I'm not smart enough, but I couldn't get pip into just showing me this package. It shows me everything. It shows me hundreds of packages for Django, and this is not very useful. They have to run it through grab and do your own thing. Why not just say I want to just customize the search,
24:24
and those things aren't the conda. The conda is nice, but as I said, you can use everything that pip and wheels and pip your eye, so you don't exclude it. You can still use it, and I do it very often, because it has only 200 packages, or maybe now 2,000 out there, but not 80,000 or 90,000, whatever.
24:41
It's on pip your eye. All right, any more interesting questions for Mike? Here we go, yes. Best question yet. Okay, so is there any standardization effort? Are you working with Python packaging authority or something,
25:04
or is it going anywhere, or will it be forever separate? Conda is great supposedly, but only in the scientific community and everyone else. This is bad. Great question, but I don't know the answer. It would be very nice.
25:21
I would like to see it this way, but actually everything is open source, and both communities could use the code from the other. Easily, no problem. I think everything is BSD or MIT license, so there shouldn't be a big problem to reuse code. I'm not really sure why this is not a bit more coordinated effort. I don't know.
25:40
It would be very nice to have one stop and join forces and do it together for these things, and I don't know why it's the case. There might be some reasons, and I haven't really found out yet why. More questions, more questions. Keep the dream alive, everyone. I have a question.
26:01
It's like a two-parter, so presumably if I install Anaconda, it's going to come with its own versions of Python as well as my system pythons that I might have installed separately, and I once had this problem where, or I don't know if anyone has seen the problem where you upgrade your operating system, and you upgrade your system Python, and all your virtual ends are now broken because something, something, something,
26:21
which I don't really understand, and you have to delete all your old virtual ends and recreate them. Is Anaconda going to magically fix that problem for me, or sub-question, if I upgrade, you know, if there's a Python 3.5.1 and a Python 3.5.2, how does Anaconda manage the upgrade process there? Or does it need to rebuild a bunch of things?
26:42
There should be a problem, so if you work with environments, those environments are pretty much isolated. They're not totally isolated. Sometimes if you use command-line tools, and they're installed like in the scripts directory, from there, if not, it goes outside. So if you say, pylint something, and you haven't installed pylint in your environment, it might take the wrong version and gives you a lot of syntax errors
27:01
because it's a pylint for two, seven, and you are in the 3.5 environment. But if you install everything, it puts everything in the directory, and there shouldn't be a problem when you update if you work with environments, and you will see if you work for a while, you will work with environments anyway all the time. Maybe the best approach is have the root just as a helper,
27:21
and then work in environments or isolated. Of course, you can always reproduce your environment, you can freeze the environment, just specify what you have installed, you get a big list of installations, and then you can give this file to somebody else, and the other person can say, okay, install from this file, and install exactly the same versions as they want. You can reproduce your environments,
27:41
and that should solve the problem. I don't know, there could be some, you can only find it out if you test it, you can do it, but theoretically, there shouldn't be a problem, practically, there might be one, I don't know. Is that only me that's ever had that problem of changing the system Python and all the virtual levels are broken? Alright.
28:00
So, I guess we're thinking that Anaconda is kind of even more isolated from system Pythons than virtual end managers. If it never has that problem. There's some voodoo there, further research required. Any more questions? Yes, yes, fantastic, alright. Nobody wants to hear my voice any longer than absolutely necessary.
28:23
Are the environments relocatable, or is there any plan for that? Can you repeat the question? Are the environments relocatable? So, I installed it somewhere, let's say I archived the environment, and
28:41
uncompressed it somewhere else, like another computer? Yeah, I just said you don't actually copy the environment, you just copy all the packages that are installed as a version. I showed you these listings here, so one of these earlier things shows you this, whatever is installed,
29:01
you have this list somewhere, do I have a list? Yeah, something like this. So you can actually freeze, no, that's a search, you can tell, okay, export all the versions I have in a file, you just get a file with all the names, equal the version, and then you can give
29:21
this file to somebody else, and somebody else can recreate exactly the same environment, exactly the same versions. Yeah, but I'm not speaking about recreating the environment, but just taking the environment and put it somewhere else. It should work, it's just a directory. The problem is not everything is copied. If I have this pandas version,
29:42
I use it in ten different environments, but it's only once in my repository, and not directly in the file, it's just a symlink. And you have to make sure that you copy it along this list, but I think there should be an option to copy everything, just zip the whole directory and put it somewhere else. It should work as long as the same operating
30:01
system. No guarantee, but theoretically it should work. You can try it out, but maybe recreating is the safer thing. If you recreate, you can also say, okay, it freezes everything with exactly the same version, but you can also go in the file and edit and say, okay, it shouldn't be exactly the same version, it should be at least the version, or you can update the version, so you can go in the file
30:21
and customise it, and then still everybody who has this file would create. And the nice thing is with Brock, I can freeze it on my Mac, and somebody else can recreate the same, and while I'm in the windows there shouldn't be a problem. If I zip the file, it would only be for the same operating system. Alright, any more for any more? It is now, oh, there
30:42
is one. Okay, you have to make it a question. You will talk very, very fast, and Mike will talk very, very fast. Okay, so let's say that I have a package, Python package on Linux that has, that needs separate installed C library, and let's say that this C library expects another C library.
31:02
So what will Conda do? It will download this library and all of its dependencies and put it in the Conda repository locally, or what will happen? There are different ways. I had the same problem, typically what I did, I just included it in the file, so I had this, you can specify the dependencies,
31:22
so if you compiled something, I had Fortran dependencies, some shared libraries somewhere, and you included, so I'm not really, myself, not really sure how to do it, the best way to do it, but there are different ways including shared libraries in there, and you can distribute the shared libraries along, and they will put the shared libraries in this environment, and they wouldn't crash
31:42
with your shared libraries, because you might have the shared libraries, maybe different versions, then it works for a while until it doesn't work anymore, because it's a different version, something is different. So you can distribute shared libraries with it, there are different ways of doing it. You still have to look at, there are a lot of recipes out there how to do it,
32:00
and probably you will find one that serves your purpose. Okay, let's give Mike another big hand, thank you very much for a great talk and enjoy the break.