Machine Learning for dummies with Python
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 3 | |
| Number of Parts | 169 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/21161 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 20163 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
Multiplication signMereologyVirtual machineArithmetic meanVideo gameSoftware developerOffice suiteDigital photographySpring (hydrology)Film editingGoodness of fitLecture/Conference
01:13
Ext functorVideo gameRoutingMultiplication signFirmwareRevision controlInheritance (object-oriented programming)Generic programmingView (database)Uniformer RaumOffice suiteComputer animation
01:50
DistanceMusical ensembleCasting (performing arts)Lecture/Conference
02:18
Video gameMachine learningPresentation of a groupVirtual machineCategory of beingPresentation of a groupMusical ensembleBitDigital photographyFlickrComputer architectureMachine learningArea
03:31
Presentation of a groupMachine learningArithmetic meanVirtual machineObservational studyExpert systemElement (mathematics)Lecture/ConferenceComputer animation
04:00
Sample (statistics)Presentation of a groupMachine learningVirtual machineVirtual machineOpen setLibrary (computing)Video gameOpen sourceSource codeTheorySampling (statistics)Machine codeLecture/ConferenceJSON
04:25
Machine learningMachine learningDependent and independent variablesPattern recognitionAlgorithmGoodness of fitSummierbarkeitMedical imagingVirtual machineComputer animation
05:10
Game theorySimilarity (geometry)Medical imagingAlgorithmComputer chessGame theoryNumberMultiplication signSoftware testingAtomic numberDeep Blue (chess computer)Universe (mathematics)Software developerLecture/ConferenceComputer animation
06:09
Inclusion mapComplex (psychology)Spring (hydrology)AlgorithmGoogolLecture/Conference
06:35
Game theoryAlpha (investment)GoogolVirtual machineMathematical singularityPredictabilityMoment (mathematics)Multiplication signRhombusBitObservational studyComputer animation
07:15
Row (database)Machine learningContent (media)E-learningOpen setMassMachine learningVideoconferencingPattern recognitionInternet forumInternet service providerVirtual machineArithmetic meanInteractive televisionLine (geometry)Scalar field1 (number)MathematicsUniverse (mathematics)Lecture/Conference
08:46
Reading (process)Electronic mailing listMultiplication signMachine learningVirtual machineLecture/Conference
09:14
Computer programmingClassical physicsNatural numberoutputPredictabilityMachine learningComputer animation
09:44
Computer programmingComplex numberMedical imagingComplex (psychology)Pattern recognitionInsertion lossImage resolutionCASE <Informatik>Revision controlRule of inferenceScaling (geometry)Lecture/ConferenceComputer animation
10:20
Machine learningComputerAlgorithmWave packetComputerNeuroinformatikVirtual machineMultiplication signLevel (video gaming)MathematicsPoint (geometry)View (database)Execution unitPrice indexSpacetimeLecture/ConferenceComputer animation
11:10
Sample (statistics)AlgorithmView (database)Point (geometry)Level (video gaming)Virtual machineOptical character recognitionAlgorithmOrder (biology)MeasurementCorrespondence (mathematics)10 (number)Machine learningMedical imagingLecture/ConferenceComputer animation
12:09
AlgorithmWave packetPredictionNetwork topologyDecision theoryVector spaceInheritance (object-oriented programming)Wave packetMultiplication signMedical imagingWater vaporCombinational logicEvent horizonProjective planeAlgorithmTheoryModemProcess (computing)MereologyMultilaterationVirtual machineFitness functionConfiguration spaceLecture/ConferenceComputer animation
13:28
Open sourceDifferent (Kate Ryan album)AlgorithmProcess (computing)PredictionWave packetData modelClefMaizeMetropolitan area networkProgrammer (hardware)State of matterDataflowTensorGraph (mathematics)MathematicsVertex (graph theory)Operator (mathematics)Array data structureArtificial neural networkDifferent (Kate Ryan album)Video gameEndliche ModelltheorieFormal languageWordPredictabilityDataflowAlgorithmOperator (mathematics)Sinc functionRevision controlSet (mathematics)Functional (mathematics)Maxima and minimaWeightVector potentialImplementationFunction (mathematics)MultiplicationArtificial neural networkFitness functionLocal ringOpen sourceLinear regressionLogistic distributionResultantWave packetMetric systemSemantics (computer science)AdditionConfiguration spaceMeasurementSelectivity (electronic)Software testingRandomizationMoment (mathematics)State of matterWater vaporMedical imagingScalar fieldRight angleMatching (graph theory)Level (video gaming)Speech synthesisCycle (graph theory)Event horizonPattern recognitionClassical physicsDemosceneSystem callMereologyLibrary (computing)Coordinate systemMachine codeLine (geometry)Multiplication signVirtual machineMatrix (mathematics)Pattern languageGoodness of fitProduct (business)Graph (mathematics)Machine learningHydraulic jumpAuthorizationImperative programmingOcean currentHeegaard splittingOptical character recognitionImage processingComputer animation
23:08
Virtual machineFreewareGoodness of fitOpen sourceComputer animation
23:32
DemonCategory of beingLoop (music)Core dumpNatural languagePrototypeMultiplication signDifferent (Kate Ryan album)Physical systemDependent and independent variablesLevel (video gaming)Endliche ModelltheorieComputer hardwareCASE <Informatik>Library (computing)Process (computing)SoftwareFunctional (mathematics)Virtual machineArtificial neural networkAlgorithmComputer graphics (computer science)Multiple Regression4 (number)FrequencyRevision controlFormal languageActive contour modelRight angleMoment (mathematics)Vector spaceSampling (statistics)Graph coloringMereologyType theoryComputer animationMeeting/Interview
Transcript: English(auto-generated)
00:01
Good afternoon, everyone. It's my pleasure to introduce Javier Arias, who's a senior developer at Telefónica in Barcelona, here in Spain, and he's gonna be talking about machine learning for the means using Python, of course.
00:23
Do you hear me well during the back? Yeah, it's okay? Okay, if not, please let me know, because if I lower my voice, let me take part of the contest, of the photos, because, police, also for my mama.
00:56
Okay, this could be a day in your life.
01:10
This could be today. It's about time to leave the office, and your phone tells you the best route to go home. It's fun, because you never told your phone
01:21
where home is, or what's the appropriate time to leave the office. In fact, on Fridays, the phone will tell you the best route to your parents' home, because on Fridays, you visit your parents. So, you go to a parking, and you happen to be a happy Tesla owner.
01:42
Not that everybody of us can beat, but let's imagine for a minute. In the latest version of the firmware's car, there is an autopilot feature. With autopilot, the car is able to keep the lane by moving the steering wheel, keep the speed,
02:00
and the distance with the rest of the cars. And not only that, the car is able to learn, not only from its own experiences, but from experiences from the Teslas from around the world. So, you get home, and you want to play some music, but you don't feel like choosing any music,
02:22
so you trust on Spotify to play some music for you that you don't know, but you actually like. While listening to the music, you check your photos, which are very well organized in categories like here,
02:40
with architecture and arts, but you never tag them. It was Flickr that is able to look inside your photos and see what's inside, and tag them accordingly. These things are happening daily to thousands, or even millions of people from around the world, and all of them have something in common.
03:02
Do you know what it is? It's machine learning. Machine learning is already in your life. It's everyone around us, and it will be very important in the next years. So, what's this presentation about?
03:20
I'll try to explain a little bit why machine learning is so important for us as users, as engineers, and for companies, and for the rest of the world. Also, I'll try to explain about my journey with machine learning. My name is Javier Diaz, and I am a backend engineer.
03:43
Six months ago, I didn't know anything about machine learning, and I'm not an expert. It would need years of study and practice to do so, but you can get started pretty quick,
04:00
and do very interesting and funny things, because there are many technologies and many resources around that are free, that are open source, and of course, using Python and libraries with Python. And also, I'll try to explain some very basic machine learning concepts,
04:20
and we'll see a couple of code samples. So, as many of you know, machine learning will be very important during the next years, but one of the first questions that came to my mind when starting with it is, is really machine learning intelligent?
04:40
Are these algorithms already better than us? And the response is that in some concrete questions, in some concrete aspects, machines and algorithms are already better than us. One example of that is image recognition. ImageNet is hosting a yearly challenge
05:00
on machine learning, on image recognition, and the performance of the winner algorithms have been improving a lot during the last years, thanks to the adoption of deep learning and this kind of technologies. In 2015, the winner algorithm performed better than human at recognizing things inside images.
05:23
Something similar happened to chess about 20 years ago, when IBM's Deep Blue defeated the world champion, Garry Kasparov. I remember vividly reading about this in the news, and reading it in the newspaper.
05:40
It was 20 years ago, maybe. That's why some colleagues tell me that I'm a senior developer. But there are, it was quite a milestone at the time, but there are other games that are much more complex than chess. For example, the game of Go. The game of Go is so complex
06:02
that the number of movements is bigger than the number of atoms in the universe. For such a complex game, it was thought, for such a complex game, it's very important to play with intuition. And for that reason, many people thought that algorithms could never win the champions,
06:23
until this spring, when AlphaGo by Google defeated the world champion, Lucidor. This thing, algorithms being better than us, and concrete tasks, are happening more and more frequently. And for that reason, many people are making
06:42
apocalyptic predictions on the moment that machines will be more intelligent than us. It's the singularity point. But, okay, I don't want to be so apocalyptic. I want to talk to you about my journey. As I said, six months ago, I didn't know anything about machine learning,
07:01
and I thought that it would take weeks or months of study just to get started. But that's not true. If you are a dummy like me, you don't know anything about machine learning, you can get started, and in a little bit of time, you can get very useful and very surprising things, at least for me.
07:22
When I started, I decided to study, not by using courses or books, but by using massive open online courses. And these courses have videos and exercises, and there are different providers. There are forums where you can discuss with other people
07:40
about the things you are doing. And there are many different providers, so you have to do your research here. What I chose was the Udacity Introduction to Machine Learning. But it could be any other one, which will fit for you. It's very well organized, and it's using Python and Sklearn, and this was a plus for me.
08:01
And also, for me, it was very important that you can do it at your own pace, because I can't stick to deadlines. So this was a plus for me. And one thing that caught my attention was the subtitle, which is Battle Recognition for Fun and Profit.
08:20
I can't imagine about the profit, but what about the fun? And the course starts like this. A couple of friends sharing a bottle of wine, not bad. And it ends like this, with same friends and more wine. So it seems that machine learning can also be fun after all.
08:41
So what is machine learning? I want to explain very basic concepts and some insights that, many times, are not well explained or just overlooked when you start with machine learning. I think that there are things that are important, at least for me, I want to share them with you.
09:01
Of course, this is not a short course, and you should learn and read books and do courses and these things. But when we work in machine learning, we want to solve a complex problem. So we have something here in the middle. And you have some inputs, which is the data.
09:20
Machine learning is features. And we have to do a prediction, which is the outcome of this thing here in the middle. The first approach, classical approach, is programming. Probably all of us here know how to program. So you have to understand your problem, and you have to explain it step by step,
09:42
in baby steps, to the computer, so that it can follow your steps to achieve the solution. But the problem is that, for very complex domains, such as image recognition, medicine, these kind of things, this doesn't scale.
10:02
Because let's imagine that we have to solve something for medicine. Medicine, maybe you have to quote thousands and thousands of rules, and these rules are not exact, because medicine is not an exact science. So maybe you have conflicts, and you have to solve them.
10:20
And it's very difficult to solve. But if you use machine learning, the approach is totally different. Instead of explaining the computer, what are the steps to solve the problem, you show the computer some real world data, some examples of data, and you let the algorithm learn,
10:42
and take its own conclusions from the data. This has huge implications, and it's that we can teach computers to do things that we don't know how to do. And I'll show you that in some minutes.
11:00
So I'll try to explain from a user point of view, I'm not going to enter into mathematical details, because I studied mathematics something like 23 years ago, and I couldn't remember, but I'll try to explain from the point of view of users
11:21
how machine learning works at a high level. And we will solve an example for character recognition. So we will have thousands of images like this, each one is containing a character, a letter. And we will have the labels telling us that the first letter is an F,
11:40
the second an E, and so on. So the first step in machine learning is getting our data. We have that blue image that corresponds to an F there, the yellow image, which is a G, and so on. Second step is to choose an algorithm. Lorena, in the talk before this,
12:02
was talking about Naive Bayes, but there are many, many tens or maybe hundreds of algorithms out there. There is super vector machines, or k-means, decision trees, neural networks, many of them. And you have to choose. And there are different algorithms
12:21
that can be a good fit for your project. And also, these algorithms can be configured. So you have to choose a combination of algorithms and configuration. So this is part of the machine learning art. And then you train your algorithm.
12:42
For training the algorithm, you start by showing some images, and you say, okay, this blue image is an F. And this yellow image is a G, and so on. In some respect, it's like showing a baby to read.
13:01
It's like teaching a baby to read. First step is to get predictions. So you show new data to the algorithm, and it will tell you, okay, I think this is a D. But the tricky question here is, is that letter really a D?
13:20
Is our algorithm predicting correctly the letters inside? We will answer this question later. So we have a lot of tools, and we have different languages, such as MATLAB and R,
13:45
but here we're going to talk about Python, of course. And I don't have to share here the beauty of Python, but it's a very good fit, and it's following its own philosophy of patterns included. There are many, many libraries with very good quality
14:03
that will help you to solve problems very easily. And we will solve the character recognition problem using Python, and SKlearn is one of the most popular libraries out there. It's open-source with Python, and the documentation is wonderful,
14:21
not just for the library, but just as a reference of algorithms and methodologies and everything. And SKlearn gives support to the full life cycle of machine learning. So we will solve this problem. We will do the following steps.
14:41
We will get the features with labels. We already have the data. We will choose and configure an algorithm. We will be using logistic regression with no configuration. We will train the algorithm. Then we will do predictions, and then we will validate them.
15:00
So let's get started with the example. We have this big dataset here. It's thousands of images with the labels. We want to separate them in two different datasets. One dataset is for training. The other is for testing, for validating our results. And we will use train test split function from SKlearn,
15:23
and we will split our dataset and labels. We will give a size for our training dataset. Size matters, but we will not explain why here. And there is a random state that happens to be a constant,
15:41
and this shocked me in the very beginning. Why the hell are we giving a random state that happens to be a constant? And the answer is that the train test split will separate our big dataset in a random set of images for the training and the test.
16:03
But by passing this constant, the selection of images for each dataset will be the same always, so that we can compare different configuration of the algorithms or different algorithms. Then we initialize our algorithm, logistic regression, no configuration,
16:22
and we do the fit. The fit is the training, and we pass the training dataset and the labels. This is the part where we teach the baby to read. But it's very easy. We don't have to iterate and do things. SKlearn does it for us. To do predictions, we just call the predict function
16:42
on our classifier, and that's all. But the question is if our predictions were good or bad. To solve this question, we will use accuracy. There are many different ways in machine learning, but we will use accuracies. It's the simplest one, and we don't have time for more.
17:03
So what we do is do predictions on our test dataset, and we have some test predictions. What we do is compare the test labels, which is the ground truth, the things we know are truth, to the test predictions. And this way we know the percentage of images
17:23
that we have been predicting correctly. With this example I just gave you, we got 89% of accuracy. This means that of 500, 5,000 images, we're predicting, well, 90% almost.
17:40
In just five lines of code. And this is what I mentioned before. I never did image recognition or image processing. It would take me weeks or months to implement such a thing without machine learning. And we did it here in 20 minutes together.
18:00
Of course we can improve these results. We can play with training data, as Lorena from the previous talk mentioned before. We can change the algorithm and the configurations. And this is very easy using SKlearn, because the APIs for the different algorithms is almost the same.
18:21
And you can give SKlearn different configurations for one algorithm, and it will test all the different configurations and give you the best one. So we already seen this very simple example
18:42
with classical machine learning. Currently I'm doing another course at Udacity of deep learning. I just did the very first lesson, and I did the first exercise of couple of exercises. And I want to present them to you,
19:01
because I think that it's part of code that you more or less can understand. I'm not going to explain neural networks, and please don't ask it, because I don't know anything about it. But I think that you can get an intuition on how we are going to structure the code.
19:20
So this is the course I'm still doing, so I'm going to run. We will use TensorFlow, which is a library by Google, and who is the author of the course as well. And it seems that deep learning is not so fun, because there is no wine in the course for the moment,
19:40
but I'm still in the first two lessons. I will, I keep my hope. So we will be using TensorFlow, and I'll jump directly to the code. The difference between SKlearn and TensorFlow is that SKlearn is using an imperative API,
20:03
and in TensorFlow what you do is describe a set of mathematical operations in a kind of graph, and then you execute that graph with your data. So you have to know what are the mathematical operations to implement your neural network.
20:23
So please don't make questions about this, but this is the simplest possible neural network in the world. This is two layers, neural network, and here in the left we have an image, X, and there on the right, you're right,
20:40
we have a prediction. The prediction is a matrix with a set of probabilities for each letter. So we will pick the letter with more probability. And the implementation is very easy. The image is a matrix, and we will do the matrix multiplied by a matrix of weights
21:01
and we will add another matrix, which is the bias, and we will apply a ReLU function, which is a kind of filter. This is what I understood from the first lesson. And you have then, you are making a pipe. The result from each one is going to be piped to the next.
21:22
Then we go to the second layer, which is again a matrix multiplication, an addition of a bias, and we will get a matrix of weights. And then we will do a softmax, which is a function that transforms a set of weights in a set of probabilities.
21:42
So the implementation, it's very simple. It's, if you remember, you have, remember, please? What we do is, we do a multiplication of matrices, of our test dataset with the first matrix of weights
22:01
for the first layer, and we add the biases. This is layer one logits. And then we apply the ReLU on the layer one logits. And we will have the output for the first layer. Then we repeat. We do the matrix multiplication,
22:23
and then the addition, and then, whoop. And then we apply the softmax over the output of the previous operation. And this is the implementation of a simple neural network. This is the production neural network.
22:40
You have to train it so that you can have these different matrices with correct values for your problem, and it's much more complex, and maybe next year I will explain that. So this is what I wanted to share with you. The idea is that, I'll do a very quick summary.
23:05
Machine learning is here, already in our lives. And it will become very important for everybody. So if you are a dummy like me, machine learning, don't be afraid. You can do very funny things with the resources and tools
23:22
that we have there outside, with very good quality. Free open source to use, and you can do very funny things. So if you want to use them and do interesting things, just do it for your own profit, and of course for fun. Thank you.
23:53
Time for questions.
24:15
Thank you. Compared to regression analysis,
24:22
neural networks are known to be, better take a lot of time. But in your experience, at least you're using Python. It's not as fast as C++.
24:45
Thank you for the question. So, in my experience, the question is that, as I understand the question, and correct me if I'm wrong, the question is that how compares the performance
25:02
from the first example and from the second, and also how compares that with the performance with other languages, such as C++. And the response is very easy for me, because I'm not sure what to answer, but in my experience, the performance of the,
25:24
at least for these neural networks, is more than enough. And there have been many advances in last year, with the adoption of graphic GPUs,
25:41
and different algorithms. For example, the ReLU function we are using here, instead of sinusoidal functions, have made possible to train very complex networks. In any case, I don't know how it compares with C++ or others, but what is sure,
26:01
that if you want to train very complex models, you will need very, very expensive hardware. But you can play with it with just a small laptop, in my experience. Thank you.
26:23
I also have a small question. Like, there are a lot of tools with different level of support, for example, which provide basically API for machine learning, like API.ai, Watson from IBM, something from Google. My question is, have you used them, and is it worth to play around with them?
26:42
For example, for text recognition, they do a lot of interesting stuff, as well as they claim to. Or is it better to just develop something on your own? Thank you. I don't have a lot of experience with that system. I just, some weeks ago, I did a small prototype
27:02
of a chatbot, using natural language processing libraries from Microsoft, in that case. And my experience is that these models are already very well trained for very specific purposes. And if you have the time and money,
27:23
and if you're a core business, probably it's better for you to train your own models and have people specialized. If it's some side thing, it's not main core business, you don't have the money, probably these already built models are more than good enough for you,
27:40
but for very specific things, you will not have already trained things. But it's my experience. Thank you. Questions? So at 5.15, we've got some lightning talks.
28:00
Otherwise, tomorrow at 9 a.m., there will be a fantastic keynote by Paul Indurant, who will be talking about the use of Python at Walt Disney, and I'm told it's a great talk. So I encourage you all to come. Let's thank the speakers again. Thank you.
Recommendations
Series of 5 media