Make you IOT Smarter with Tensorflow Lite...
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47483 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Projective planeSoftwareInternet der DingeCartesian coordinate systemField (computer science)GravitationNumberRight angleBootingVirtual machineSoftware developerCloud computingElectronic design automationInternetworkingComputer animation
00:51
Virtual machineField (computer science)Form (programming)Computer animation
01:15
Graph (mathematics)WeightData modelLaptopData conversionInternet der DingeInferenceEndliche ModelltheorieArtificial neural networkPredictabilityMultiplication signProduct (business)Computer animation
02:29
DigitizingMultiplication signFocus (optics)Game theoryComputer animation
03:00
System programmingArchitectureThermal conductivityVector potentialFluxPopulation densityVector potentialGame controllerLevel (video gaming)Water vaporProjective planeConnectivity (graph theory)Product (business)Physical system1 (number)Computer animation
03:42
System programmingDataflowSurface2 (number)Water vaporDifferent (Kate Ryan album)Category of beingPressureRootkitTwitterPlanningUsabilityMusical ensembleOperator (mathematics)Performance appraisalPrototypePhysical systemMultiplication signComputer animation
04:53
ArchitectureSystem programmingMotherboardMultiplication signRule of inferenceComputer animation
05:36
ResultantGraph (mathematics)Point cloudPhysical systemComputer animation
05:58
Cloud computingComputer animationSource codeXML
06:15
ArchitectureSystem programmingEndliche ModelltheorieArchaeological field surveyLaptopElectronic data processingVirtual machineInformation engineeringDatabaseMachine learningComputer animation
06:33
Machine learningPhysical systemVirtual machineIntegrated development environmentPhysical systemAreaNatural numberArithmetic meanComputer animation
07:03
Inclusion mapMachine learningPhysical systemMachine learningForm (programming)AreaPhysical systemInstance (computer science)Computer animation
07:20
Graph (mathematics)Point cloudScalabilityComputer networkInformation privacyCartesian coordinate systemVirtual machineInstance (computer science)CASE <Informatik>Physical systemPersonal digital assistantForcing (mathematics)Computer animation
07:53
Computing platformMachine learningGoogolOpen sourceLibrary (computing)Data modelArtificial neural networkTensorWhiteboardGraph (mathematics)Analog-to-digital converterInterpreter (computing)InferenceHeat transferAndroid (robot)MicrocontrollerDifferent (Kate Ryan album)Mathematical optimizationDistanceWeb 2.0GoogolLie groupDenial-of-service attackComputing platformEndliche ModelltheorieWave packetMereologyShared memorySet (mathematics)Right anglePoint (geometry)Virtual machineInterpreter (computing)DataflowRevision controlSoftwareData conversionOnline helpArithmetic meanOpen sourceArtificial neural networkSemiconductor memoryMobile WebInferenceMachine learning2 (number)Computer animation
10:01
Geometric quantizationMathematical optimizationGraphics processing unitDigital signal processorDatabase normalizationBuildingData modelWave packetTensoroutputPreprocessorRead-only memoryInferenceStructural loadFunction (mathematics)Semiconductor memoryLaptopInferenceData conversionDifferent (Kate Ryan album)Geometric quantizationEndliche ModelltheorieBefehlsprozessorWave packetoutputMathematical optimizationArtificial neural networkFunction (mathematics)IntegerInstance (computer science)MicrocontrollerComputer filePoint cloudDecision theoryNeuroinformatikInformation securityMereologyIntegrated development environmentGame controllerPoint (geometry)Sound effectData managementArithmetic meanProcess (computing)Pattern languageBeat (acoustics)Zoom lensCellular automatonRight angleTerm (mathematics)Computer animation
11:48
LaptopRevision controlWebsitePhysical systemDirected graphData managementLaptopNichtlineares GleichungssystemComputer animation
12:09
Interpreter (computing)LaptopBuildingComputer animation
12:45
LaptopInterpreter (computing)BuildingInstallation artRun time (program lifecycle phase)Line (geometry)Task (computing)Interpreter (computing)Computer animation
13:06
Wave packetBuildingData conversionMereologyArtificial neural networkWeightData modelInferenceSharewareEmailExecution unitGeometric quantizationStructural loadInterpreter (computing)IterationPlot (narrative)AreaMereologyInferenceEndliche ModelltheorieLaptopWeightData conversionParameter (computer programming)ResultantComputer fileWave packetSimulationRight angleMultiplication signMicrocontrollerElectronic mailing listInterpreter (computing)Structural loadTime zoneCollisionTunisEvent horizonRecurrence relationComputer animation
16:13
Rekursives neuronales NetzMilitary operationReinforcement learningMaizeGraph (mathematics)Recurrence relationArtificial neural networkEndliche ModelltheorieWave packetProcess (computing)Virtual machineConnectivity (graph theory)Revision controlArithmetic progressionWhiteboardInferenceMereologyComputer animation
17:11
System callMultiplication signLogic gateSharewareBitLink (knot theory)Machine codeComputer animation
17:38
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:05
Good morning, everybody. Thank you for coming. My name is Alexey Duke, and I'm the director of research at Atherton, a small company in France that is working in the field of
00:20
the Internet of Things, and we are working on electronic design and software, cloud application development, machine learning. And we do some research, collaborative research, and the project I will show you is the frontier between research and industrial applications.
00:42
So today we will speak about machine learning on automated devices and TensorFlow Lite. So how many of you heard about TensorFlow Lite? Oh, many people. And how many of you are working in the field of machine learning? Today you work every day on machine learning?
01:08
Yeah, okay. So it's mostly for most of you that you use TensorFlow Lite. And how many of you heard about vertical forming? About 50% of the audience.
01:23
Okay. So anyway, today I will try to introduce quickly what is indoor vertical forming, why we want to use intelligence at the age. I will try to explain quickly how
01:41
to use TensorFlow Lite and how it works, how to set up your device, your Raspberry Pi to use TensorFlow Lite on it, and then to build and train a small model to predict the latest weight, the latest growth. That's not the main topic of this talk,
02:04
but we will see quickly what we can do. And then how to deploy and to convert a Keras model, a deep neural network model to be able to use on a Raspberry Pi.
02:21
And then we will see how you can make prediction and inference on IoT devices. So what is indoor vertical forming? So vertical forming is a way to grow vegetable and food in stark layers, like in a building, like in containers,
02:41
in a cabinet. And the cabinet you see here is in our office, is the device that we are working on it. And we won't like to grow vegetable and lettuce in stark layers like it is. So the main advantage
03:02
of this kind of device and vertical forming is that you can have very controlled growth and you can monitor everything and control everything. You can check the temperature, the humidity, the level of CO2, and the reduced potentiality of the soil.
03:25
And so we designed this cabinet during our project and we put many sensors into it. So we have the CO2, dissolved oxygen, electrical conductivity, and the water pH, humidity, temperature, many sensors. And we can also control how we irrigate
03:47
the system, how we put water into it. So mainly we are trying different kinds of irrigation systems and we have one on each stack, one on each layer.
04:04
And the first category of irrigation system is aeroponics. It's when you dry the vegetables without a soil, so the main is HP8,
04:24
high pressure aeroponics, and the second is nebulization. It's like it creates a kind of fog in the surface of the lettuce. And the second kind of irrigation system is hydroponics.
04:43
So it's when the roots of the vegetable are constantly dry in the water. So our system, all the sensors and the actuator are controlled by a motherboard
05:01
that is Raspberry Pi and we are now working on making it more reliable with a custom board, with a custom embedded device. But the first time for prototyping we use Raspberry Pi. So that's what we call the motherboard. So the motherboard is developed in Python
05:21
and we run TensorFlow Lite into it and all the intelligence of the device is there. And we also use OpenCV to control camera that we use to monitor the growth of the vegetable. So here is the camera we use and the LED that we can also control.
05:47
And we use OpenCV to compute the size of the leaf of the lettuce. Then all the data of the system are sent into the cloud.
06:08
And then it's our cloud application to monitor, to live monitor the device. And then we can connect later to the database in the LED laptops to design the model
06:26
to work to do data engineering and data processing before doing machine learning. So why we want to do machine learning at the edge and not in the cloud? So we would like you to know if the system works as we expect
06:44
when we would be able to eat the lettuce, how we can control the environment to make the vegetable better and look better and taste better.
07:01
And we would like to make it grow faster. And there are some people that are working in the area in MIT and they published some paper on improving the taste with the basil using machine learning and vertical forming.
07:21
So why doing machine learning at the edge? So sometimes we don't want to make the system already connected to internet. We want to make it more scalable. We don't want to make it need Wi-Fi, for instance. We want to reduce the latency.
07:40
And it's not really the case for this application. But for other kind of application like voice assistance, intelligence at the edge can solve privacy issues. So what is TensorFlow? TensorFlow is a platform, open source platform by Google to do machine learning.
08:03
The last release is 2.0 has been released in October last year. And it helps you to create and train different kind of machine learning model. You can do neural network but not only. So there are a lot of tools that come with TensorFlow to do machine learning at the edge
08:25
and mobile platform on a device. But you also have TensorFlow.js to do machine learning in the navigator, in the web. So different kind of tools that come with TensorFlow. So TensorFlow Lite comes with two parts.
08:41
The first one is a converter that helps you to convert deep neural network that has been with TensorFlow and Keras into TensorFlow Lite model to make it able to be run on a smaller device like a mobile phone, like an embedded device, Raspberry Pi, and also microcontrollers.
09:03
And the other part, the second part of TensorFlow Lite is a TensorFlow Lite interpreter that is what you will put into your device. And that's this kind of software that will run inference on the device. So there are different version of the interpreter depending on if you are on a Linux,
09:25
on a mobile phone, or on a microcontroller. And TensorFlow Lite help you to design smaller model to make it able to fit into constrained device with low memory footprints and also to improve the inference speed.
09:43
But with TensorFlow Lite, you can't train your model. Your model is frozen. You can't do retraining and transfer learning with TensorFlow Lite. And so you just can't do inference, no training.
10:02
So it come with different kind of optimizer to do some training in your neural network just before training it. And there are also post training quantization so you can reduce the memory footprint using 8-bit integer instead of floating point, for instance.
10:28
And you can also have, for the inference part, you can have some delegates to move the computation from the microcontroller from the CPU to the GPU if you have the chance to have a GPU in your device.
10:42
Or in your GPU if you use some CPU on your device. So what is a workflow to use TensorFlow Lite? So the first thing you have to do is to import your datasets,
11:00
to work on the data, to do some preprocessing, normalization, set your feature, and you build your model with TensorFlow on your laptop. You train your model on your laptop or in your GPU in the cloud. And then you convert it using TensorFlow converter on your laptop. And this is the TFLite file that you will put into your device.
11:24
So everything there is done on your laptop. And on your embedded device you load your model, the TFLite model, you preprocess the input data from the sensor, and you allocate the memory, you run inference,
11:41
and then you interpret the output of the inference, and you take decision. So on your laptop it's quite easy to use TensorFlow Lite, to use TensorFlow, so you just need to install Python and create a vital environment, instead of dependency,
12:04
and you can install TensorFlow using Py package manager. And on your Raspberry Pi it's more complicated, you need to install only the interpreter,
12:20
so you can use the official package manager, but it's not always up to date, so you can build by yourself on your device, or you can cross-compile it, but it can take some time. But there are some people that are offering a package
12:42
that is already built for your device, and that's the fastest way, if you're not familiar with cross-compilation. So basically on your Raspberry Pi you just need to get the interpreter
13:03
using this common line. So right now I will show you a quick demo, the first part is to build your model, to train your model to process your data, I will go further because not the topic of this talk, the more interesting part is the conversion into a TensorFlow Lite model,
13:25
and then I will try to show you how to run inference on your Raspberry Pi. So I will use your Jupyter notebook to show you that,
13:41
so the first thing I will do is to load the datasets, and then the different parameters are important, so the light, the temperature, and the kind of irrigation.
14:04
So here is how I build the model, so I use the Keras API and I just will use a simple model with two dense layers with 64 neurons on each layer.
14:21
So here is what my model looks like, so then I have already trained, because it can take a lot of time, the model, and then there I run inference, and I see that there I will predict, just to check,
14:43
that my model is correct, so I can predict the weight of the lettuce after some day I plant it, and then there is an interesting part, I will convert the model into TensorFlow Lite, so I can use the Python API there,
15:02
and I show that my model then after TensorFlow Lite conversion is 20 bytes. I can also put it in the .h file to put it in a microcontroller, and then after quantization,
15:23
I see that my model is reduced to 8 bytes only instead of 20. So now I will run on the Raspberry Pi, so I am now there on the Raspberry Pi, I will load my data, I feed my interpreter,
15:43
and then I will do exactly the same things on the Raspberry Pi, and I will plot the results, and there I plot the results that are quite similar to the main model on the laptop,
16:03
and we see that TensorFlow Lite doesn't reduce the performance of the model. So TensorFlow Lite has some limitations, you can't do every kind of machine learning model with it,
16:22
you can do recurrent neural network like LSTM, but it's a work in progress at Google, and I think that they will probably release this soon, and in the beta version you can already try it.
16:41
And further work, we can do training ideas and not only influence, also training to do transfer learning, to do federated learning, and you can use this kind of device like the Corel board, or the TPU that is a small component you can put into your custom embedded device board,
17:05
or you can use the USB there if you have a Raspberry Pi to improve the training process. I think that we are a bit out of time, so I can take some questions later offline, I'm there all the day, so feel free to ping me,
17:22
and you can get all the codes of the demo I showed there on my GitHub, and following this link or sending this here. So thank you for following me.