neat-EO.pink: Computer Vision framework for Geospatial Imagery
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/46902 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
WindowField (computer science)Sinc functionForm (programming)BuildingMathematical analysisFocus (optics)Machine visionSoftware frameworkComputational visualisticsBridging (networking)Presentation of a groupPhysical lawLoop (music)Extension (kinesiology)System callProcess (computing)Computer animation
01:06
Loop (music)CyberneticsWiener, NorbertLoop (music)Point (geometry)Computer animation
01:33
PixelInformationAuthorizationComputer animation
01:52
PixelPixelFunction (mathematics)DistancePoint (geometry)Endliche ModelltheorieoutputFunktionalanalysisArtificial neural networkInsertion lossData compressionComputational visualisticsWhiteboardWave packetArithmetic meanInformationComputer animation
02:58
Graph (mathematics)Data compressionData modelTask (computing)InformationWave packetDirection (geometry)Data compressionBridging (networking)Computer animation
03:31
Software frameworkMachine visionFunction (mathematics)Mathematical analysisPairwise comparisonInsertion lossBridging (networking)Instance (computer science)Computational visualisticsIncidence algebraNetwork topologyFunction (mathematics)Different (Kate Ryan album)Task (computing)Mathematical analysisComputer animationProgram flowchart
04:03
UsabilityGreen's functionData modelSign (mathematics)Endliche ModelltheorieInstance (computer science)Asynchronous Transfer ModeGreen computingMultiplication signNetwork topologyDifferent (Kate Ryan album)BuildingOnline helpComputer animation
05:01
Function (mathematics)Pairwise comparisonoutputInsertion lossMerkmalsextraktionDifferent (Kate Ryan album)Function (mathematics)TorusMathematicsIdentity managementNetwork topologyInstance (computer science)AreaoutputComputational visualisticsSoftware frameworkPredictabilityEndliche ModelltheorieWave packetComputer animationProgram flowchart
06:06
Just-in-Time-CompilerFile formatServer (computing)Programmable read-only memoryRevision controlWave packetPredictionData modeloutputComputer-generated imageryCompilation albumRaster graphicsVector spaceInterface (computing)Line (geometry)Different (Kate Ryan album)Product (business)Line (geometry)Interface (computing)Computer animation
06:25
PredictionPairwise comparisonVector spaceDifferent (Kate Ryan album)Auditory maskingPredictabilityComputer animationProgram flowchart
06:46
Open setOpen sourceBridging (networking)SoftwareMereologyLogic gateComputational visualisticsDiagram
07:19
Installation artInferenceNetwork topologyLine (geometry)QuicksortWave packetSoftwareResultantInterface (computing)Semiconductor memoryPoint (geometry)Rule of inferenceThumbnailComputer animation
08:12
File formatGraphics processing unitSemiconductor memoryMultiplication signGraphics processing unitPoint (geometry)BuildingSymmetry (physics)Computer animation
08:30
Open setSet (mathematics)Point (geometry)BuildingWave packetPredictabilityProcess (computing)Goodness of fitComputer animation
09:09
Pairwise comparisonMathematical analysisInstance (computer science)Computer animation
09:25
Binary fileBuildingMultiplication signRight angleKnotTheoryInstance (computer science)2 (number)1 (number)Computer animation
10:01
Meta elementComputer configurationResultantTesselationPredictabilityComputer configurationProcess (computing)Focus (optics)Product (business)TwitterMedical imagingComputer animation
10:31
PredictionScale (map)MultiplicationGraphics processing unitScaling (geometry)PredictabilityWave packetGraphics processing unitScaling (geometry)Computer animation
10:49
MultiplicationSocial classWeight functionComputer configurationComputer configurationLattice (order)Social classWeightWikiComputer animation
11:22
PredictionThumbnailTopologyImage resolutionTournament (medieval)Right angleOnline helpWave packetLimit (category theory)ResultantImage resolutionSet (mathematics)Product (business)CodeImaginary numberInstance (computer science)VotingData compressionSurfaceSoftwareComputer hardwarePredictabilityNetwork topology1 (number)Computer animation
13:28
Open sourceRaster graphicsMachine visionStandard deviationMusical ensembleData fusionMultiplicationOpen sourceProjective planeMachine visionVotingParameter (computer programming)Level (video gaming)Computational visualisticsInterface (computing)outputArithmetic progressionLine (geometry)Template (C++)Web 2.0Data fusionFile formatComputer animation
15:01
MathematicsMachine visionSoftware frameworkOpen sourceExpert systemData modelOpen setSlide ruleField (computer science)Software frameworkRevision controlEndliche ModelltheorieData compressionPoint (geometry)Open sourceMachine visionComputational visualisticsStatisticsMultiplication signProjective planeWeb 2.0Cellular automatonOctahedronOpen setInformationComputer animation
17:13
Open sourceCAN busMachine visionRaster graphicsFlow separationSoftware frameworkGoodness of fitInformationComputer animation
17:53
Computer configurationMeta elementBuildingBinary fileInstallation artInferenceOrder (biology)WeightResultantExecution unitCodierung <Programmierung>MereologyComputer animation
18:22
Open setGraph coloringCondition numberSelf-organizationNoise (electronics)Video gameArmImaginary numberoutputFlow separationEndliche ModelltheorieAugmented realityShift operatorDiagram
19:06
Installation artInferenceMathematical analysisMeta elementComputer configurationImage resolutionTopologyMachine visionOpen sourceRaster graphicsCAN busFacebookPoint cloudComputer animation
Transcript: English(auto-generated)
00:05
Olivier Courtem, I've been working in GIS fields since years, and I founded Datapink company. One of our main focus is to build bridges between GIS fields and deep learning stuff.
00:27
So how are we able to extract insights from geospatial data through the latest available tool that we have? We already knew how to do it with spatial analysis in a classical way, but what can we extend with new tools to go further?
00:45
That's it. And so this presentation is about a computer vision tool able to extract information from imagery. That's it. That was my one minute presentation.
01:01
So the framework is called neteo.pink. And if we go back to the history, everything begin with a loop. So it's a vinyl loop one century ago. And the point is to say that you have to understand what is wrong before to be able to fix it.
01:24
And you look again and again. But since you are not able to understand what is wrong, there is nothing to do about. So if we look about the authorizations nowadays, it's widely used. There is a vast amount of information available, but we are unable to use it really.
01:46
And most of the pixels acquired, we don't do anything with them. It's a waste. So the idea, the need is to say, OK, we are able to gather pixels. But what can we do with this pixel to be able to switch from pixel to insights?
02:05
That's the key point. If we look about the deep learning stuff, supervised learning is quite simple. For example, you have one input, one expected output. And you train a neural network till it is able to compute the output from the input.
02:22
And the key point is the loss function. The loss function is the ability to compute the distance between what it performs and what it expects. And since you are able to compute a meaningful distance function, you are then able to find a way to converge to a solution.
02:44
So since you train a model rightly, you are able only with the input to use your trained model to compute an output. That's it. So it's only a way to train a model. So what is really a trained model?
03:01
Well, there are several ways to understand it. We will focus only on one because it is simple to understand. It's a compression. You only compress your information, the whole imagery information, and you accept to lose a lot of information. But you keep all the information you need to achieve your task.
03:23
So it's just a direct compression stuff to lose information, but the one you have to keep for the classification. So once you understand it, Neteo.pink is just a way to build a bridge between geospatial data and deep learning stuff.
03:42
There are three tasks we are focused on. First is quality analysis. So once you train your data and you are able to compute an output, you will be able also to compare this output to an uncertain dataset. And so to put in evidence, if there is a significant difference, yes or no.
04:03
And for instance here, in pink, it's what is predicted by the model. In pink, it's what is predicted by the model. In green, it's the labels. And in the gray is when both are agree.
04:20
So it's a quick way to be able to check if the model and the label are matching, yes or no. And to see that this one, for instance, it's because it's behind the trees. So it's harder for the model to find a building because it's hidden.
04:42
And it's also interesting to see that if you zoom out, there are Spotify differences that will help you to save time because we will focus only on the path where there are enough differences between your two datasets.
05:03
One other thing you can perform with exactly the same framework is change detection. Because in this scenario, you will train a model and use an alternate input. For example, something like one year later or two years later or one week at any time.
05:25
And you will compute an alternate output and compare the difference. And the last one is feature extraction. You will train your model with a small dataset, a train area, and you will use a wider imagery input to compute outputs.
05:48
For instance, you will only label a small area. And once your model is trained, you will be able to launch a prediction on a wider area and thereafter to vectorize it.
06:02
So to be able to use the tree scenario, we have several little tools you can assemble with Lego stuff. So there are small tools and you can change them to create your own workflow.
06:23
So it's a command line interface. And the whole idea is the ability to deal with different kinds of imagery, different kinds of well-known labels formats, to compute the prediction and thereafter to do something meaningful as insights from the prediction mask you generated.
06:43
So that's the key concept. About the stacks, we reuse some well-known computer software from the GIS field. Some came from Python imagery and some came from deep learning world.
07:05
So it's a bridge between these worlds. It's full open source except the NVIDIA part because it's not open source by NVIDIA itself.
07:20
It's easy to deploy. It's not because we use a lot of software that we didn't package. So it's just a single line to install the whole stuff. There is a 101 tutorial to help you to do it by yourself and so to learn by yourself by doing it with real-world data. And something from the install to the data preparation training and so on
07:47
takes something like two, three hours to launch the whole training session and the whole results. So it's also available online.
08:01
So if you want to look at it right now, you just have to click on it and you click on the different picture to zoom in the leaflet interfaces. So all you need in fact is imagery and we've seen that it's not a problem anymore, GPU and you need at least a recent GPU with enough memory and labels.
08:27
And so the key point is the label because most of the time we have labels but not accurate enough. So if you look here on OSM building the data, we have buildings from DC imagery.
08:44
It's true. But they are not accurate enough to be sure that your trained model, if you use this kind of data, will be really accurate. So it's something important to keep in mind that if you are garbage in label,
09:02
you will have a garbage out prediction. Yeah. So the point here is also to reuse this technique to check that your data set training is quality enough. And so to compare your labels you use here with these very tools
09:25
and so also be able to keep or not some labels because for instance here, these buildings appear in the label but it's not right related to this imagery
09:42
because right now on this imagery there is no buildings at this time. So the most common way is to remove these ones and there is an integrated tool to help you to keep yes and no in a second.
10:02
So what's new inside an ETO.pink? First is the ability to enhance the quality of the prediction despite the fact that we use tiles. So it's well known. We use meta tiles to enlarge the focus.
10:20
Obviously, it will take longer if we choose this option but the result here is cleaner. So if we slow the whole process, we have to improve it some other way. So we add the multi-GPU scaling to help to use as many GPUs you can get on a single host.
10:45
And for the training of prediction. So you can scale it up and also we add the multi-classes. So you are not only obliged to use it on a single class. You can use on several classes at once.
11:00
And we also provide an iterated unbalanced option to help you to give weights related to the classes if they are not distributed in the same way in your dataset.
11:23
So what's the limit right now? First, the kind of imagery you want to predict on must be quite related to the ones you already trained on. Remember, it's a kind of compression. So you can't expect a good result if the kind of imagery has nothing related to the ones you use for the training.
11:49
About the labels, we've seen that you need something accurate. And the amount of labels you need is something like thousands. So at the very least, one thousand or a few thousand, it depends, but not a dozen.
12:05
And also, right now, it doesn't deal with topology. So if you have something you want to extract and it's topology related, something like a network or roads, for instance, it doesn't work well. It's far better to behave with surfaces, any kind of surfaces.
12:24
Right now, we are still working hard on it, and we are looking for funding for sponsors in any way. It could be related to code, so pull requests are really welcome. It could be related to money funding.
12:41
It could be related to hardware funding and so on. So help us to increase the gain accuracy, especially on low resolution. We talked just before about Copernicus and Sentinel. The next step, obviously, will be to behave and to increase the resolution even before we perform training and prediction on it.
13:04
The topology, obviously, and to reduce the amount of labels we need before being able to have an accurate training. And also, go on the performance improvement, because it's not necessary anymore to have a huge infrastructure to be able to use it.
13:25
You can, but it's not mandatory. There is alternative open source to this project. There is RasterVision, YouLearn, Robosat, Solaris. So why choosing this one?
13:40
A few arguments. We really focus on just under compliancy. So it's really easy and standard compliant to reuse all the geospatial format that you work daily with. Your data preparation will be easy and fast. There is a built-in web UI interface which helps you to check at every step that everything is online, yes or no.
14:06
It's modular and extensible, so it's not something you can't extend easily. On the contrary, you can really easily add new tools, add new interfaces, add new templates, and so on.
14:21
It also handles multiband imagery and also handles data fusion. So you can, for instance, use vector, rasterize your vector, and add an input from your imagery. So it's really the same kind that a GIS map.
14:43
You compose your map by adding several layers, and with this stack, you can train and then compute. It's high-performance and it's accurate, because we use and reuse the latest computer vision papers available.
15:02
If this field interests you, there is in one slide all the best resources to learn from it. And yeah, one slide about the company. To take away, there is right now Androcl Open Source IE 4.0 framework available.
15:23
The performance is already okay to use it at a country level. You do not need anymore to be a computer vision scientist to use it. A geospatial guy who understands the idea of the compression can do it right now.
15:41
Plain open data could be used to train a model because you can use it step by step, and so to refine your labels till they are accurate enough. And funding and pull requests are really welcome. That's it.
16:05
For once, we have more than one minute time for questions, so bring it on. Yes, please. How does this compare to the Google web page? It was just a rename. Could you repeat the question a little louder, please?
16:22
There's another framework called Robosat Pink which is similar? Yeah, Robosat Pink was the name of the previous version. Niteo.pink is a rename. It's a 0.0.7 version of the 0.6 Robosat Pink version. The point is, there is Robosat Project and Robosat Pink.
16:43
And people, a lot of people, made the mistake. And when I told Robosat Pink, a lot of people understand Robosat. And so at the end, it was a mess. So I said, OK, we stop it. We rename it. So it's Niteo. There is no Niteo anywhere. So it's a new name and goal.
17:01
So it's a 0.0.7 version. Yep. Do you have any statistics on Niteo? Do you relate it to these ones? No. So right now, I don't have any information that anyone compares this whole framework.
17:25
It will be a really good idea on several aspects related to accuracy, related to performances, and related to how easy it is to use them. Yep. It's a need, in fact. But I don't know anyone who did it before.
17:45
The example of the used PyTorch. Yep. Yeah. It's a unit-like with an encoder. And you reuse an encoder as a ResNet-like.
18:03
So you can choose any kind of ResNet as an encoder. It's a unit-like. And there is also a co-py from the encoder to the decoder as a unit, but also on the decoder part.
18:23
Because you already have all the PyTorch and the augmentation. Augmentation is data augmentation. Yeah, data augmentation. And data augmentation is able to deal with multichannel. And data augmentation is able to deal with several kinds of imagery, color shifting.
18:44
So you can add a lot of noise in your model. And because you add a lot of noise, the model is forced to generalize enough to be able to still work, even if the imagery in the input will slightly change. Some big augmentation. Yep.
19:05
Then once again, thank you very much. Thanks.