fAIr - Free and Open Source AI for Humanitarian Mapping
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66400 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Gamma functionEndliche ModelltheorieTexture mappingThumbnailMappingProjective planeProcedural programmingSoftware developerOpen sourceLink (knot theory)Open setDependent and independent variablesFreewareTexture mappingTerm (mathematics)Computer virusPerfect groupFront and back ends
01:00
Texture mappingEndliche ModelltheorieWave packetMappingProjective planeAreaCodeVirtual machineOpen setSoftware developerMachine learningEndliche ModelltheorieSlide rule
02:03
Service (economics)Endliche ModelltheorieLocal ringData modelOpen sourceTexture mappingRight angleDecision theoryArithmetic meanOpen setLoop (music)PredictabilityBuildingEndliche ModelltheorieSpacetimeMedical imagingMereologyOpen sourceSlide ruleLocal ringChannel capacitySet (mathematics)Computing platformTexture mappingAreaMappingNatural numberInformationRule of inferenceDiagram
04:06
Texture mappingPower (physics)CollaborationismEndliche ModelltheorieInformationLink (knot theory)Right angleLattice (order)Alpha (investment)Revision controlOpen setBuildingArithmetic meanAreaLocal ringSoftwareSet (mathematics)
06:03
Endliche ModelltheorieData modelEndliche ModelltheorieData managementAreaFeedbackProjective planeSampling (statistics)Revision controlMappingTexture mappingComputer architectureInternet service providerTask (computing)Slide ruleMedical imagingService (economics)Local ringSatelliteWave packetBuildingRaw image formatOpen setSet (mathematics)TesselationVector spaceStapeldateiMereologyOnline helpOrder (biology)PredictabilityProcess (computing)Computer hardwarePrisoner's dilemmaValidity (statistics)Document management systemState of matterTunisProgram flowchart
10:46
Endliche ModelltheorieLibrary (computing)MetadataGeometryEndliche ModelltheorieCodeUtility softwareElectric generatorRegular graphTesselationUniqueness quantificationBound stateMedical imagingDebuggerExtension (kinesiology)Task (computing)PreprocessorMereologyPredictabilityOrder (biology)Physical systemInferenceBuildingProcess (computing)Front and back endsWave packetComputer reservations systemLevel (video gaming)FeedbackCASE <Informatik>Data managementVector spaceRaster graphicsOnline helpAuditory maskingTexture mappingBinary codeComputer architectureDifferent (Kate Ryan album)CuboidSoftware developerProgram flowchart
14:46
Term (mathematics)Endliche ModelltheorieFile formatMedical imagingService (economics)Text editorTexture mappingInternet service providerComputer animation
15:19
Inclusion mapFeedbackOpen sourceIdentity managementWorld Wide Web ConsortiumInformation privacyEndliche ModelltheorieTexture mappingFeedbackBuildingCollaborationismProcess (computing)Point (geometry)Machine learningComputer animation
16:02
ThumbnailEndliche ModelltheorieUniform resource locatorAreaSet (mathematics)Texture mappingOpen setMappingPrisoner's dilemmaEndliche ModelltheorieComputer animation
16:28
Endliche ModelltheorieAreaWave packetMappingSign (mathematics)Computer animation
16:43
Data miningWeb 2.0Local ringZoom lensTexture mappingEndliche ModelltheorieAsynchronous Transfer ModeEndliche ModelltheorieRadical (chemistry)Instance (computer science)Goodness of fitGraph (mathematics)Arithmetic meanBuildingProcess (computing)AreaPredictabilityWave packetMedical imagingResultantData structure2 (number)Computer animation
17:58
Endliche ModelltheorieData modelFeedbackPredictabilityView (database)Endliche ModelltheorieData managementComputer animation
18:17
Endliche ModelltheorieRippingComputer animation
18:45
Texture mappingEndliche Modelltheorie
Transcript: English(auto-generated)
00:08
Hello. Hope everyone can hear me. Perfect. Hello everyone. I'm Chithis. I'm from Humanitarian Open Street Map team. I've been working as a backend developer in Humanitarian Open Street Map team and I'm working in a project called FAIR since last year.
00:27
And active development has been ongoing and today I am going to share with you how we are thinking, how FAIR works and then what are the procedures we followed and why we want to use AI in Humanitarian mapping.
00:45
So, terms starts FAIR from, FAIR stands for Freedom and Free Open Source AI for percentage and responsibility for our communities within Humanitarian mapping.
01:00
So, why FAIR and for whom FAIR is designed to? This picture is actually from a project done by Open Mapping Hub. One of our colleague Nirav. So, they were mapping ponds in India. So, people like communities that we serve, communities on the ground.
01:21
FAIR is designed for community to be able to use AI without having appropriate knowledge of like how machine learning works, how training should be happening without having a coding knowledge. So, that means they will be able to apply machine learning in their project using their own data in their area.
01:43
So, this is why FAIR is designed and this is what we are targeting for. And FAIR also serves another kind of user which is technical users that allows developers to share their machine learning open models so that other communities will be able to use it.
02:01
I'll be talking about it further on next slide. So, as I'm mentioning why FAIR is, HART was running a pilot. AI has been in topic of discussion for a long in Humanitarian Open Street Map space. Since 2019 and 20, we have been running a pilot mapping buildings in Tanzania and some part of Africa.
02:25
And then we observed that models that we apply, actually Microsoft helped us mapping, digitizing the buildings. And there is lack of AI openness. Models are not open. We do not know why model made that prediction and then there was a bias in models.
02:46
Meaning, let's say, in especially places like Africa, places like ruler area, informal settlements and also in slum area, the nature of the building, the texture of the image that community is using was far different than the image that was used to train models.
03:06
And then model was making biases. And there was lack of human in the loop. So, we were literally like using already predicted features. And then there was no human involved when AI predictions are made, when
03:21
like there was no community involved. They don't know how data set was produced. And then since they are using it. And another thing is lack of AI capacity building, meaning like when, as I mentioned in earlier, when AI was designed, they don't know what were the base model applied, what is the architecture.
03:41
Like at least what was the base model for, what it is suitable for and is it really fits on the local data set. It is really fits on the local imagery. Right? So, they were not even involved on the decision making and this is why we thought, okay, we need a free and open source AI platform, easily available that can be used by community for the community.
04:07
So, the fair is, the alpha version of fair is here, but it's not, we have traveled through a couple of, we have volunteers, it has not been possible without all of their efforts.
04:21
So, there are humanitarian researchers, there is also one cool researcher volunteer of ours on the top, Chris did research on how local models have the better accuracy rather than having a global model. Meaning, let's say for on, as I give example of informal settlement, if model is created using the
04:45
informal settlement data and applied on the informal area, then those local models performs better than global model. I will also drop a link of the researcher as well. That is also one of the volunteer we worked
05:02
with and there is a ramp model, which we are using as a base model for building footprint detection right now. Ramp is applicable, AI developed for micro planning, developed by Dave Global, it is based on the open set is building data set and then community, local engineers, as you can see, there is a, we
05:25
also participate in NetHope AI challenge, credits to our colleague, Sena, she also holds NetHope AI meetings regularly. And then we also had a collaboration with a company called Omdina, which involved 50 AI engineers from all over the world,
05:45
who did research on, which we initially we were looking for building footprint detection, and then they did a research how, which model would be performing better, performed pre-processing, post-processing steps, which will be, I will be talking about on next step.
06:04
So this is a general workflow, how FAIR works, how we are working towards local model and implemented it. So I'll be explaining general workflow first, and then I'll be explaining the detailed technical architecture of the FAIR on the next slide.
06:23
So what happens is, initially we need to get the images for the area, let's say, the community can fly drone over the area, we have also tested this approach on here, right, prison, where we are having conference, and then drone can capture
06:44
the image, it could be satellite image, but it should be open, and it should follow the license that we are using. And so it can be uploaded to open aerial map that will generate a TMS, not necessarily open aerial map, it can be any TMS provider that compiles with the license, and then it will be going through the tasking manager.
07:07
Why tasking manager? We want to do manual mapping first, because model need to know what are the features, what are the area that we want to detect, so with the tasking manager we define a project area, we run a pilot on a sample area, let's say, if a drone is bigger for a hole
07:25
over prison, so we will run manual mapping for small area of here, which we think it would be enough for the sample data, enough for the training. And then that will be validated through the validators, it will make sure each building aligns with exactly
07:41
in the imagery, so that we will have a better training data, and then push to open street map. So once the data is pushed to open street map, it is faced by the service called raw data API, which is also API hardware, humanitarian open street map has been working on, and it is fast, it fetches the data every minute, and then it will get the data.
08:03
So now we have image as a TMS from open data map, data validated ready for training as a vector data set that can be used as a label that is available for us, which works as a data set for a local AI model.
08:21
So here on local data set, we have OSM features as a label, TMS as a images, tiles as images, and both are used to supply runner training on local model. Local model is basically a fine-tuned version of the base model, so here, as I mentioned before, currently we are using ramp as a base model.
08:44
So this base model can be, this is also a topic of research and topic of interest for us, in order to expand our feature detection process, let's say, later on, pond, road, the base model could be different, and then community will be choosing, let's say for
09:01
a building, maybe my ramp could work better for a pond, maybe some other yellow detection could work, a yellow base model could work better, and then that will be used in order to fine-tune that model with this local data set, and it will create a new checkpoint that will be specially created for that local area.
09:22
And then that local area, with the help of training, will be validated, so whoever community manager creating a model, they will be validating the accuracy of the model, they will run the prediction, see if it is becoming better and better, if not, there will be multiple trainings. So maybe he can try to add data sets, maybe labels are not aligning to the satellite imagery or increasing epoch and batch size, and
09:50
then he will finally, once he is satisfied, okay, this accuracy will make it better, is better, and we can use it for mapping on the further area, rest of the area, he will publish the model, and now here, one of the, another part we are looking at fair is feedback.
10:06
So as I mentioned before, model can have biases, model may not work on the area that we subject them to, wish them to do, so what we do here, let's say model made a bad prediction, oh no, you detected a feature that
10:22
is not supposed to detect, then mapper can provide a feedback back to the model that goes back to the manager, manager can validate the feedback, and then reapply the feedback on the newer version of the model, so model will be getting better day by day. As long as it is receiving the feedback.
10:41
And finally, that model is being mapped and pushed to the OpenStreetMap back. This is the general workflow, and this is the fair architecture currently, so here I will quickly explain what are the problems we tackled, what is the process we followed, so fair has a backend, backend is designed in Django,
11:02
fair also has a frontend, frontend is designed in React, there is another PIF package, which is a Python library, called fair utilities, so fair utilities is designed that contains code for specific running models, training model, running inference on them, pre-processing, post-processing, and feedback code.
11:24
So that is fair backend had just connecting fair utilities, that can be, for example, for developers who want to share their model or try it on their local, they can directly use fair utilities. As a PIF package and run it on a Colab or a Docker containers, so in pre-processing what
11:43
we do is, we get a TMS, we get a tile from TMS, we convert that tile, since it is, we convert that each tile into image, and then we georeference them based on their bounds, so that vector labels we get from OASM will directly
12:01
assign to each tile, since each tile will have their own coordinate, so they will be unique, and we reproject them back to the same CRS, because labels coming from OpenStreetMap is in 432x, 26, the TMS we are getting is in projected coordinate system, so we reproject it back to projected coordinate system, and then we create a training metadata generation,
12:25
what I mean by metadata generation, it could be different upon the base model, so RAMP requires chips binary mask, they have requirements, but YOLO might not, might require another metadata generation, so there is a metadata generation part, and the challenge we tackled here is, first initially it was
12:47
hard for us to exactly match the tiles from the image with the OASM labels that we are getting, because OASM labels we are getting as a vector, we need to convert it to the raster, so that both will be on the raster and supplied to the model. But this challenge, we overcome that challenge with the help of the bounding box, with the help of the unique tile ID, so that
13:09
each, like we will first figure out where the OASM features are coming from, assign it to the exact tile ID that was for image. And yes, there are base models, base models are connected to fair utilities,
13:23
currently RAMP is supported, building is out, right now we are working on building, and then later on, post processing part, so there was also quite a challenge for us, because our model was making prediction on mask, we were getting a mask, but our goal is to map it in a way, that means we need a geometry out
13:40
of it, so we need to merge all the rasters to a single raster, so that we can extract a vector from there, and then we run our regularization, so we tried to follow a different approach on regularization, so first approach was, we are using safely geometry, safely has a feature, safely can generalize features, we are trying safely before,
14:04
we also tried Geopandas, but there is one cool generalization technique that Josum uses, if you have mapped in OASM, if you trace Q feature, it will generalize the geometry automatically for you, and then we were also looking on it, and we got inspired from it, and then we used the Geopandas and safely libraries in order to adapt with it, it is still not perfect, but we are working on it.
14:28
So yes, all of those are exposed through the API, that API can be connected to the front end, currently connected to the front end, but we are looking at third-party extension, meaning, let's say it can be directly used through Josum, or the tasking manager, and ID, which is our further future use cases.
14:46
So yeah, this is the imagery license that I was talking about, it should be open, it is CC by 4.0, also used same by OpenADL map, and whatever image is coming to fair, it should be licensed to this. And this is, you might wonder, let's say, the imagery services we use in
15:05
OASM editor, Bing, Google, Maxshar, why we cannot use them is because of this license, if we get TMS with this license from any imagery service provider, we will be able to use it. So yeah, as I mentioned, this is our intention for future.
15:24
We want to work on need of the communities, later on expand, let's say from buildings to pond, land use, whatever a community is in need. So currently we are working on feedback loop, to try to finalize it, and conflation, so that predicted feature does not overlap with the existing OASM features.
15:44
And then we also work in open, we want to find other customization, other processes, as long as the process is transparent, we believe in research, this is also a point of collaboration with other researchers and machine learning enthusiastic, and then we are also looking to mitigate biases later on.
16:04
So yeah, this is the, I wanted to attach a screenshot from this area, because we created a model on this prison area, you shout out to Loric, who flew a drone over the prison, and he captured this imagery around the conference area, and uploaded it to the open data map,
16:24
so we created a data set, and then we did a manual mapping, kudos to Nirav, he did a manual mapping, and then we were able to find, this is the area, this is the area we did a manual mapping on, we created training labels, and then later on we used that training,
16:44
and I did a multiple attempt, as you can see, initially it was on 378 image, I moved to 1029, and then again I moved to 255 to figure out the best model on that area, and then finally I got a good working, with good accuracy 93, which is 2
17:02
.4, so FAIR also works in open, meaning you can see all the terminal working on, when we create a model, we will see the terminal, what is what FAIR instance is doing on, and what's going on, also there is a folder structure, directly exposed to API, so we can download all the images that were used during the training process,
17:24
whatever model created or process created during the process, and there is also a graph. So yes, this is the result we got, and then we created a model, we were able to, as you can see on that GIF, we were able to detect the buildings on the rest of the area,
17:41
and it took us around 15 seconds to predict, yes, there it comes, so this is the prediction that we created a model just yesterday, and then it was, we were able to bring it back to Josham, and then push it back to OSM. So, yeah, another thing I quickly want to touch about, we were working on,
18:03
which is feedback, as I mentioned before, if Mapple does not like a prediction, he will make a feedback, there is a manager view, manager can validate each feedback, so that he decides which feedback needs to be feedback to the model, like implemented back to the model, and that's us, thank you, I also get involved, on the left Omran, Omran is an AI
18:26
engineer, this is me on the, and after that Sina, you can also reach out to Sina, and there is our GitHub repo, any person from heart OSM you find, you quickly grab him and ask about FAIR, we are here, thank you.