We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

emissions API

00:00

Formal Metadata

Title
emissions API
Subtitle
Or how to get engaged in Public Interest Tech
Alternative Title
emissions API: a service to easily access air quality data from remote sensing
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
The European Space Agency’s Sentinel-5P satellite is built to monitor air quality data (carbon hydroxide, sulfur monoxide, ozone, …). All data gathered are publicly available …if you know what to do with those data sets, great, but if not: Emissions API’s mission is to provide easy access to this data without the need of being an expert in satellite data analysis and without having to process terabytes of data. This way, we hope to empower others to easily build apps that use this data – e.g. visually showing emissions of countries over time. Achievements of climate goals are so far only verifiable for a very small group of people with specialized know-how. As a result, public discussion remains abstract and elusive for many people. Easy access to emissions data provides a more general audience with the opportunity to form a fact-based opinion. For example, one could evaluate the effectiveness of environmental regulations – such as diesel driving bans in inner cities or new sulfur limits in shipping–by comparing actual measurements from before and after on a map. Emissions API is a solution that provides simple access to emissions data of climate-relevant gases. For this purpose, data of the European Space Agency’s Sentinel-5P earth observation satellite will be prepared in such a way that it allows programmers easy access without the need to have a scientific background in the field. The project strives to create an application interface which lowers the barrier to use the data for visualization and/or analysis. Tackling the problem The project’s core is an API, which can be used to query the processed data. For this purpose, we develop a cloud service which queries the freely accessible data of Sentinel-5P, aggregates it, stores it in a cache and makes it available. Target audience This project targets developers who want to build their own services based on the satellite data of the Copernicus program, but who do not want to work with huge amounts of scientific data directly. We will provide examples and libraries to quickly get you started without being an expert in satellite data analysis.
33
35
Thumbnail
23:38
52
Thumbnail
30:38
53
Thumbnail
16:18
65
71
Thumbnail
14:24
72
Thumbnail
18:02
75
Thumbnail
19:35
101
Thumbnail
12:59
106
123
Thumbnail
25:58
146
Thumbnail
47:36
157
Thumbnail
51:32
166
172
Thumbnail
22:49
182
Thumbnail
25:44
186
Thumbnail
40:18
190
195
225
Thumbnail
23:41
273
281
284
Thumbnail
09:08
285
289
Thumbnail
26:03
290
297
Thumbnail
19:29
328
Thumbnail
24:11
379
Thumbnail
20:10
385
Thumbnail
28:37
393
Thumbnail
09:10
430
438
Suspension (chemistry)Touch typingPresentation of a groupProjective planeBitType theorySystem callVoting1 (number)Software developerComputer programmingComputer animation
Euclidean vectorPrototypeArithmetic meanSoftwareSystem callOpen sourceGroup actionGoodness of fitSoftware frameworkPrototypeBitProjective planeComputer programmingTouch typingSoftware developerHypermediaCausalityProcess (computing)Computer animation
MeasurementVisualization (computer graphics)Computer programmingSound effectProduct (business)Open setConcentricMatter waveNetwork topology
Musical ensembleVisualization (computer graphics)TrailInformationOrder (biology)Goodness of fitReduction of orderSelf-organizationComputer animation
Open setComputer fileOpen setMathematical analysisProjective planeService (economics)Set (mathematics)BitSpacetimeBinary codeProcess (computing)Constraint (mathematics)Spectrum (functional analysis)State of matter1 (number)Electronic data processingLevel (video gaming)BelegleserState observerVotingLattice (order)WebsiteView (database)Musical ensembleComputer animation
BelegleserOrbitComputer fileDisk read-and-write headComputer animation
Line (geometry)MeasurementCASE <Informatik>Pole (complex analysis)Office suitePoint cloudComputer animation
MeasurementMultiplication signPairwise comparisonCASE <Informatik>WindowLine (geometry)Computer architectureProcess (computing)Uniform resource locatorComputer animation
ArchitectureInterface (computing)State of matterPrototypeService (economics)Home pageVisualization (computer graphics)DatabaseLibrary (computing)Representational state transferPrototypeState of matterSpecial unitary groupLevel (video gaming)Lie groupOffice suiteProduct (business)Interface (computing)Multiplication signComputer animation
Query languageAveragePolygonVisualization (computer graphics)MeasurementFile formatPairwise comparisonNegative numberForm (programming)Diagram
Different (Kate Ryan album)AreaPoint (geometry)Sound effectMachine visionPhysical systemComputer fileSubject indexingComputer animation
Term (mathematics)Product (business)Open setOnline helpDigital signalProjective planeUniverse (mathematics)Subject indexingVisualization (computer graphics)Point (geometry)Product (business)Term (mathematics)Special unitary groupView (database)Computer animation
TwitterProduct (business)TwitterMereologyComputer animation
PrototypeOpen setMereologySpeech synthesisComputer animation
Point cloudFacebookOpen source
Transcript: English(auto-generated)
Okay, so, now we have Timo Ligero at Blockmeyer, who's talking about the Emissions API. Yes, thank you very much. Yeah, thank you for letting me speak here at FOSTAM and present our project, Emissions API.
To add a little suspense to the presentation, I changed the title a little bit, Emissions API or How to Get Engaged in Public Interest Take, because I wanted to take into account a little bit of our origin story, how we got to make this project, because I found it interesting to share it with people.
Afterwards, we also get a little bit into the details of the data and the technical aspects. But first, I want to show you what Public Interest Take is about, because we first got in touch with Public Interest Take
when we heard about a call from the Prototype Fund in Germany. That is a federally funded program that lets developer fund a small prototype of a project they want to develop, and they propose this framework of Public Interest Take.
It is basically a very broad and simple to understand concept that you develop technology that serves the public good. So it adds a little bit to this FOSTA idea that software is not only open source and legal, but it also has some kind of social meaning for the society. And we, as developers, as a small group of colleagues,
we found that very interesting, and we were immediately hooked by this idea, because we always found that, well, it's nice that we know technical stuff because of our day jobs, but somehow we would like to also put it in use to solve problems that maybe have a meaning deeper than,
or that it goes further along than just the technical aspects, but more solves problems of society as well. And so we really liked this idea, and we said, well, we just want to apply to this call, to this funding call,
and we're gonna pitch an idea. And the first ideas we had was, well, have you heard about the satellite Sentinel-5-P that has emissions data? And we had heard of it, and we thought, well, that's really cool. There's open data about emissions.
We want to build some cool tools with it where you can maybe analyze policy effects on emissions and maybe build some cool visualizations. And so the Sentinel-5 satellite, they provide a lot of data products for emissions.
It's a program from the ESA, from the Copernicus program, and it's basically spectrophotometric measurements where they can analyze the wavelength, and then they can calculate the concentrations
of trace gases and emission data. So we were then, had some ideas what you maybe could do with this. So we were thinking about maybe tracking ships at sea
because there are no sensors, and ships use a lot of heavy fuel oils, and there was supposedly a reduction in international ban on heavy fuel oils to reduce sulfur dioxide emissions, and we thought, oh, maybe we can track that,
or think of maybe diesel bans in city centers, and we could just verify how policies would influence actual emissions, only that we very quickly realized that it's not that easy.
You can just take this data and analyze it easily, and then put it maybe in some visualization. So we thought, well, this is open, but it's not really easy to access. So the good thing is that it's there. So this is really awesome. We can access this data,
and we also don't have to do all this fancy analysis of the spectral analysis. So the ESA already gives that to us, and it's pre-processed data, and the problem that we realized is that it's packed in binary data sets that are not really too easy to employ in a program
or in an interactive visualization, for example. So these are NC files. It's a scientific data format, and when you filter this data, you get chunks of files that represent one flyover of the whole satellite.
So if you are interested in a region, you also have to do a lot of processing from the data that you already pre-filtered. It's pretty large files, and also, generally, there's a lot of data processing involved beforehand. So we took a step back, and that was actually when Emissions API was born. So when we came up with this idea,
we need public infrastructure for open data. We need an easy access to this satellite space emission data, and we wanted to build an infrastructure service that provides this data, that takes it from the ESA, does some pre-processing, and gives it to a user who wants to use this data
in a more easily to employ format. Also, we saw ourselves a little bit as a data literacy project because satellite data is not, I'll come to that later, but there are a lot of peculiarities that maybe you would not expect
when you just think of a satellite. You think, well, there's a lot of data around. It's pretty dense, and you can maybe make some nice visualizations, but there are also some constraints that are important to know. Yeah, so I would like to take a dive into the more technical aspects on the data. How does this work?
The satellite actually flies over the Earth and produces scan lines, so you can think of it maybe as a flatbed scanner for the Earth, and you actually get, so you get a continuous picture of the Earth around the orbit of the satellite,
and when you get one of these data sets, one of these files, and you just plot the data, it's something like this. So here you can already see some general aspects of this data. First of all, we filled it for Germany here,
but you get a lot of Africa of the Antarctic as well, and you get nothing, for example, of the North Pole. This is simply due to the fact that the satellites are based, so the measurements are based on light, and there's no active light source on the satellite, so it needs to have sunlight to get data,
so you will never get data at night, for example. Also, you can maybe see that there are some holes or that it's not as smooth as you would expect, the line in general. Everywhere where there's clouds, you don't have data,
and this is just some of the things. Now that when you think of how the satellite flies over the Earth, you don't have a measurement for each time of day in every place, but you actually get this line that moves over Germany, in this case here, for example,
and you see where you have the measurements. For Germany, it's a time window of maybe, let's say, 10 minutes where the satellite is passing Germany, and you just get data for that time, that location, so when you want to, for example, measure emissions during rush hour,
and the satellite is not there during that time, you simply don't have data about that, so you cannot have comparisons over one day, for example. These are some of the important things to consider about the data. Now I'm gonna switch to the technical side
and our architecture. What we did, we download the data from ESA, we pre-process it in a way that we can use it more easily, we cache it in a way, or we save it in a Postgres database, and we will provide a service as a REST interface
where you can query by region, by time, and you get a JSON in return that you can usually, pretty easily, put into one of your usual JavaScript frameworks, for example, for visualization.
I have to say that we are doing this since September, and we are now in a state where it kind of works. You can check out our homepage and our UI. We have a working prototype. At the moment, we only have the carbon monoxide data in our database because it is what we started with,
because it was easy to get going with it, and we still need to add some more of the other products. We have a lot of examples already on our homepage where you can see how to use the data, how maybe to make visualizations with this data that we have, and of course, we also needed to develop some tools around this service
and one of them, the Sentinel-5-DL, is a download library that is on PyPI where you can filter and download Sentinel-5 data automatically. It already works quite well, so if you just want to tinker around yourself, you can also use some of our libraries.
For some examples of how the query would work, so here you have a cURL command with a query of our API. You can filter by country or you can also put your own polygon into it and by days,
and you get a JSON format with the values. You can then plot this. This is, for example, just February in Germany, February 2019, and the average for each day. So you can, like this, you can easily make comparisons,
for example, between measurement days. Then you can plug it into your favorite visualization. Here, we did an example with Dec-GL that just kind of looks cool
because you have the 3D effect and you see the different areas in Germany and their visions at that point when the flyover was taking place. We still have a lot of challenges to resolve. We realized during the process, obviously,
that this is way too much data. We somehow need to reduce it. We are currently playing around with geospatial indexing systems, but maybe somebody else has a better idea how to do it. We also need a long-term host. Currently, we got some credit from Amazon, from AWS,
and we also have some universities that are interested in hosting our project, but we still need a solution that has enough power for long-term as well. And of course, we need to import more emission data, more of the other products from the satellite.
Yeah, so this is our story of getting involved in public interest take. For us, from that point of view, the real interesting takeaway was that there is a lot of interesting data out there, but usually, it's just very difficult to access
and we probably need a lot more people to get working on this infrastructure to build, actually, infrastructure that comes before building a tool or a visualization. And so the easiest way, if you want to get started in this
is that you just get started with our project and have us developing or use our product. First off, you would go to emissionsapi.org, check the examples, check what we are doing. You can find us on GitHub, obviously, and hit us up on Twitter to let us know what we are missing
and what you would like to have in this product. I want to thank also our sponsors, Protype Fund, that is part of the Open Knowledge Foundation in Germany, and all of this is sponsored in the end by the German Ministry of Education and Research.
Thank you very much.