emissions API
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47002 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 202054 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Suspension (chemistry)Touch typingPresentation of a groupProjective planeBitType theorySystem callVoting1 (number)Software developerComputer programmingComputer animation
00:54
Euclidean vectorPrototypeArithmetic meanSoftwareSystem callOpen sourceGroup actionGoodness of fitSoftware frameworkPrototypeBitProjective planeComputer programmingTouch typingSoftware developerHypermediaCausalityProcess (computing)Computer animation
02:28
MeasurementVisualization (computer graphics)Computer programmingSound effectProduct (business)Open setConcentricMatter waveNetwork topology
03:35
Musical ensembleVisualization (computer graphics)TrailInformationOrder (biology)Goodness of fitReduction of orderSelf-organizationComputer animation
04:29
Open setComputer fileOpen setMathematical analysisProjective planeService (economics)Set (mathematics)BitSpacetimeBinary codeProcess (computing)Constraint (mathematics)Spectrum (functional analysis)State of matter1 (number)Electronic data processingLevel (video gaming)BelegleserState observerVotingLattice (order)WebsiteView (database)Musical ensembleComputer animation
06:38
BelegleserOrbitComputer fileDisk read-and-write headComputer animation
07:07
Line (geometry)MeasurementCASE <Informatik>Pole (complex analysis)Office suitePoint cloudComputer animation
08:05
MeasurementMultiplication signPairwise comparisonCASE <Informatik>WindowLine (geometry)Computer architectureProcess (computing)Uniform resource locatorComputer animation
09:00
ArchitectureInterface (computing)State of matterPrototypeService (economics)Home pageVisualization (computer graphics)DatabaseLibrary (computing)Representational state transferPrototypeState of matterSpecial unitary groupLevel (video gaming)Lie groupOffice suiteProduct (business)Interface (computing)Multiplication signComputer animation
10:45
Query languageAveragePolygonVisualization (computer graphics)MeasurementFile formatPairwise comparisonNegative numberForm (programming)Diagram
11:32
Different (Kate Ryan album)AreaPoint (geometry)Sound effectMachine visionPhysical systemComputer fileSubject indexingComputer animation
11:55
Term (mathematics)Product (business)Open setOnline helpDigital signalProjective planeUniverse (mathematics)Subject indexingVisualization (computer graphics)Point (geometry)Product (business)Term (mathematics)Special unitary groupView (database)Computer animation
13:21
TwitterProduct (business)TwitterMereologyComputer animation
13:46
PrototypeOpen setMereologySpeech synthesisComputer animation
14:09
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:07
Okay, so, now we have Timo Ligero at Blockmeyer, who's talking about the Emissions API. Yes, thank you very much. Yeah, thank you for letting me speak here at FOSTAM and present our project, Emissions API.
00:23
To add a little suspense to the presentation, I changed the title a little bit, Emissions API or How to Get Engaged in Public Interest Take, because I wanted to take into account a little bit of our origin story, how we got to make this project, because I found it interesting to share it with people.
00:44
Afterwards, we also get a little bit into the details of the data and the technical aspects. But first, I want to show you what Public Interest Take is about, because we first got in touch with Public Interest Take
01:00
when we heard about a call from the Prototype Fund in Germany. That is a federally funded program that lets developer fund a small prototype of a project they want to develop, and they propose this framework of Public Interest Take.
01:22
It is basically a very broad and simple to understand concept that you develop technology that serves the public good. So it adds a little bit to this FOSTA idea that software is not only open source and legal, but it also has some kind of social meaning for the society. And we, as developers, as a small group of colleagues,
01:44
we found that very interesting, and we were immediately hooked by this idea, because we always found that, well, it's nice that we know technical stuff because of our day jobs, but somehow we would like to also put it in use to solve problems that maybe have a meaning deeper than,
02:04
or that it goes further along than just the technical aspects, but more solves problems of society as well. And so we really liked this idea, and we said, well, we just want to apply to this call, to this funding call,
02:21
and we're gonna pitch an idea. And the first ideas we had was, well, have you heard about the satellite Sentinel-5-P that has emissions data? And we had heard of it, and we thought, well, that's really cool. There's open data about emissions.
02:41
We want to build some cool tools with it where you can maybe analyze policy effects on emissions and maybe build some cool visualizations. And so the Sentinel-5 satellite, they provide a lot of data products for emissions.
03:03
It's a program from the ESA, from the Copernicus program, and it's basically spectrophotometric measurements where they can analyze the wavelength, and then they can calculate the concentrations
03:24
of trace gases and emission data. So we were then, had some ideas what you maybe could do with this. So we were thinking about maybe tracking ships at sea
03:44
because there are no sensors, and ships use a lot of heavy fuel oils, and there was supposedly a reduction in international ban on heavy fuel oils to reduce sulfur dioxide emissions, and we thought, oh, maybe we can track that,
04:03
or think of maybe diesel bans in city centers, and we could just verify how policies would influence actual emissions, only that we very quickly realized that it's not that easy.
04:21
You can just take this data and analyze it easily, and then put it maybe in some visualization. So we thought, well, this is open, but it's not really easy to access. So the good thing is that it's there. So this is really awesome. We can access this data,
04:42
and we also don't have to do all this fancy analysis of the spectral analysis. So the ESA already gives that to us, and it's pre-processed data, and the problem that we realized is that it's packed in binary data sets that are not really too easy to employ in a program
05:04
or in an interactive visualization, for example. So these are NC files. It's a scientific data format, and when you filter this data, you get chunks of files that represent one flyover of the whole satellite.
05:22
So if you are interested in a region, you also have to do a lot of processing from the data that you already pre-filtered. It's pretty large files, and also, generally, there's a lot of data processing involved beforehand. So we took a step back, and that was actually when Emissions API was born. So when we came up with this idea,
05:41
we need public infrastructure for open data. We need an easy access to this satellite space emission data, and we wanted to build an infrastructure service that provides this data, that takes it from the ESA, does some pre-processing, and gives it to a user who wants to use this data
06:00
in a more easily to employ format. Also, we saw ourselves a little bit as a data literacy project because satellite data is not, I'll come to that later, but there are a lot of peculiarities that maybe you would not expect
06:20
when you just think of a satellite. You think, well, there's a lot of data around. It's pretty dense, and you can maybe make some nice visualizations, but there are also some constraints that are important to know. Yeah, so I would like to take a dive into the more technical aspects on the data. How does this work?
06:40
The satellite actually flies over the Earth and produces scan lines, so you can think of it maybe as a flatbed scanner for the Earth, and you actually get, so you get a continuous picture of the Earth around the orbit of the satellite,
07:03
and when you get one of these data sets, one of these files, and you just plot the data, it's something like this. So here you can already see some general aspects of this data. First of all, we filled it for Germany here,
07:20
but you get a lot of Africa of the Antarctic as well, and you get nothing, for example, of the North Pole. This is simply due to the fact that the satellites are based, so the measurements are based on light, and there's no active light source on the satellite, so it needs to have sunlight to get data,
07:44
so you will never get data at night, for example. Also, you can maybe see that there are some holes or that it's not as smooth as you would expect, the line in general. Everywhere where there's clouds, you don't have data,
08:01
and this is just some of the things. Now that when you think of how the satellite flies over the Earth, you don't have a measurement for each time of day in every place, but you actually get this line that moves over Germany, in this case here, for example,
08:21
and you see where you have the measurements. For Germany, it's a time window of maybe, let's say, 10 minutes where the satellite is passing Germany, and you just get data for that time, that location, so when you want to, for example, measure emissions during rush hour,
08:40
and the satellite is not there during that time, you simply don't have data about that, so you cannot have comparisons over one day, for example. These are some of the important things to consider about the data. Now I'm gonna switch to the technical side
09:01
and our architecture. What we did, we download the data from ESA, we pre-process it in a way that we can use it more easily, we cache it in a way, or we save it in a Postgres database, and we will provide a service as a REST interface
09:22
where you can query by region, by time, and you get a JSON in return that you can usually, pretty easily, put into one of your usual JavaScript frameworks, for example, for visualization.
09:41
I have to say that we are doing this since September, and we are now in a state where it kind of works. You can check out our homepage and our UI. We have a working prototype. At the moment, we only have the carbon monoxide data in our database because it is what we started with,
10:02
because it was easy to get going with it, and we still need to add some more of the other products. We have a lot of examples already on our homepage where you can see how to use the data, how maybe to make visualizations with this data that we have, and of course, we also needed to develop some tools around this service
10:22
and one of them, the Sentinel-5-DL, is a download library that is on PyPI where you can filter and download Sentinel-5 data automatically. It already works quite well, so if you just want to tinker around yourself, you can also use some of our libraries.
10:45
For some examples of how the query would work, so here you have a cURL command with a query of our API. You can filter by country or you can also put your own polygon into it and by days,
11:04
and you get a JSON format with the values. You can then plot this. This is, for example, just February in Germany, February 2019, and the average for each day. So you can, like this, you can easily make comparisons,
11:26
for example, between measurement days. Then you can plug it into your favorite visualization. Here, we did an example with Dec-GL that just kind of looks cool
11:40
because you have the 3D effect and you see the different areas in Germany and their visions at that point when the flyover was taking place. We still have a lot of challenges to resolve. We realized during the process, obviously,
12:01
that this is way too much data. We somehow need to reduce it. We are currently playing around with geospatial indexing systems, but maybe somebody else has a better idea how to do it. We also need a long-term host. Currently, we got some credit from Amazon, from AWS,
12:21
and we also have some universities that are interested in hosting our project, but we still need a solution that has enough power for long-term as well. And of course, we need to import more emission data, more of the other products from the satellite.
12:40
Yeah, so this is our story of getting involved in public interest take. For us, from that point of view, the real interesting takeaway was that there is a lot of interesting data out there, but usually, it's just very difficult to access
13:00
and we probably need a lot more people to get working on this infrastructure to build, actually, infrastructure that comes before building a tool or a visualization. And so the easiest way, if you want to get started in this
13:20
is that you just get started with our project and have us developing or use our product. First off, you would go to emissionsapi.org, check the examples, check what we are doing. You can find us on GitHub, obviously, and hit us up on Twitter to let us know what we are missing
13:40
and what you would like to have in this product. I want to thank also our sponsors, Protype Fund, that is part of the Open Knowledge Foundation in Germany, and all of this is sponsored in the end by the German Ministry of Education and Research.
14:00
Thank you very much.