EOX: Adding static type hints to fiona
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66341 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Type theoryStatic random-access memory
00:08
Service (economics)Static random-access memoryType theorySteady state (chemistry)Group actionConfiguration spaceError messageRun time (program lifecycle phase)Mathematical analysisComputer fileKey (cryptography)Information overloadCommunications protocolType theoryCodeKey (cryptography)Error messageStatic random-access memoryPolygonStress (mechanics)Point (geometry)MereologyRevision controlComputer fileObject (grammar)Social classElectronic data processingMathematical analysisExpert systemShared memoryInterface (computing)Visualization (computer graphics)GeometrySatelliteInformation overloadSoftware testingRight angleBitRun time (program lifecycle phase)Row (database)Electronic mailing listArrow of timeLevel (video gaming)Line (geometry)Machine visionSuite (music)TrailNumberTupleCommunications protocolComputer animation
Transcript: English(auto-generated)
00:08
Hi again, welcome to my talk adding static type hints to Fiona. My name is Stefan. I work for UX. We deal with satellite data processing and satellite data visualization.
00:22
The company is based in Vienna, Austria. We use Fiona in data import pipelines, data export pipelines. Static type annotations for Python were introduced in Python version 3.5, which was released eight years ago. The type annotations gained widespread traction in the Python ecosystem, but not so much yet
00:46
in the geospatial part of the Python ecosystem. So the idea is to find errors in your Python code before actually running the code. So by static analysis and comparing the types that you use in the code, you are
01:03
notified about errors that might occur when you run the code later. A year ago, discussion started in the Fiona issue track about adding type annotations to the Fiona code base. The vision was to add type hints as sidecar files, so-called type stub files, to the
01:24
stable version, the current stable version 1.9 of Fiona, and then later inline these type annotations into the code base in version 2.0. I would like to note that this does not change any runtime behavior of the code,
01:40
so this is purely for static analysis before running the code itself. The approach of adding type annotations to the Fiona code base was first to use a tool that auto-generates static types from the code base that is already there. There are some tools that do static analysis, but we found one tool called
02:03
market type by Instagram, which does runtime analysis of the code. So it runs the code, it compares all the types, all the objects that are used in the Python code, and then generates type annotation files from that. And it was much more reliable than the static analysis that other tools do.
02:20
We use the pytest suite for that, so market type runs the tests of Fiona and then finds out the object types that are used in the code. We configured mypy to actually use these type annotation files, and mypy found
02:41
about 100 errors that could not be inferred by market type itself. So we went through the whole list manually and fixed those errors and added types here and there that market type could not infer. And the goal was to add stricter types to the public API of Fiona,
03:00
for example, the write records method. And this is where we're currently a bit stuck, because there's one problem in the write records, and the public API accepts objects that are like a geoJSON or that convert to the geo interface, done the methods of Python objects.
03:21
We tried two approaches, one was to use the type Dict class, but a geoJSON allows any keys inside of the object. So there has to be a type key, which could be a point or could be polygon, and then there has to be a coordinates key for a point.
03:41
The coordinates value is always a tuple of two numbers, for example. But the type Dict class in Python does not allow extra keys, so we cannot add any keys, for example, that a geoJSON might have. The second approach was to use a protocol class
04:00
with overloads for the get item done the methods. So we tried to mimic a dictionary in Python. We found that it cannot overload literal types. So I cannot make an overload for the type point
04:22
and then have another value for the type polygon. We could not make a fix for that. So this is where we're currently stuck, and if you have any ideas for other approaches, I would be very interested in that, so you can approach me after the session or just write in the GitHub issue tracker of Fiona and share your ideas, please.
04:44
Thank you for listening.