Evolving JSTS to a modern port of JTS using AST transformation

48 views

Formal Metadata

Title
Evolving JSTS to a modern port of JTS using AST transformation
Title of Series
Part Number
40
Number of Parts
193
Author
Harrtell, Björn
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI
Publisher
FOSS4G, Open Source Geospatial Foundation (OSGeo)
Release Date
2016
Language
English

Content Metadata

Subject Area
Abstract
JTS is a well known Java library of spatial predicates and functions for processing geometry. JSTS has since 2011 provided a port of JTS to JavaScript that provides that proven library for use in browsers and other popular JavaScript runtimes. It brings the functionality of JTS to the browser without having to make a roundtrip to a server which enables advanced spatial operations to web applications even in offline conditions and is, for example, used by MapBox as a significant part of turf.js. The initial effort to create JSTS was a long, arduous and manual process and has since lagged behind JTS missing crucial new bug fixes and features. Resorting to additional manual porting effort was not in anyone's interest, it seems. Instead there was dreams of an automated translation from Java to JavaScript. In this talk I will explain how I managed to automatically translate JTS to create a new version of JSTS that supersedes the old manual port with a more complete, correct and up to date implementation.
Keywords
Sweco Position AB

Related Material

Loading...
Code Java applet Multiplication sign View (database) Source code Mereology Information technology consulting Programmer (hardware) Medical imaging Meeting/Interview Personal digital assistant Core dump Library (computing) Position operator Social class God Algorithm Product (category theory) File format Geometry Computer file Constructor (object-oriented programming) Open source Bit Variable (mathematics) Functional (mathematics) Position operator Process (computing) Programmer (hardware) Data storage device Mathematical singularity output Quantum Software testing Quicksort Row (database) Web page Topology Implementation Computer file Open source Algorithm Line (geometry) Mass Field (computer science) Bridging (networking) Software Scripting language Software testing Implementation Computer-assisted translation Posterior probability Multiplication Suite (music) Scripting language Projective plane Polygon Java applet Code Total S.A. Line (geometry) Cartesian coordinate system Computer animation Software Personal digital assistant Revision control Library (computing)
Point (geometry) Beta function Code Java applet Line (geometry) Set (mathematics) Multiplication sign Branch (computer science) Mereology Formal language Revision control Social class Personal digital assistant Authorization Scripting language Implementation Abstraction Social class Stability theory Beta function Standard deviation Theory of relativity Poisson-Klammer Computer file Polygon Projective plane Java applet Electronic mailing list Code Existence Functional (mathematics) Demoscene Syntaxbaum Abstract syntax tree Compiler Google Web Toolkit Process (computing) Computer animation Integrated development environment Network topology Personal digital assistant Software testing Quicksort Writing Library (computing)
Read-only memory Computer file Constructor (object-oriented programming) Java applet Code Transformation (genetics) Translation (relic) Parsing Coordinate system Mereology Operator overloading Formal language Social class Latent heat Representation (politics) Cuboid Scripting language Data structure Abstraction Subtraction Graphical user interface Social class Default (computer science) Process (computing) Scripting language Java applet Line (geometry) Syntaxbaum Abstract syntax tree Process (computing) Computer animation Network topology Integrated development environment Logic Representation (politics) Abstraction Library (computing)
Web page Domain name Presentation of a group Computer file Constructor (object-oriented programming) Code Java applet Ring (mathematics) Geometry Virtual machine Translation (relic) Parameter (computer programming) Coordinate system Operator overloading Revision control Social class Goodness of fit Operator (mathematics) Electronic meeting system Energy level Software testing Traffic reporting Units of measurement Social class Form (programming) Default (computer science) Standard deviation Touchscreen Validity (statistics) Inheritance (object-oriented programming) Constructor (object-oriented programming) Length Sound effect Coma Berenices Bit Functional (mathematics) System call Computer animation Personal digital assistant Function (mathematics) Mixed reality Convex hull Software testing Right angle Systems engineering
Point (geometry) Web page Implementation Wage labour Code Multiplication sign Geometry Sheaf (mathematics) Function (mathematics) Mereology Rule of inference Open set Revision control Mathematics Meeting/Interview Stress (mechanics) Computer configuration Operator (mathematics) Cuboid Energy level Software testing Traffic reporting Macro (computer science) Units of measurement Position operator Social class Physical system Default (computer science) Limit (category theory) Software maintenance Functional (mathematics) Local Group Computer animation Network topology Personal digital assistant Buffer solution Vertex (graph theory) output Absolute value Resultant Library (computing)
so it so the lower 1 would say
that this is the talk about that is which is a part of the job multiple-use sweet to and
so a little bit about me my name is pure the hospital and I come from Sweden from a town called mount if you don't know that on it's totality that's just beside Copenhagen over a bridge not yeah I work as a programmer and consultant at Sre?ko position to and I have worked in this field about 10 years so with this is my 1st time at 1st Jesus on happens to be here and I look forward to the rest of this conference the and yes the job multiple View sweets many probably know here this is a Java library implementing various algorithms it can for example calculate intersections of polygons and from this image at an old web page for some reason there is a lot of diastase old web pages if it dates back to 2002 I believe and has been used in open source GIS software like quantum GIS posterior and you will see and so I have to no I don't have to but I want to give them due credits to this guy here this is Martin dairies in he founded the judges projects and I believe he received this all cats award in 2011 he is the founder of the data's products and so on this job of sub library has been ported to C + + and it's called you used in past years also it has been ported to Python and C sharp and with my project J is it's being ported to Java scripts a little bit of history of my projects I started in 2011 1 so it's sort of getting a hold back and back then it was sort manual effort and my reason to to do it was mostly out of just curiosity curiosity it it was possible I didn't have a goal to create a feature completes poets and I didn't look at the original source in any depth I just started to try to translate so uh and some of the core part to see if it was how it would do go from there so this small beginnings so all of this ports stars and look at the class called coordinates which of course is so central classic to in a unitary library it has a constructor looking like that it actually has 4 constructors but this is 1 of them God so it's pretty pretty easy to understand this is just text the axis and stores them mass member variables the and so the translate this to just rip was really easy side and this is designed to be easy so and is translated into that's this is in the 1st Committee in 2011 so it's almost the same as the function simulating a constructive I ignore this z axis until later in the project on yeah I'll sing in ignore that the rows for constructing I reached a working implementation also in 2011 in August and by that time I had Port said 158 files and 30 thousand lines of code and if I had known that that was required just to get it basically working and I don't think I would have started doing this but by didn't realize that until I was halfway and so on I can't give up now that and I was able to also report the test cases from In this turned out to be quite easy because injured the test cases most of them are specified in the XML format so have to put in a code I just have to pass the XML taking inputs and verified against expected up so I got over 400 test cases to verify that my porch was going in the right way and this was absolutely critical to preceding the project and so there are about
halfway into the point I was getting tired of rewriting code when I was doing things like this the job our collection classes right like array list I translated into brackets which is I just scriptory and it looks needs but it also requires rewriting the code so I realize that too to not have to do this much work it's better to port that collection classes supported array list and some other once and also that's the rest of the ports accelerated and it was used in many cases this was mostly a matter of searching replace actually so it became the nearest to nearest but I was halfway into the project so and and I didn't want to redo the old parts that I we wrote to monitoring and this turned out to be problematic and in the end some features were which never working in the old days just like cascaded polygon union which is so if you wanted to solve a lot of polygons it's a good cooperation and so on this year I have released the version 1 . 0 and the road to that was starting in 2015 the old man aboard I haven't done anything with it adjusts stayed where it was some small world fixes perhaps and it starts to lag behind the upstream Java code and share I wanted to update it but I wouldn't wanna go back to this manual boring work so in made 2015 an issue was opened uh called automated support at the time and I was able to to get the interest of uh march in Derry the original authors he the we had a lot of discussions about home what was needed to do this work and the encouragement uh it was very good to to having this discussion that motivated me to keep trying to do this and I made rapid rapid progress in early 2016 a 1st beta release in January and the stable release in February so yes so how was this done 1st started to look at some existing tools and the most known JavaScript Java to JavaScript compiler is probably Google Web Toolkit and also called I don't know how to pronounce in english and relations but it's it can translate in almost any job according to John true but the problem is that it's not supposed to create JavaScript that it's useful for other dollar strips you can export some of it but it doesn't translate the API so it's obfuscators the code so you have to write everything in Java and this is not what I wanted injustice I wanted to create a library that can use you use from other JavaScript code I looked at some other alternatives but in the end and I couldn't find anything of worked yeah but also in 2015 I started to learn a lot about the new JavaScript standard called 6 or a conscript 2015 is however my naming news strange naming that's silly if you have year 6 and aggressive that and anyway I sort of write code in this new standard because it was possible to transport it down to to JavaScript that worked in all branches and I got curious curious about how this was possible it that was possible using a method called called transpiration transform the new language into their own language and this is us using something behind the scenes called up abstract syntax trees and they have actually been existing for a long time there are out there are other parts of any compiler and they are also used in integrated development environments like eclipse to to create functions like autocompletion and center checking so for free or script there's actually a defective standard called ES tree specified by by accident
and it says represented in pure Jason and when I learned about this stuff I started to think it's perhaps it is possible to to to create
this transformation automatically without having to be Eagle so but but for that to be possible I had to find something good enough to pass the Java code and after some search I realize that have the best 1 and was 2 years a pasture inside an integrated development environment like eclipse so looked at this part of eclipse called GATT and it's perhaps not so in normal use of it but it can be used as kind of a library when developing code without graphical user interface to like read a Java code file and and get a syntax tree out of it town so this resulted in a tool called job to tree that wrote is written in Scala probably my favorite language so if there's some scholar haters here I want to talk to you later selling sky was able to 0 define the E Street specification in 300 lines of code because this is so that I wanted that to be the target so I needed to define the East respect come so that's that's what I had in memory how structure and then I could use the job library Jackson to serialize outages quite simple but but what was not simple the translation logic and from the Eclipse Java 8 abstracts industry to the Iast tree representation so that's worse than the big part of the work and then I'm out from my tool out Jason and I have to get to jail for that there is actually several tools available so I use 1 called a string do and yet a In the process of doing that their translation logic there were some difficulties at 1st I thought he has 6 would be a good target because it has a class keyword and a class that they're the classes that looks like Java classes and I actually got it working at 1st but I also had some hard to fix a box and I had to start writing this so ugly work-arounds and it turns out that the classes in job 1 year 6 are not the same there have some small differences that on very good to to assume that they were at the same and this disposed down the overloading supporting job that is not existing in Dell script OK so for the final parts of representation we're going to dive into the generated to coordinate class that we looked at before and this is my 1st version and beta 1 so that the class so that the to see
the markings but because some of that that screen in the class keyword used is in their new joseph standard and it has a constructive and then it has all this crap that we don't want to look at too much because it doesn't look good it's very necessary in light of inter functions so instead let's take up take a look at that I managed to improve its and to this it fits somehow presentation page and it's still there is still used in class keyword and managed to reduce the levels of In functions but but I still have is of see In Europe function called overloaded why happen because I needed to simulate these 4 constructors of coronet class and in fact conscripts is 6 classes it's not allowed
to the construct cannot of call itself that's not allowed so I needed to create a function in the constructed colors but and that worked for for this case but sooner I encountered become more complex cases with derived class that needed to call the super class constructor and stuff like that and then it all fell apart so and the final translation in their in their release candidates you start standard its function and for the coordinates and this code is almost to readable yes and you it it can call itself so if you call them the corners constructor with no arguments it's it checks for that right here then it will initialize itself as according at x y 0 this this is generated code I didn't write it so so also it turned out that before I reached this version of that that the other versions there were a lot slower than the and the old manual ports so it's not until I refined to this to this code that it acted got a bit faster than the old man and so on to reach the stable release after I got to these can is 7 I had only 1 failing tests out of 500 units the and I didn't have an effect on test that's 1 you for a file on the case and it was about that is valid operation that checks for a geometry valid to their lead at the and it's was with the specific geometry and the recent that this failed turned out to to be because in in g test there's a class called doubled its which uses some functions in the Java virtual machine that that works is that manipulation stuff so there's nothing like it in so it didn't work as the translation so in the end I had to port that form manually which yes I have failed then creates an automated ports yeah have to but in almost all domains reports that so and is more complete than the old 1 and faster to such great have does so for me thanks to this thank you the but but but but but but but but but to the course
what OK the question was if there's an example so far this test used in in the wild and I have no nothing to demonstrate right here but you can see a j stress in use at the OpenLayers 3 page and there's an example with this system this is uses the buffering function to buffer road section those who you're here the year if you the other were you may be something similar or on the men get a question was if I know about the other reports if there are using the same methods to achieve those sports and yes I know about them and they do not use that method there are manual labor and I think I have a quote from Martin Davis that turning the Java code into C + + which are much more difficult work than he had expected from a lot of time put put into those think especially the simplest for support and they have the issues so far maintenance keeping it up-to-date just because of the high of all this so I just press these you can use your commentary that the pipe importers have actually reported it goes into the reduced KPI but it's provides a Python API that is not that is more Pythonesque so is a wrapper against get the you have the same operation you know what you want to because you don't thank you reality the question was if I have the same functionality I have in this new version near 100 % of the functionality there I guess there are some parts so that that is not translated correctly it's the parts that has no test cases but and there are a 500 test cases trying out most of the functionality so but I have no guarantee there but is a complete sports the and the presence of the I would is the story of the what is repeat last part we what all yeah because anything if there's some requirements for open there's so and no there's not in at the stars of the manual part I do still players in in in Inderjit dataset but I should I remove that also from the manual part of no it's a free-standing library here I I could mention I I do have an input output class and that can translate the OpenLayers class directly so you don't have to go to an intermediate formants but it's optional feature going what through all the money that you have William the you know and you have the features of all the other rules that are going to be working on the issue of racial groups what used to 2 years that the 1st all what you hold hold on to the political all they have to compare the it it should have the same limitations and the question was if there's a limit limits to their operations implemented in genesis and and as far as I know it should have the same limitations as the original ports and there are the the operational port is advertised as that a robust implementation so so it can handle handle precision issues but if the geometry is not valid or if you are at very small position issues then then you can get a topology and then you have to preprocess your input to around it's not it's but you have to check that your input is valid it has to be OK see valid to for the operations to work if you put if you inputs polygon that is not valid and the result is undefined it can you might was the OK then this might yeah that that probably is a box of the left with the the then the question is do can define what i mean with precision issues the default position is floating points to some decimal point and I remember how so far but but you can the JTS also has precision knowledge that you can switch to a threat to a chosen level of precision and sometimes you have to do that just to you have to round down the gentry into 2 something with less I say if you have vertices that are too close but this is endemic you're talking about macros level stuff we're None of Nazi we don't have units in ingest we're talking about the limits of mathematics such so so it depends on the corner system of course what what is a centimeter but if we're talking about extremely small yeah companies
Loading...
Feedback

Timings

  521 ms - page object

Version

AV-Portal 3.9.2 (c7d7a940c57b22d0bc6d7f70d6f13fde2ef2d4b8)