Logo TIB AV-Portal Logo TIB AV-Portal

Evolving JSTS to a modern port of JTS using AST transformation

Video in TIB AV-Portal: Evolving JSTS to a modern port of JTS using AST transformation

Formal Metadata

Evolving JSTS to a modern port of JTS using AST transformation
Title of Series
Part Number
Number of Parts
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
JTS is a well known Java library of spatial predicates and functions for processing geometry. JSTS has since 2011 provided a port of JTS to JavaScript that provides that proven library for use in browsers and other popular JavaScript runtimes. It brings the functionality of JTS to the browser without having to make a roundtrip to a server which enables advanced spatial operations to web applications even in offline conditions and is, for example, used by MapBox as a significant part of turf.js. The initial effort to create JSTS was a long, arduous and manual process and has since lagged behind JTS missing crucial new bug fixes and features. Resorting to additional manual porting effort was not in anyone's interest, it seems. Instead there was dreams of an automated translation from Java to JavaScript. In this talk I will explain how I managed to automatically translate JTS to create a new version of JSTS that supersedes the old manual port with a more complete, correct and up to date implementation.
Keywords Sweco Position AB

Related Material

programming script code Java time views sources part consultant image geometric Meeting/Interview case core libraries position classes God script algorithm formating files construction Open Source storage bits variables position processes programme singularities Quantum input testing sort record web pages topological functionality implementation Open Source files algorithm lines Mass fields production bridges software testing implementation Cats A-posteriori-Wahrscheinlichkeit Multiple suite projects polygons Java code total lines applications Computer animation software case versions libraries
point functionality beta code Java lines time branch part information versions case sets Authorization implementation Abstract classes stable script beta standards relation bracket files projects polygons Java The list code non-existence Demo expression trees AST compiler Google Web Toolkit processes Computer animation environment case topology testing classes sort write libraries
script files construction Java Transformers code translation coordination part Überladen information specific different memory case Representation box structure Abstract classes default script parsing management Java lines expression trees graphical user interface processes Computer animation environment logic topology classes Representation Abstract libraries
web pages functionality presentation files construction code Java unit machine translation argument coordination Überladen versions goodness operations case Em's ring level testing report classes form default domain standards screens Super validation construction breadth effects coma bits system call Computer animation case functions mix classes convex hull testing Right systems engineering geometric
point web pages functionality Actions implementation labor Java code time unit sheaf 3rd open subsets functions part rules versions mathematics Meeting/Interview configuration operations reduce level box testing macros report position classes systems default traction limitations maintenance Computer animation case topology buffer vertices input absolute values Results geometric libraries
so it so the lower 1 would say
that this is the talk about that is which is a part of the job multiple-use sweet to and
so a little bit about me my name is pure the hospital and I come from Sweden from a town called mount if you don't know that on it's totality that's just beside Copenhagen over a bridge not yeah I work as a programmer and consultant at Sre?ko position to and I have worked in this field about 10 years so with this is my 1st time at 1st Jesus on happens to be here and I look forward to the rest of this conference the and yes the job multiple View sweets many probably know here this is a Java library implementing various algorithms it can for example calculate intersections of polygons and from this image at an old web page for some reason there is a lot of diastase old web pages if it dates back to 2002 I believe and has been used in open source GIS software like quantum GIS posterior and you will see and so I have to no I don't have to but I want to give them due credits to this guy here this is Martin dairies in he founded the judges projects and I believe he received this all cats award in 2011 he is the founder of the data's products and so on this job of sub library has been ported to C + + and it's called you used in past years also it has been ported to Python and C sharp and with my project J is it's being ported to Java scripts a little bit of history of my projects I started in 2011 1 so it's sort of getting a hold back and back then it was sort manual effort and my reason to to do it was mostly out of just curiosity curiosity it it was possible I didn't have a goal to create a feature completes poets and I didn't look at the original source in any depth I just started to try to translate so uh and some of the core part to see if it was how it would do go from there so this small beginnings so all of this ports stars and look at the class called coordinates which of course is so central classic to in a unitary library it has a constructor looking like that it actually has 4 constructors but this is 1 of them God so it's pretty pretty easy to understand this is just text the axis and stores them mass member variables the and so the translate this to just rip was really easy side and this is designed to be easy so and is translated into that's this is in the 1st Committee in 2011 so it's almost the same as the function simulating a constructive I ignore this z axis until later in the project on yeah I'll sing in ignore that the rows for constructing I reached a working implementation also in 2011 in August and by that time I had Port said 158 files and 30 thousand lines of code and if I had known that that was required just to get it basically working and I don't think I would have started doing this but by didn't realize that until I was halfway and so on I can't give up now that and I was able to also report the test cases from In this turned out to be quite easy because injured the test cases most of them are specified in the XML format so have to put in a code I just have to pass the XML taking inputs and verified against expected up so I got over 400 test cases to verify that my porch was going in the right way and this was absolutely critical to preceding the project and so there are about
halfway into the point I was getting tired of rewriting code when I was doing things like this the job our collection classes right like array list I translated into brackets which is I just scriptory and it looks needs but it also requires rewriting the code so I realize that too to not have to do this much work it's better to port that collection classes supported array list and some other once and also that's the rest of the ports accelerated and it was used in many cases this was mostly a matter of searching replace actually so it became the nearest to nearest but I was halfway into the project so and and I didn't want to redo the old parts that I we wrote to monitoring and this turned out to be problematic and in the end some features were which never working in the old days just like cascaded polygon union which is so if you wanted to solve a lot of polygons it's a good cooperation and so on this year I have released the version 1 . 0 and the road to that was starting in 2015 the old man aboard I haven't done anything with it adjusts stayed where it was some small world fixes perhaps and it starts to lag behind the upstream Java code and share I wanted to update it but I wouldn't wanna go back to this manual boring work so in made 2015 an issue was opened uh called automated support at the time and I was able to to get the interest of uh march in Derry the original authors he the we had a lot of discussions about home what was needed to do this work and the encouragement uh it was very good to to having this discussion that motivated me to keep trying to do this and I made rapid rapid progress in early 2016 a 1st beta release in January and the stable release in February so yes so how was this done 1st started to look at some existing tools and the most known JavaScript Java to JavaScript compiler is probably Google Web Toolkit and also called I don't know how to pronounce in english and relations but it's it can translate in almost any job according to John true but the problem is that it's not supposed to create JavaScript that it's useful for other dollar strips you can export some of it but it doesn't translate the API so it's obfuscators the code so you have to write everything in Java and this is not what I wanted injustice I wanted to create a library that can use you use from other JavaScript code I looked at some other alternatives but in the end and I couldn't find anything of worked yeah but also in 2015 I started to learn a lot about the new JavaScript standard called 6 or a conscript 2015 is however my naming news strange naming that's silly if you have year 6 and aggressive that and anyway I sort of write code in this new standard because it was possible to transport it down to to JavaScript that worked in all branches and I got curious curious about how this was possible it that was possible using a method called called transpiration transform the new language into their own language and this is us using something behind the scenes called up abstract syntax trees and they have actually been existing for a long time there are out there are other parts of any compiler and they are also used in integrated development environments like eclipse to to create functions like autocompletion and center checking so for free or script there's actually a defective standard called ES tree specified by by accident
and it says represented in pure Jason and when I learned about this stuff I started to think it's perhaps it is possible to to to create
this transformation automatically without having to be Eagle so but but for that to be possible I had to find something good enough to pass the Java code and after some search I realize that have the best 1 and was 2 years a pasture inside an integrated development environment like eclipse so looked at this part of eclipse called GATT and it's perhaps not so in normal use of it but it can be used as kind of a library when developing code without graphical user interface to like read a Java code file and and get a syntax tree out of it town so this resulted in a tool called job to tree that wrote is written in Scala probably my favorite language so if there's some scholar haters here I want to talk to you later selling sky was able to 0 define the E Street specification in 300 lines of code because this is so that I wanted that to be the target so I needed to define the East respect come so that's that's what I had in memory how structure and then I could use the job library Jackson to serialize outages quite simple but but what was not simple the translation logic and from the Eclipse Java 8 abstracts industry to the Iast tree representation so that's worse than the big part of the work and then I'm out from my tool out Jason and I have to get to jail for that there is actually several tools available so I use 1 called a string do and yet a In the process of doing that their translation logic there were some difficulties at 1st I thought he has 6 would be a good target because it has a class keyword and a class that they're the classes that looks like Java classes and I actually got it working at 1st but I also had some hard to fix a box and I had to start writing this so ugly work-arounds and it turns out that the classes in job 1 year 6 are not the same there have some small differences that on very good to to assume that they were at the same and this disposed down the overloading supporting job that is not existing in Dell script OK so for the final parts of representation we're going to dive into the generated to coordinate class that we looked at before and this is my 1st version and beta 1 so that the class so that the to see
the markings but because some of that that screen in the class keyword used is in their new joseph standard and it has a constructive and then it has all this crap that we don't want to look at too much because it doesn't look good it's very necessary in light of inter functions so instead let's take up take a look at that I managed to improve its and to this it fits somehow presentation page and it's still there is still used in class keyword and managed to reduce the levels of In functions but but I still have is of see In Europe function called overloaded why happen because I needed to simulate these 4 constructors of coronet class and in fact conscripts is 6 classes it's not allowed
to the construct cannot of call itself that's not allowed so I needed to create a function in the constructed colors but and that worked for for this case but sooner I encountered become more complex cases with derived class that needed to call the super class constructor and stuff like that and then it all fell apart so and the final translation in their in their release candidates you start standard its function and for the coordinates and this code is almost to readable yes and you it it can call itself so if you call them the corners constructor with no arguments it's it checks for that right here then it will initialize itself as according at x y 0 this this is generated code I didn't write it so so also it turned out that before I reached this version of that that the other versions there were a lot slower than the and the old manual ports so it's not until I refined to this to this code that it acted got a bit faster than the old man and so on to reach the stable release after I got to these can is 7 I had only 1 failing tests out of 500 units the and I didn't have an effect on test that's 1 you for a file on the case and it was about that is valid operation that checks for a geometry valid to their lead at the and it's was with the specific geometry and the recent that this failed turned out to to be because in in g test there's a class called doubled its which uses some functions in the Java virtual machine that that works is that manipulation stuff so there's nothing like it in so it didn't work as the translation so in the end I had to port that form manually which yes I have failed then creates an automated ports yeah have to but in almost all domains reports that so and is more complete than the old 1 and faster to such great have does so for me thanks to this thank you the but but but but but but but but but to the course
what OK the question was if there's an example so far this test used in in the wild and I have no nothing to demonstrate right here but you can see a j stress in use at the OpenLayers 3 page and there's an example with this system this is uses the buffering function to buffer road section those who you're here the year if you the other were you may be something similar or on the men get a question was if I know about the other reports if there are using the same methods to achieve those sports and yes I know about them and they do not use that method there are manual labor and I think I have a quote from Martin Davis that turning the Java code into C + + which are much more difficult work than he had expected from a lot of time put put into those think especially the simplest for support and they have the issues so far maintenance keeping it up-to-date just because of the high of all this so I just press these you can use your commentary that the pipe importers have actually reported it goes into the reduced KPI but it's provides a Python API that is not that is more Pythonesque so is a wrapper against get the you have the same operation you know what you want to because you don't thank you reality the question was if I have the same functionality I have in this new version near 100 % of the functionality there I guess there are some parts so that that is not translated correctly it's the parts that has no test cases but and there are a 500 test cases trying out most of the functionality so but I have no guarantee there but is a complete sports the and the presence of the I would is the story of the what is repeat last part we what all yeah because anything if there's some requirements for open there's so and no there's not in at the stars of the manual part I do still players in in in Inderjit dataset but I should I remove that also from the manual part of no it's a free-standing library here I I could mention I I do have an input output class and that can translate the OpenLayers class directly so you don't have to go to an intermediate formants but it's optional feature going what through all the money that you have William the you know and you have the features of all the other rules that are going to be working on the issue of racial groups what used to 2 years that the 1st all what you hold hold on to the political all they have to compare the it it should have the same limitations and the question was if there's a limit limits to their operations implemented in genesis and and as far as I know it should have the same limitations as the original ports and there are the the operational port is advertised as that a robust implementation so so it can handle handle precision issues but if the geometry is not valid or if you are at very small position issues then then you can get a topology and then you have to preprocess your input to around it's not it's but you have to check that your input is valid it has to be OK see valid to for the operations to work if you put if you inputs polygon that is not valid and the result is undefined it can you might was the OK then this might yeah that that probably is a box of the left with the the then the question is do can define what i mean with precision issues the default position is floating points to some decimal point and I remember how so far but but you can the JTS also has precision knowledge that you can switch to a threat to a chosen level of precision and sometimes you have to do that just to you have to round down the gentry into 2 something with less I say if you have vertices that are too close but this is endemic you're talking about macros level stuff we're None of Nazi we don't have units in ingest we're talking about the limits of mathematics such so so it depends on the corner system of course what what is a centimeter but if we're talking about extremely small yeah companies