The Blue Hub, an integrated analysis platform with a WebGIS front-end to exploit maritime Big Data

Video thumbnail (Frame 0) Video thumbnail (Frame 3212) Video thumbnail (Frame 4906) Video thumbnail (Frame 6737) Video thumbnail (Frame 7956) Video thumbnail (Frame 8893) Video thumbnail (Frame 10064) Video thumbnail (Frame 10670) Video thumbnail (Frame 12180) Video thumbnail (Frame 12969) Video thumbnail (Frame 17251) Video thumbnail (Frame 18478) Video thumbnail (Frame 19106) Video thumbnail (Frame 20276) Video thumbnail (Frame 22271) Video thumbnail (Frame 22793) Video thumbnail (Frame 23437) Video thumbnail (Frame 24420) Video thumbnail (Frame 28782) Video thumbnail (Frame 34236) Video thumbnail (Frame 35306) Video thumbnail (Frame 35909) Video thumbnail (Frame 39707)
Video in TIB AV-Portal: The Blue Hub, an integrated analysis platform with a WebGIS front-end to exploit maritime Big Data

Formal Metadata

The Blue Hub, an integrated analysis platform with a WebGIS front-end to exploit maritime Big Data
Title of Series
Part Number
Number of Parts
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date
Production Place

Content Metadata

Subject Area
At the Joint Research Centre (JRC), scientists involved in maritime situational awareness are confronted with a growing volume of data. Every day millions of ship positions from terrestrial and satellite receivers are gathered globally and in real-time, as well as optical and radar Earth Observation images, leading to a significant variety of data. To support the researchers, policy makers and operational authorities in their activities a analysis platform with WebGIS functionality has been developed with the aim of turning data into valuable information and demonstrating pre-operational tools for maritime awareness. The platform is mostly based on FOSS software and consists of a front-end visualization tool and a back-end analysis engine. Fusion algorithms provide the ability to integrate data from multiple sources on the fly. A series of tools provide predictive analysis, activity mapping, anomaly detection, and cross disciplinary information, to support maritime security and safety and to improve marine knowledge. The web application is developed using open source programming languages (e.g. Javascript, Python), frameworks (e.g. Django, Geoserver), and interchange data format (JSON) to enable researchers to seamlessly integrate ad hoc algorithms developed in scientific languages (e.g. R, Matlab). A case study will be presented, showing examples of how the WebGIS architecture can provide visualisation and analysis tools to support decision makers and scientific and operational actors in the fields of fisheries science, maritime spatial planning, and maritime surveillance.
Keywords JRC
Satellite Presentation of a group Context awareness Service (economics) Decision theory Knowledge extraction Mathematical analysis Solid geometry Content (media) Mereology Non-classical logic Time domain Goodness of fit Different (Kate Ryan album) Robotics Operator (mathematics) Personal digital assistant Computing platform Process (computing) Computing platform Position operator Physical system Computer architecture Area Domain name Service (economics) Multiplication Computer-generated imagery Software developer Structural load Mathematical analysis Funktionalanalysis Type theory Process (computing) Event horizon Computer animation Personal digital assistant Uniform resource name Internet service provider Computing platform Computing platform Communications protocol Computer architecture Row (database)
Service (economics) Multiplication sign Set (mathematics) Insertion loss Mereology Expected value Different (Kate Ryan album) Profil (magazine) Repository (publishing) Endliche Modelltheorie Lie group Computing platform Position operator Physical system Social class Predictability Standard deviation Trail Information Internet service provider Funktionalanalysis Variable (mathematics) Web application Process (computing) Computer animation Visualization (computer graphics) Repository (publishing) Website Computer architecture
Context awareness Computer animation Green's function Website Mereology Position operator
Trajectory Software bug Uniform resource locator Identifiability Trail Trajectory Computer animation Information Direction (geometry) Authorization Mathematical analysis Routing
Workstation <Musikinstrument> 12 (number) Multiplication Greatest element Information Codierung <Programmierung> Closed set Vector potential Timestamp Area Position operator Type theory Message passing Uniform resource locator Explosion Computer animation Robotics Function (mathematics) Formal verification Position operator Physical system
Satellite Overlay-Netz Raw image format Satellite Computer-generated imagery Information Computer-generated imagery Projective plane Knowledge extraction Maxima and minima Mathematical analysis Funktionalanalysis Content (media) Mereology Emulation Web application Event horizon Computer animation 4 (number) Personal digital assistant Personal digital assistant Computing platform
Trajectory Mapping Mountain pass Collaborationism Cellular automaton Knowledge extraction Set (mathematics) Element (mathematics) 2 (number) Geometry Object (grammar) Energy level Arc (geometry) Dean number Physical system Metropolitan area network Collaborationism Raw image format Trajectory Information Ext functor Funktionalanalysis Numbering scheme Population density Process (computing) Event horizon Computer animation Software Personal digital assistant Computing platform Resultant Row (database)
Satellite Scripting language Serial port File format Time zone Set (mathematics) Database Special unitary group Dimensional analysis Area Geometry Mathematics Estimator Mechanism design Cube Moving average Process (computing) Physical system Area File format Ext functor Staff (military) Funktionalanalysis Flow separation Virtual machine Mechanism design Type theory Right angle Data structure Row (database) Spacetime Mapping Service (economics) Table (information) Observational study Algorithm Cellular automaton Ultraviolet photoelectron spectroscopy Virtual machine 3 (number) Streaming media Heat transfer Infinity Average Distance Event horizon Coprocessor Element (mathematics) Goodness of fit Population density Arithmetic mean Representation (politics) Spacetime Computer multitasking Data structure Summierbarkeit Message passing Conditional-access module Raw image format Server (computing) Cellular automaton Java applet Dimensional analysis Heat transfer Database CAN bus Event horizon Computer animation Software Library (computing)
Euclidean vector Table (information) Line (geometry) Network operating system Multiplication sign Connectivity (graph theory) Zoom lens Streaming media Event horizon Dimensional analysis Cube Program slicing Energy level Row (database) Computer multitasking Process (computing) Summierbarkeit Data type Raw image format Simulation Multiplication Information Streaming media Database Type theory Process (computing) Event horizon Computer animation Cube Energy level Row (database)
Polygon Table (information) Zoom lens Set (mathematics) Parameter (computer programming) Bound state Rule of inference Web 2.0 Geometry Degree (graph theory) Population density Cube Energy level Computer multitasking Volumenvisualisierung Right angle Data type Zoom lens Server (computing) Tape drive Parameter (computer programming) Measurement Type theory Population density Sample (statistics) Event horizon Computer animation Cube Right angle Energy level Integer
Point (geometry) Satellite Area Polygon Slide rule Zoom lens Service (economics) Scaling (geometry) Information Chemical equation View (database) Polygon Zoom lens Bound state Parameter (computer programming) Database Bound state Type theory Event horizon Computer animation Energy level Divisor Energy level Volumenvisualisierung
Polygon Distribution (mathematics) Mapping Information Set (mathematics) Funktionalanalysis Transformation (genetics) Type theory Population density Degree (graph theory) Computer animation Personal digital assistant Uniform resource name Cube Program slicing Artistic rendering Representation (politics) Computer multitasking Volumenvisualisierung Data type Extension (kinesiology)
Point (geometry) Mapping Information Transformation (genetics) Plotter Zoom lens Execution unit Gene cluster System call Graph coloring Mathematics Population density Computer animation Different (Kate Ryan album) Single-precision floating-point format Artistic rendering Extension (kinesiology) Extension (kinesiology)
Standard deviation Service (economics) Sequel Divisor Algorithm Information overload State of matter Direction (geometry) Cellular automaton Mass Mereology Formal language Web 2.0 Telecommunication Different (Kate Ryan album) Vector space Repository (publishing) Data mining Artistic rendering Estimation Process (computing) Endliche Modelltheorie Computing platform World Wide Web Consortium Service (economics) Standard deviation Computer-generated imagery Key (cryptography) Software developer Information overload Database System call Data mining Process (computing) Computer animation Vector space 4 (number) Query language Repository (publishing) Telecommunication Artistic rendering Key (cryptography) Musical ensemble Table (information) Task (computing) Resultant Library (computing) Row (database)
Point (geometry) Axiom of choice Greatest element Revision control Sign (mathematics) Population density Software testing Physical system Time zone Matching (graph theory) Key (cryptography) Compass (drafting) Moment (mathematics) Projective plane Sound effect Menu (computing) Database Funktionalanalysis System call Human migration Type theory Exterior algebra Process (computing) Website Quicksort
Satellite Polygon Table (information) Algorithm Database Mathematical analysis Average Geometry Degree (graph theory) Artistic rendering Moving average Spacetime Process (computing) Volumenvisualisierung Message passing Data type Metropolitan area network Matching (graph theory) Computer-generated imagery Server (computing) Java applet Dimensional analysis Symbol table Web application Population density Event horizon Computer animation
Satellite Trajectory Computer-generated imagery Projective plane Mathematical analysis Rule of inference
Metre Point (geometry) Group action Focus (optics) Server (computing) Information Forcing (mathematics) Variance Price index Mereology Machine vision Front and back ends Frequency Process (computing) Computer animation Ontology Right angle Extension (kinesiology) Information security
OK land so we will start with the last topic of decisions midday engines will give merely layer by opponents 12 and which we reason to speak to you about the brewer but In many can be the the good afternoon I'm interested in the area and software developer at the Joint Research Centre and and member of the European Commission today hand here 2 per cent that want platform is a software platform that he developed the tool will handle billions of records it can on my dying context solid satellite images and so on with see my presentation we go very quickly through a platform overview of some of the functionality of the system and then we read the inside and use case that is related to the haven't Bayes knowledge discovery and then the loudest let's just part of the presentation will be dedicated on the official development so this stock what is the grew out blob is that Joint Research Centre here analysis platform that we use to collect process analyze these lights and loads of data on the maritime domain like she position on the ship position robots startles and that of Bermuda Cayman coming from most of the light like sending a 1 sentence to 1 another the main actors of the system are of scientists policy makers service for the European Commission and operational alternatives logic architecture of the system so and that we have multiple data provider that has a different type of data with different protocols
so with different formats like in a manner copy of all the or by using an HTTP streaming there has to be an and so on that the 1st the 1st the bottom part of the system is there the gardening part there we have multiple tools that you use to gather this information process this information then the put it inside of variables history that is represented mainly by positive that the base with positive sense sanction and more recently among would make then we have the knowledge and discovery tools that are a set of tools that allow our scientists meals a day crunch all the data that we stored inside the the repository introduces felt and put it back again decided that it was agreed to be lies or exchange we extend platform with the other 2 models that are visualization and essentially get about the web application and ended up exchange we have a lot of services that that expose data of different standards
of 1 for all for example I didn't 1 of the 1st set of functionality that implement inside the system used might situation picture so with this functionality we we know what is going to happen in with a prediction of 15 minutes on this C of and this is a different tionality did you expect is assigned the functionality that you you to track of the best seller and provide you with some information about this that accountability like the here and the expected time of arrival of all around destination and now there are 2 poodles but this is just the 1st feature and we have used our our data to map of fishing activities that just by analyzing the speed profiles of some truly we we're able to cluster this information leading 5 3 classes that are like this site is sitting at a loss of what the best so he's doing so we can identify and where it was fishing hosting and what is and when you the best so we're going to the people are always
inside the same context the and by using the power that in our dataset and we we we could be it and intensity of fishing on fishing marked and that is publicy available on our website in both for us that and that former
then we worked on and behavior detection in these examples we have a lot of data on the right bottle to the position of the vessel position and then we able to the 5 stunned the part of the movement for example on the that you can see the daughter green on the right path but then there are a lot of In this example the best so suddenly changed its
direction and then iron know to to to return and Becky Jena and all noun the expected of them and the expected route so we can identify these and other type of of of anomaly and we can produce a for authorities for example that you need to another
example is trajectory analyzes so all of we we can analyze the trajectory of the vessel and we can cannot we we glossary we the data and we we could for example identify of the party location just by using the information about movement of the sheep and other
being tyrosine no features seeds the IIS radio location this is the type of it's a system that best so used to send messages about what is going to top and on the sheeple where and and solve for by by collecting exactly the same message by multiple base station on the ground there and buy up touching on its timestamp but we can use this information to all of most to let the position of the vessel so in this example that I all Eric specially on the right bottom is added the interesting simple where the vessel declared to be in the position indicated by the blue our robots we detected that that the true
position on the where exactly the close the portal General and other
functionality that is the satellite image analyzes cytosol all and we we can accept and a
lot of information from something like my images and then overlay the images on the web application for a visa of competition then In this
case the simplest case uh in this is and part of the project is at and these knowledge discovery
and movement of functionality that that we built in collaboration with the Italian Costco essentially in we fetch from the raw data a lot of information OK I sure you in a few on few seconds are records we use we preprocessed this information that we accept trajectory from the software and written in Matlab in this case then no by building the situation of of what this happening on the of the global level we produced for each of us so that we identify a set of elements the 7 so I was sent to this system that process data and by using edgeR references little agreed that can build their own uh
and and some interesting results like specially for for
the of about what's the and you haven't that here we have more to talk about and like for example proximity elements all for example when approaching another 1 they're ups to something illegal or and likes changing staff goods and so on our own we have all the type of event like the 1 that muscle entering and area we interested only in the cell
the rest of this this 1 on in this example we have a lot of the the density representation all the elements of the entry into a single cell with is size satellite is an hour of the CEO of drug use of 0 . 2 0 0 4 the any In this example we sure that the density of the same event but just for fishing vessel and we are able to say yeah these are the low efficient that cell and that this is the idea where they are fishing we have a huge database as a tool for locating by using prosperous estimation we can say that the receiver more than 160 6 millions of records several months and then our system we stored more than 11 billions of records to process over time and space dimensions to do this we use a lot of technology and we use a matter called John called bite on the simplest blasts jobless created intentionally also openly years so best speaking there and we use a lot of jails said that lot of just have a functionality and Boston what is the problem now the problems of the study the scientists have had OK and I'm joking obviously but it 1 of the key point because of the the rights of the reading them out loud but higher and they're not used to build the I t systems or software we have multiple cooperating the process and that there are running on a different machine and that obviously we had to solve the problem of of the of the and rendering the performance of and so the 1st question is how we can we transfer data between the processor and machine and how can the scientists very he's the change the data structure they they want to exchange we've the processor without admitting that getting so we need a common that transfer mechanism in a common serialization format for these we choose the adjacent serialization blasted 0 and q library as messaging system essentially the Albanese generates a set of elements and and these these those islands are encoded hinges on format to stream into a service and then see inside the said that the bistable or more with more detail than what to reveal was a multidimensional of Delaware also Cuba and now we have this
process so minus plus plus that continuously process of the stream of event and the immediately update to the need to be the major see inside the multidimensional Cuban stored the you haven't inside the database the dimension that to use are the data without that time component not interesting for these example and the dispatch of dimension is I'm not to let alone as usual about the discrete value including the narrow column through the this 12 agreed that they show a few slides before that all the dimensionality vessel type and even by blasting especially dinky yeah and that we call simulation for performance reasons that we decide
to use the special dimensional row and column 288 in that generated multiple zoom level by preprocessing the information inside the tube and then and then using also the vessel typing given that we created 720 seeks slice of the cube this is for improved performance surrendering so In
this example we have a simple could lies so he's
going to foster care anyway In well I would like to show you a small detail but I cannot rule back
yeah that's it sorry but a simple quiz lies Caesar OK uniquely identified by the zoom level and that's it so that that's a type and haven't fortunately on only the only measure that he want inside a cube user density and sold on the web J. supplication and just have a real and the low and a set of parameter tool to access to the right due tool short to to render as fast as possible used a few
tricks for these that I want
to share with you my 1st 1 and then we increase the 5 sides of of in the in the Dublin as service
because we notice that for this experiment is more convenient when you bond and then we need web-application passes the information about the balance of the view that the user to visualizing based on the zoom level that uses at the user user's using so the past the bounds of to all of the uh the database query to restrict the area of interest that want to we slides a solid we now improved acquired performance then we have a lot of people so all the parameter like of the the type of rendering the and we we can decide that based on the Duma before you want to run their uh things like polygonal points because you can imagine when you zoom old-style to you are these slides something of the scale of Europol toward the using polygons tool to represent the satellite is quite useless because he's in this you cannot distinguish from point and the point is faster run so in this example the we use a with this is a
representation of the slice of the cube beta vessel type blasts the polygonal and with a distribution of the best social inside the
not on the same set we can access to all the information in this case we have a distribution over the course of the ground of the fishing vessels another interesting
functionality we introduce the is that he cannot that
these has been implemented by using just over rendering transformation you WPS extension and this is amazing because it's of faster than units did you did you did you expect it render beta very quickly and you find is the fine because the changes will white zoom so all these can use some information about the clustering of the of the
density and the urinating layer and engage said it was a very big problem Because to plot of all those points that we were able to do just that but because the 1st stop and to what to was to create 1 single layer year with 4 different color densities sense to render everything we found there will for this approach OK we fit in of problem we following solution we don't know about so we discovered that if we as just 7 to and that 1 single call for our points is very very very fast so we decided to create for the yes and
then that might rendering Alderley years so badly we overlaid all the results and to get to
the final and the final image last part
what we're doing here we are removing and we are moving forward direction to use more must sequel that the database because our developers histories changing we have that are more unstructured data and that of that you want to analyze the data and we want to understand what is inside this those data and for this reason we start using model sequel also for these expect for this experiment we used to be you so records of they 11 not to be extracted the can be stored inside the mall would be for performance reasons for the same reason that the knowledge and discovery to are going to be extended to 2 and the new data and by writing also and you are going to and but using new data mining approach and that for the what is concerned the did the rendering we are creating a new Web Tools and that we have stopped using more of a vector B that's all more than fast more direct Jason usage more vector ending in the state of the Dublin conclusions from our we were able to conclude this tool to to to to build a platform because we standardized the the access to the repository sold by creating libraries so by creating services of sometime often to me tool to keep the process of lunch by some scientist and the like to the scientist will be calls late start the query that around a full table scan over 11 billions of records so you can imagine that the mass OK so libraries and services so that we can control law and optimize the answers to the the communication is another key factor salt and but using standard that we we could communicate to the data between different process and the more important the between IT guys scientist and another thing that always related to scientists please don't put warriors the band studios just deals Saul and so what would like to overload and now we've technical EST stuff like that lets us to be out to optimize something so and they're not used to this and we use standards whenever possible like in a manner of innocent FEs no obviously there is no 1 technology that will give us the solution about use it that again and so to loosen the languages for the thank you so much so thank you think you does anybody have any
questions OK I just wanted on the relationship between you and the more next year the EU and projects testing marine data year yeah we find all we have a few compass points about essentially we receive the data we have the the agreement that we 1st extend company like a necessary as like the on a regimen calls the of Italian calls so long and as they are sending us the data so we didn't we have just context-free final it was a century later it seems quite similar to me but they too much about both of them but they seem very similar this choice of the choice it the rest of the team that you need to know what the hell is left from the projection zone going yet another project is still ongoing the user is running at the bottom of the understanding the project served by the start analyzing all sort of type of data that are more related to movement and migration in in the latest period but that the system is not the perfect isn't going there is available policy on our website so all you can now that they have and they have an overview of all the functionality of contact us if you want to read some people some of the key things In recent years of looking just as the underlying database questions with the recent versions of prescribes that implement a lot of no risk you functionally supporting varies from 1 to 2 etc. and you look at it as an alternative to monitor the way yeah we need we're on the best exactly in the same manner that effect so we and we we took the same that that we rounded pessimistic but the sense of the latest version of all this legislation normally be exactly the same test they a and that our wedding around that database fast among would be and lower on that's why we decide to use moment because we have to process a lot of rackets just imagine that to build the density it takes something like 15 days so on or it's a very long process in the now we not using the special attention on 1 would be the the rendering use built by using the open year and then client-side that as much as possible and obviously by using just have to the only thing that I don't think I'm particularly interested in you we'll look because of my research we might come from South Africa and now we do have a similar project that will be incorporated in the C L S is full begin people bolts shipping companies that this each 1 the device but they are eager shipping the activities that they don't they just each of the idea as been they do that things some have you considered incorporating is a sign of data indirectly that there is any match a lot of and aware and that we
are almost something on the shelf as you know there was a project
uh but to reply OK anyway should be assessing OK and there
was a slight where we show
and satellite image related on the web application but if you break the tension there where local was most symbols so on that humans because we process see a satellite images and then now we with that's processing weaker match the cooperating
sheep because sending a message and
not cooperating ship so there was a project that are related to the African idea called the mark that rules that are related to security and above the idea of of the West Africa and that we use the satellite images tools to to improve the results so of so you
have considers do something like like I don't know which also as well as spatial extension so you could perform some special functions on or is it something that you know you didn't know all we we took a consideration of group will the user goal Bob when we did this is what is analyzes we don't have the infrastructure of tools there and for a system of and the ball with the process all of those rights and fortunately things have changed in the latest period so we can do more spherical that's why we don't use get that was my other question so do you all users servers to host everything and in the Instituto user quality now so you you indexically yeah fiscally internally to the sodium OK so that wasn't is explained below the estimated variances that vision think knows of questions and I have 1 you as as you have implemented some kind of the best people in business so I I'm wondering ladies and you use the last which is which is implemented useful 7 years whereas the mean simply because we we decide to continue to use force because the other side are used to to use the positive as in that usually they belong to the want to learn another taller when they have something that they know how to query like the other 1 is based on the part of you as a back end user look and just come into so much that you realize everything this information and I will investigate the point that so we can do with something better with thank you it is being made from the because the ontologies used massively as the European Commission and the focus of this for his indications last mentioning thank you so much for such any is efficient so thank you very much for a nice evening and come to seismic this evening to meters you before security