Fusing Structured and Unstructured Data for Geospatial Insights in Lumify

34 views

Formal Metadata

Title
Fusing Structured and Unstructured Data for Geospatial Insights in Lumify
Title of Series
Number of Parts
14
Author
Greenbacker, Charlie
Feng, Susan
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DOI
Publisher
LocationTech, Andrew Ross
Release Date
2014
Language
English
Production Year
2014
Production Place
Washington, DC

Content Metadata

Subject Area
Abstract
Lumify is an open source big data integration, analytics, and visualization platform designed to help users discover connections and explore relationships in their data. It can ingest anything from spreadsheets and text documents, to images and video, representing this diverse data as a collection of entities, properties, and relationships between entities. Everything is stored in a scalable and secure graph database to enable advanced social network analysis and complex graph traversals. Built on proven open source technologies for big data like Hadoop, Storm, and Accumulo, Lumify supports a variety of mission-critical use cases centered around the emerging concepts of activity-based intelligence (ABI), object-based production (OBP), and human geography (HG). Its intuitive web-based user interface provides a suite of analytic options with multiple views on the data, including 2D and 3D graphs, full-text faceted search, histograms with aggregate statistics, and an interactive geographic map exploration feature. This talk will demonstrate how Lumify can be used to fuse structured and unstructured data from multiple sources into a unified knowledge base, and then analyze that knowledge to uncover hidden connections and actionable insights buried within the data's geospatial context.
Loading...
Computer animation Open source Directed graph
Database transaction Code Graph (mathematics) Multiplication sign Scientific modelling Mathematical analysis Data analysis Event horizon Open set Metadata Data model Linker (computing) Knowledge representation and reasoning Ontology Visualization (computer graphics) Information Subtraction Graph (mathematics) Link (knot theory) Graph (mathematics) Projective plane Open source Instance (computer science) Grass (card game) Category of being Arithmetic mean Event horizon Computer animation Visualization (computer graphics) Ontology Computing platform Self-organization Key (cryptography) Quicksort Representation (politics) Data type Data structure
Point (geometry) Filter <Stochastik> Digital filter Mapping Graph (mathematics) Computer-generated imagery Virtual machine Mathematical analysis RAID Raw image format Number Mathematics Linker (computing) Radius Single-precision floating-point format Videoconferencing Associative property Subtraction Raw image format Process (computing) Geometry View (database) Building Mathematical analysis Drop (liquid) Group action Luftreibung Event horizon Computer animation Ontology Quicksort Videoconferencing Electronic visual display Resultant
Email Building Confidence interval Graph (mathematics) Texture mapping Multiplication sign View (database) Area Machine learning Military operation Visualization (computer graphics) Google Maps Videoconferencing Physical law Physical system Family Process (computing) Texture mapping Computer file Moment (mathematics) Constructor (object-oriented programming) Motion capture Computer configuration Self-organization Quicksort Information security Physical system Resultant Row (database) Point (geometry) Computer-generated imagery Temporal logic Mathematical analysis Escape character Subtraction Directed graph Context awareness Rule of inference Graph (mathematics) Variety (linguistics) Twin prime Group action Cartesian coordinate system System call Uniform resource locator Computer animation Visualization (computer graphics) Multimedia Videoconferencing Graph (mathematics)
Web page State of matter Multiplication sign Strukturierte Daten Login Field (computer science) Twitter Spreadsheet Sign (mathematics) Semi-structured model Data structure Associative property Subtraction Email Key (cryptography) Information Open source Metadata Message passing Word Uniform resource locator Spreadsheet Computer animation Database Quicksort Data type Row (database) Address space Field (mathematics)
Context awareness Real number Mathematical singularity Letterpress printing Virtual machine Parsing Image resolution Database Process (computing) Multiplication Hydraulic jump Email Process (computing) Geometry Computer file Projective plane Open source Mereology Cartesian coordinate system Formal language Uniform resource locator Image resolution Computer animation Right angle Quicksort Data type Row (database) Field (mathematics)
Octahedron Musical ensemble Game controller System call User interface Computer file View (database) Virtual machine Bit rate Hypermedia Statistics HTTP cookie Formal grammar Physical system Window Default (computer science) Installation art Area Standard deviation Spacetime Scaling (geometry) Slide rule View (database) Structural load Computer file Mathematical analysis Bookmark (World Wide Web) Web application Number Uniform resource locator Event horizon Computer animation Visualization (computer graphics) Smart card Resultant Matching (graph theory)
System call Manufacturing execution system State of matter Dot product Simultaneous localization and mapping Computer-generated imagery Programmable read-only memory MIDI Mereology Discrete element method Computer icon Emulation Summation Photographic mosaic Pointer (computer programming) Chain Centralizer and normalizer Degree (graph theory) Peer-to-peer Moving average Statistics Conditional-access module Arc (geometry) Window Chi-squared distribution Default (computer science) Metropolitan area network Raw image format View (database) Computer file Core dump GEDCOM Bookmark (World Wide Web) Maxima and minima Number Uniform resource locator Event horizon Computer animation Oval Infinite conjugacy class property Dew point Theorem Videoconferencing
Point (geometry) Frame problem Slide rule Musical ensemble System call Manufacturing execution system Simultaneous localization and mapping Computer-generated imagery Discrete element method Event horizon Emulation Area Finite element method Arithmetic mean Radius Statistics No free lunch in search and optimization Gamma function Uniform boundedness principle Conditional-access module Window Chi-squared distribution Default (computer science) Metropolitan area network Operations research Data dictionary View (database) Computer file State of matter Letterpress printing Bookmark (World Wide Web) Maxima and minima Number Computer animation Estimation Personal area network Resultant Reverse engineering
Point (geometry) Octahedron Distribution (mathematics) Home page Hand fan Tendon Pointer (computer programming) Hypermedia Moving average Physical law Gamma function Window Metropolitan area network Mapping View (database) Computer file Coma Berenices Bookmark (World Wide Web) Length of stay Liquid Algebra Computer animation Website Graphical user interface Units of measurement Resultant
Email Frame problem Structural load Simultaneous localization and mapping Discrete element method Emulation Chain Pointer (computer programming) Conditional-access module Metropolitan area network Chi-squared distribution Window Metropolitan area network Time zone View (database) Online help Computer file Core dump Bookmark (World Wide Web) Quantum state Plane (geometry) Inclusion map Computer animation Oval Computer cluster Graphical user interface Units of measurement Resultant
Point (geometry) MUD Mapping State of matter Network operating system Tournament (medieval) Programmable read-only memory Sigma-algebra Hand fan Emulation Java remote method invocation Spring (hydrology) Chain Cantor set Physical law Conditional-access module Metropolitan area network Window Metropolitan area network Satellite Mapping View (database) Computer file Total S.A. Color management Bookmark (World Wide Web) Annulus (mathematics) Maxima and minima Radius Computer animation Uniform resource name Decimal Personal area network Form (programming) Wide area network
Email Metropolitan area network Musical ensemble Probability density function Melting Electronic data interchange Slide rule View (database) Multiplication sign Computer file Demo (music) File format Volume (thermodynamics) Bookmark (World Wide Web) Hand fan Dressing (medical) Duality (mathematics) Malware Computer animation Moving average Information security Uniform boundedness principle Window
Computer animation Strategy game Multiplication sign Constructor (object-oriented programming) Queue (abstract data type) Right angle
Point (geometry) Programmer (hardware) Number Database transaction Online help Computer network Multiplication sign Website
Area Computer animation Graph (mathematics) Multiplication sign Open source Code Physical law Right angle Staff (military) Scalability
Area Trail Computer animation Natural number Multiplication sign
my name is jolly greenback this season my friend and co worker and were talking about using structured and unstructured it together for geospatial insights and find it won't happen in by the cost of this
time so 1st ball what is confined to conceal size sauce Big Data Analysis and Visualization by 1 of the project to build the start of by mostly injury by Altria engineered by the sauce meaning open sauce Patalay since the code all get so pleased free to download cheque and from the UK
and so coupled concept understand what happened coming out of the following RAF base knowledge model and fine so essentially serving Ontology which is the customer of the user organisations data my it the finds the different things relationship to the 1 real model in the data to be and how really insists the definitions and these relationships and properties and these are anything that you want to model mutatis essential the prop announced people places organisations events transactions those sort relationships are links between entities towels and it is related to those of the edges of McGrath which can be are between different types of entities 0 per cent in the leader of the organisation 1 person can be related to another that sort of thing properties are essentially an e-mail Metadata about the entities or relationships so person in the 1st and last time perhaps employment relationship start and that sort of thing and the grass is a collection of all the instances of those and it is relation chips and properties at a defined and of the and Ontologies the abstract definition and the graph as the collection of almost instantiated instances of those things so acute
by used over the last different things
1st might start by searching the data to a new Sir Chiricahua trafficking and the 1st 3 results at the document video images of the raw files ingested by the fine of the things will are those entities that were extracted result either by automatic Machine Processing upon in just or perhaps a human user 1 along and added things change things that sort of thing aren't and not just write your not to search and buy he worried constrained by coupled filters a couple of facets of the search for the 1st thing we did was constrained results to only give me results the capitalist geolocation associated with the ball within the thousand 400 of the the point the and and also have a date associate with them between those 2 points on their so fast that it will Search people also use of the providing a lot of link analysis of profits are planning a single Entity on Torygraph for the 1st person year want to start long different items of persons related to to assert that the battle of the phone number of the person known search for half an hour we have company and there were relationship between those and the person who owns the company we spend
time building up the graphic at this point we have more people on the half the companies the associated with profits of applications in a way of phone call records and different things like that the bill that the graph knowledge and visualize about what we do want to
work with different kinds of more than 80 especially construction data things like Texticon images video audiophiles sort of what you see here is the confidence of the time after and process by the flight of 2 different machine learning tools and also humans have gone through and and extracting results and easily have people on their places locations organisations temporal references that have been certified elevated from the undiscussable the text for example of the 100 mountains is not lead to an actual geospatial and but for the moment at which would allow to specialise in this is what we are talking about any data that you have that has sought geotag associated with that can be aggregated in views via we can work with any any mapping system that supports openly that Blue use Jubarah of Google Maps just for them was welcomed to the sold so worried
that geospatial they come from the reasons why we really work with 3
different types of sauces Tuesday structured some unstructured unstructured so structure is the sort of the stuff that comes pretax the traditional geospatial data something that there are a has assigned to its that something like the downloading tweets that had been Jeep yes and perhaps you have a basic records are sensor logs passenger gesticulated estate associate with work that following the problem about it is a lot of time in working with some structure and unstructured data that need to do some work to extract enriched you spatial information an associate with the aid of an example of some Research a would be a spreadsheet perhaps it has columns for City country the city state country messages just words strains just Texas fields and you know it provides unstructured telling us with those locations are but we have to do some work for a week and in a sign that Wednesday's 1 of the unstructured again that was attacks acting like emails to ports of key pages whenever we need to do some work to identify the places mentioned in the text disambiguate them result to get to record a loss of among but the way that we work
with that some of unstructured and unstructured areas with full called played which is an noble sauce you 1st and you crosses a tool that essentially takes unstructured taxes and so many of emails ports 20 its going to automatically to find any of location names mentioned that free running Texas City country's mountains reverse would remained location and it's going to resolve the names on 2 gazetteer so tax comes in in in Kensington Panamao command of the US over the were doing here Brooklyn and as geospatial and the resolution we find the names of places in text trainers and together a record of the real magic of Lehman another you off like a plane and as it does disambiguation of the biggest placings publicist print from right by comic actually that Springfield and banque which bring about the play and salsa sprinkled from uses the context of the which which Springfield for example the the 1st of of soft project will likely actually just added in the future of finally him those and that some of Structured small type are location names might find databases Richard Wagner and on the injury the plate and the and find out more about the place in which to live and die if you interested in exactly how
the works its magic really commination machine learning Madeleine processing and tourist a algorithms but if you were application thousands last year to offer the talk about leaving were details about how it works out of the mess that talking and please go to clean and sort and jump
in the rate down to duodecimal of 1 of running
on the rest of this year ago so
long to find a on the Web applications to use to instal Tamarama machine of runs in the match play so the slogan here to this is what these received the 1st looking at Systems the work space complaints where they will be wracked the data search for this analysis and visualization and the 1st thing on my views to equip search of a wild card searches Stephen loaded into my system will file scale began a before the standards of loads of the small based said it was not Search Estonia and you have under were all of those on the original files only just via the have documents Henderson for damages from the body of files and then everything else the locations the people contacted of those early on and then I said that in extracted either by the system of self automatically or by music come along and do some work on the media and the gun here have wanted some countryside and see this country's back in the search results by control down just look at the country's and you have a lack interest live their area of the
state of the economy here this is the
status of about drug trafficking so central South American locations after this is the start of the analyst status alone Mexico iconic laconic baseball part in the problems of the different documents that its mentioned and his doctorate here that look
at the shows the of really a preview the conference of the Police on the slide for anywhere that the document is bowled like sandals here United States Mexico singers entities at work Tredegar's all was reason and unlikely events legacy yet doesn't just find several here the Columbia proved that Basildon down automatically likely than going and and finding out may be on to a different Search to
add a filter lecture before the slides the on constrained by results not to look at all the you all possible results but and the constraining here to see about 500 kilometers around the point which happens to be a solo and only 11 results here 11 results of fall into that on the phone to that results Eichinger's again slept with the click on the capital solo reversal Cecilia Galashiels there are you to 3 dozen companies that have have had ordered Derby had ordered in the city's amid that is something images of an at grab all those results
strange after McGrath here and
so you know in the McGrath you really makes sense to look at things and I have connexions between the end of his reign at the nation's between these results on Sunday on a switch the Matthew look at things such a specially sites which automatic conceded the Jews facial distribution of the that just the search for a new real simple were just put in the hands of a map on and 1 of these the new is that this is a lot of things right close to each other rather than have literally and how each other see can see where they are in the most consistent we're all those automatically so as a similar see the 6 things about their eyes and was made into a lot to the Middle pieces and a little away all all the way down the media known search from search to be on a search directly from map and go created new workspace map and another points late another Blakesley to work from and new things right from the recommend here
is a man to air zone and
was he was going designer of the Komatsu results with a radius and in the middle of the zone and dragged that the roughly covers the citizen and the again now owned by
searching its data Holdings directly from the map of drama radius announces of tournament here again items amount to see the point of going on sale to the next as the man total by just
go along the state aid here again
maybe that's interesting to me is an analyst at the time he spends in the No bring them along the way to
start for more details such as a really really Pratama dress and see
more of their Justin more with comes the
Susan after were to be happening to them and were so the handover Susan are nice to have time some people who say that you actually I of have the right and but I fully acting through the public have a right of way on Friday that it and the
other with with a tie in with the the right see that strategy and construction that to face long wait in a queue for the
method is to to use the site as a whole were on an renowned lead time will not be fully fit for the World Cup finals and beneath
the right and by the end of the programme will be the high point of the right hand in the air the have their
delight in the fact that this this people have complained that it was the right thing the it in the right now is really a kind this he has been working and and will have the area at the time of the attack the way it is and how it can be found that the act of taking
off from the son of a well known fact that the Fed will have to go and see the at the fact that the public would like to know how to build a new 1 on the way up at Old this with a lot of regularly at the right time at the end of April the any extracts of things I would do it such what has struggled with the lack of something like that you will end in the
open air in the room by the time my son now think that there could be 1 of the reef the death of the following the idea that people who don't like the fact trained on the right track and tried to find out what and the actor and writer nature of the area by the end leader of the free and director waiting on Blue Gene L Cpl the and
Loading...
Feedback

Timings

  822 ms - page object

Version

AV-Portal 3.9.1 (0da88e96ae8dbbf323d1005dc12c7aa41dfc5a31)