ArangoBnB: Fullstack JavaScript with GeoJSON
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Alternative Title |
| |
Title of Series | ||
Number of Parts | 47 | |
Author | ||
License | CC Attribution 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/59559 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
| |
Keywords |
00:00
Java appletScripting languageRepository (publishing)Public domainStack (abstract data type)Demo (music)PlanningFront and back endsTwitterSoftware developerFinitary relationOpen sourceEnterprise architectureDatabaseGraph (mathematics)Source codeSoftware maintenanceSingle-precision floating-point formatDemonProgram slicingProcess (computing)IterationProjective planeSoftware developerForm (programming)Theory of relativityAlgorithmStack (abstract data type)NumberData modelProduct (business)Graph (mathematics)Data analysisVirtual machineFunctional (mathematics)MetadataDatabaseSoftwareComplex (psychology)Similarity (geometry)RankingBasis <Mathematik>MultiplicationTransformation (genetics)AreaInstallation artInsertion lossLaptopEnterprise architectureMultiplication signQuicksortInteractive televisionData storage deviceINTEGRALSuite (music)Point cloudPoint (geometry)Subject indexingBitScalabilityKey (cryptography)WebsiteCloningCASE <Informatik>Analytic setDemo (music)Search engine (computing)FeedbackVector potentialoutputCubeCollaborationismWave packetFocus (optics)Different (Kate Ryan album)TwitterComputer animation
09:06
Source codeSingle-precision floating-point formatSoftware developerSoftware maintenanceData structureVariety (linguistics)File formatCodierung <Programmierung>Attribute grammarObject (grammar)Point (geometry)MultiplicationString (computer science)Line (geometry)PolygonOrder (biology)ImplementationGeometryType theoryException handlingCoordinate systemProcess (computing)Category of beingRight angleGraph (mathematics)GeometryObject (grammar)Attribute grammarType theoryPositional notationCASE <Informatik>Latent heatData structureImplementationConsistencyQuicksortInformationAreaCuboidDatabaseQuery languageProper mapService (economics)Point cloudLevel (video gaming)Pointer (computer programming)Point (geometry)WebsiteComplex (psychology)Order (biology)Source codeSingle-precision floating-point formatProjective planeTimestampComputer configurationPosition operatorSoftware developerNumberSoftware maintenanceLibrary (computing)Term (mathematics)MereologyTraverse (surveying)Link (knot theory)Variety (linguistics)File formatOnline chatMultiplicationRevision controlShape (magazine)XML
18:06
Nim-SpielGamma functionEndliche ModelltheorieToken ringFunction (mathematics)Object (grammar)Position operatorNormed vector spaceRepository (publishing)Point (geometry)Uniform resource locatorCellular automatonSubject indexingLevel (video gaming)Revision controlFormal languageProcess (computing)Line (geometry)VideoconferencingMatching (graph theory)Sign (mathematics)BitShape (magazine)GeometryQuery languageInteractive televisionView (database)Content (media)Similarity (geometry)WebsiteQuicksortLatent heatResultantInformationType theoryFilter <Stochastik>Software developerObject (grammar)String (computer science)Sheaf (mathematics)Context awarenessPersonal identification numberFunctional (mathematics)Fuzzy logicAttribute grammarRankingRing (mathematics)Gastropod shellCoordinate systemSource codeValidity (statistics)MultiplicationAreaMaxima and minima1 (number)Computer configurationComputer animation
27:06
Inheritance (object-oriented programming)Logic gateCurvatureOnline helpElectronic mailing listInformationCategory of beingFilter <Stochastik>Functional (mathematics)NumberResultantService (economics)ImplementationAmenable groupSource codeJSONXMLComputer animation
29:43
Digital filterCoordinate systemMultiplicationObservational studyLaptopInclusion mapMultiplication signRight angleQuicksortView (database)Symbol tableWhiteboardFilter <Stochastik>Web crawlerLaptopAmenable groupFront and back endsDebuggerSet (mathematics)Projective planeLevel (video gaming)Electronic mailing listAdditionXMLComputer animation
31:29
Interactive televisionObservational studyLaptopInclusion mapCASE <Informatik>Data structureFront and back endsCartesian coordinate systemCellular automatonLevel (video gaming)Front and back endsType theoryTransformation (genetics)Descriptive statisticsQuery languageArtificial lifeAttribute grammarNumberProcess (computing)DebuggerDatabaseInformationRadical (chemistry)ResultantTask (computing)Data structureProjective planeQuicksortData analysisFile formatDefault (computer science)Point (geometry)Figurate numberCartesian coordinate systemPointer (computer programming)BitData typeMereologyData modelRevision controlSource codeJSONXML
37:22
Vulnerability (computing)Library (computing)Standard deviationText editorMobile appCASE <Informatik>Division (mathematics)Set (mathematics)Computer fileCodeProjective planeXMLSource code
38:47
FacebookPiHill differential equationStatement (computer science)MereologyLevel (video gaming)XMLSource code
40:25
Level (video gaming)MereologyFunctional (mathematics)Computer animationSource code
41:15
GEDCOMFunctional (mathematics)Multiplication signLevel (video gaming)TesselationDataflowOpen setRight angleComputer iconProjective planeSource codeJSON
42:29
Coma BerenicesProjective planeTesselationRevision controlQuicksortRoutingMobile appReal numberLecture/Conference
43:24
Game theoryElectronic data interchange1 (number)DebuggerForm (programming)Client (computing)RoutingError messageData structureFront and back endsQuery languageFilter <Stochastik>Server (computing)Uniform resource locatorSpacetimeConfiguration spaceTemplate (C++)StapeldateiObject (grammar)Device driverCuboidDatabaseLevel (video gaming)Group actionProjective planeCursor (computers)Source code
46:05
Gamma function8 (number)OvalRepository (publishing)Default (computer science)PasswordGeometryInformationAmenable groupCuboidMedical imagingBlogFilter <Stochastik>Electronic mailing listMultiplication signProjective planeReading (process)CodePoint (geometry)Query languageRectangleType theoryResultantExpected valueFile formatProcess (computing)Uniform resource locatorQuicksortUser profileHeuristicObject (grammar)CASE <Informatik>XML
49:49
JSONXMLUML
Transcript: English(auto-generated)
00:06
Hello, yes, thank you, and as he said, my name is Chris Woodward. I am the developer relations engineer at OrongoDB, and today I'll be talking about OrongoBnB, a project that we made as a clone site for sort of an Airbnb clone site, vacation rental site,
00:28
where we developed everything in full stack JavaScript and with the intent of using a dataset containing GeoJSON data. This project, it was a community project, or is a community project, I should say.
00:43
We're still actively developing it. We made this with a couple other members in the OrongoDB community, and currently it is living in our OrongoDB community GitHub org under the name OrongoBnB. And we are definitely open to new contributions, and I'll talk a little bit more about that towards the end on potential ways that you can contribute.
01:04
A new way I actually found out just this morning that if you were so inclined, you could help resolve a new issue that popped up. Today's agenda will be a brief introduction to OrongoDB, what the database is.
01:22
This will just be a really short intro for those that just aren't familiar with OrongoDB. A GeoJSON introduction. This serves as almost more of an overview of the GeoJSON spec, sort of a I-read-it-for-you-so-you-don't-have-to sort of thing. And then we'll follow that up with a quick demo of the site,
01:41
and then we'll start diving into a little bit more of the details of the project itself, kind of like how we started some of the planning, some of the initial data modeling considerations, and then we'll just take a quick run through of the tech stack. I hope if I have time to provide a short demo there for getting started with some of the technologies that we use.
02:05
And then finally we'll wrap up with how to contribute and where we hope the project goes from here. As I said, my name is Chris Wilbert. I'm the Developer Relations Engineer at OrongoDB. And so sort of what that title means is I'm in charge of a lot of different things at Orongo,
02:26
but my main focus is definitely focusing on training and helping with getting started with development using OrongoDB. So this comes in the form of videos, conference talks just like this, but then also guides.
02:42
We have a large number of Python notebooks that you don't actually have to know Python really to use or to benefit from. We have a number of those all available on our website orongodb.com slash learn. So if you are interested in that sort of learning, especially with some more of this machine learning or data analytics side, that's a great place to start.
03:03
I am involved in community outreach, working with community members to do things like highlight the projects they're working on. We're always happy to do what we call a community pioneer, which is a webinar with a community member that just wants to show off how they used OrongoDB in their project.
03:21
This could be something you use in production or just a fun project you did on your own. So definitely reach out to me if you do have any interest there. And then I am also a member of the Orongo ML team, and that's our suite of tools available to machine learning professionals. We have things such as network X adapters, DGL adapters, and we also
03:43
have the ML pipeline that serves to capture metadata throughout your machine learning process. And a number of other tools available as well that we're actively developing and constantly growing that team. If you have any questions about any of that or want to reach out to me for any reason, definitely feel free to do so through Twitter.
04:05
Or you can reach me on our OrongoDB community Slack. My name there is chris.orongodb. Okay, so now let's go ahead and dive in. This will be a really, really brief intro just about what OrongoDB is.
04:20
OrongoDB, we call ourselves the first native beyond a graph database for the first native multi-model database. OrongoDB was started as a company in 2015. And at this point, we actually have two headquarters, one located in San Francisco and the other in Cologne, Germany. We are an open source product.
04:40
We were developed with the community from the start. We have the Apache 2.0 license, and we also have an enterprise edition alongside our community edition. We have well over 65 employees at this point. We are currently in a state of rapid growth. We have definitely well over 11 million downloads, over 600 plus product installations.
05:02
And to date, I know that this number is kind of constantly growing, but we have over 12,000 stargazers on GitHub. That's a number that we're very, very proud of as our community, of course, has been with us the entire time. And we love that this is the way we see a lot of interest coming is with the growth in stargazers.
05:23
So if you do end up liking it, please give us a star on GitHub or a review on G2 Crowd. We are a G2 leader in multiple areas on G2. So feel free to drop us a review there. Okay, so what is OrongoDB? As a native multi-model database, that gives you access to multiple data models in a single database store.
05:49
So for us, that means that you have access to a full document data model. And this comes along with all the functions and interactions that you would expect from a document data store,
06:05
as well as the ability to use your data as a key value store. And then finally, as a graph data model. Sort of the benefit of all of this is that you can treat your data the entire time as a document store.
06:23
You can do your lookups and everything just like you would normally expect. But then let's say one day you do decide that, you know, there are some relations you like to represent in your data. Well, making that same data into a graph is as simple as just, you know, adding another collection.
06:42
And now you are able to define these relationships and just the flexibility of this. Also, you could you could kind of throw both of those aside and say, actually, I just have key value data. And so then you could just use a wrong way to be as a key value store.
07:00
And then we kind of take all of this one step further by making all of this scalable. And that's really where where I think we shine, especially in the long run, is that not only are we a graph database, we are a scalable graph database. We also have a built in full text search engine, which is a bit more than a search, a full text index.
07:22
And we'll talk about that in just a second, because Arango search is actually the basis for some of the stuff that we did in this project. Arango search, and I'll cover that in a bit. But Arango search is a full text search engine that allows for fully federated text searches along with ranking and similarity functions.
07:43
And it has a number of what we call analyzers built in that allow for performing complex text transformations for existing data and new for new insertions. Whenever you insert new data, we can perform these text transformations for you automatically.
08:01
There's a lot to cover on the Arango search side. We're only going to look at a really brief slice of it today. But moving on the we also offer iterative graph processing in the form of Pregel. We have a number of built in Pregel algorithms as well as an experimental Pregel feature that we're working on right now that you feel free to test that out that allows you to define your own custom Pregel algorithms.
08:25
And as I mentioned before, I'm on the Arango ML team. And that is the team that is focused on providing graph ML and analytics tools and trying to serve those use cases more. We have a really a large number of offerings there.
08:41
And this is something where we know is emerging. So if you want to see something or you want to contribute just a input on where you'd like to see that go, please let us know. We're always always happy for any sort of feedback or or collaboration. And then moving on to cube Arango. This is our Kubernetes integration.
09:01
We actively maintain this integration because it's actually used in our ArangoDB has a cloud database offering that we call ArangoDB Oasis. It's a completely managed cloud database service. And if interested in that, that's a really simple way to get started with ArangoDB is you can have a 14 day trial of that.
09:21
And now you have your database running in the cloud. And it's just kind of the quick, simple way. But so what does all of this coming together provide the developer? And for us, we think the biggest benefit is is reducing complexity while still having all the right tools for the job. So all of this came from that polyglot persistence idea.
09:43
So being able to use the right tool for whatever job you're working with. And we think we we've achieved that for a large majority of the use cases. Now, of course, we know that maybe no SQL isn't always the right tool for your job or something like that. But it comes along with this single source of truth where you get all of the all of these data models.
10:04
And you don't have to worry about data consistency because you're always accessing the same data store. All of these have native C++ access to the data. And this also comes along with reducing the need for developer knowledge so they can learn one product, one query language and do everything they're going to need to do.
10:25
This, of course, comes along with less maintenance whenever you have less databases and faster performances, whenever you're needing to make fewer hops across to multiple different databases. This this this sort of reducing of that complexity is is just continues to have benefits there.
10:42
And everything we really target is using AQL. This is our the ArangoDB query language. And we had to do this because, as you can imagine, SQL doesn't do graph traversals and definitely doesn't support multiple data models. But if you're familiar with SQL, learning AQL should be very familiar.
11:03
OK, and I'm going to go ahead and continue on from here. I will go ahead and try and keep an eye on chat. So if you do have a question, please let me know as we go through this. I know it's easy to start talking quickly and especially with some of this stuff, if you're not already familiar with GeoJSON,
11:20
hopefully seeing some of the examples and things like that will help. But so for this section, it's going to be more of just a what is GeoJSON? Look at some of the spec details that might help define that for you. And then we'll see how we use that later on.
11:41
So GeoJSON is a format for encoding a variety of geographic data structures. I pulled this directly from GeoJSON.org. That website also has the link to the spec if you did want to read it. This is going to be the TLDR version. And specifically some of the stuff that I wanted to point out was what we use in our project.
12:01
So it's actually not a very long read either. So as far as specifications go, I would say that it's an easier read than others. So GeoJSON is just JSON, right? So it's just JavaScript object notation with some required members and attributes.
12:23
So at the end of the day, it's just JSON, but the spec just defines some of those attributes that it refers to as members. So I'll try and use that term as well. GeoJSON objects require type attributes. They can also have the bbox attribute or the bounding box to define the area that you're interested of getting these coordinates in.
12:47
And it may have what are called foreign members. Foreign members are these sort of like ways to add descriptive information to your GeoJSON object. The spec sort of says like, you know, you can do this, but ideally you should shy away from it.
13:03
And this ends up being one of those implementation specific things where someone might be implementing GeoJSON and, you know, providing a library for you to use where they say, okay, you should have these attributes because it makes sense for that use case. Example being maybe even something like a restaurant, you know, in providing the
13:23
restaurant name, you know, not part of the spec, but maybe it makes sense. And there's actually even a better way to do that. We'll see later with the properties attribute of features. But this is available there as well. And they further define what some of that stuff should look like in the specification as well.
13:42
So the like, probably the most common thing you'll see when working with GeoJSON are going to be its GeoJSON types. And the umbrella term is just the GeoJSON types. But what that actually consists of are seven geometry types and two what the specification refers to as GeoJSON types.
14:04
And GeoJSON types actually does contain all of them. So of the geometry types, we have point, multipoint, linestring, multilinestring, polygon, multipolygon, and geometry collection. There are some things that each of these require. But while working with GeoJSON, this is going to be what you come up against most often.
14:25
This will be primarily what you're concerned about. What shape is the geometry data that I'm working with? And then the GeoJSON types, the feature and feature collection serve as sort of an organizational structure. And we'll take a look at an example. Actually, we'll go ahead and do that right now.
14:41
So on the right here is, again, pulled from the specification. This is an example of a feature collection that contains features. In this case, I think we've only got, well, we've started on our second feature. But one full feature is what we're looking at here. So object types, which is the area I have highlighted in yellow.
15:01
We're looking at a GeoJSON object that is contained within the rest of this, which we'll also explain in a minute. But those types must be a geometry type. So looking back at the last, it must be one of these on the left side. And so this one is highlighting a geometry type of point.
15:21
So that's just a single point on the map. And it must contain coordinates array with zero or two or more positions. So why the callout for zero is that, again, this is all just JSON. So for JSON, null values are supported. Generally, I mean, you're going to have either two or more.
15:42
Two in this situation is the most common. There is an option for having third, which tells you about elevation. And you can even add some more to this, depending on the implementation that you're working with. One thing was like timestamp.
16:00
But for the most part, you stick with two to three is, I think, the most common. And I've even seen the switching of longitude, latitude. So I want to make a callout for that to make sure that you know whatever implementation you're working with. The specification does say long lat is the correct order. But as we all know, that sometimes developers don't mind adding their own flavor to things.
16:29
And so the shapes, so sort of what this all looks like, the number of coordinates and that sort of thing, really depends on the geometry type as well. And we'll take a look, really a nice example that is available online for you to play around with as well.
16:47
So feature objects. So this is it needs to have the type of feature has a geometry member. So that one we were looking at earlier is our geometry member of this feature and feature collection.
17:02
And the proper properties member can be any JSON object. So this is where I was talking about, where if you're defining information about like restaurants, you could put a property value here, such as restaurant name, and then its name. And then the features collection is the top level structure for defining the geo JSON features that you have.
17:28
And so this is really this covers a large majority of what you're going to run, come across while working with geo JSON out in the real world. So one one way I thought that might help kind of bring this all together would be to take a look at this pretty cool site I found called geo JSON IO.
17:49
So let's go ahead and pull that up now. So this is just a pretty neat implementation of geo JSON where you have a map. You can go around and make points. So what we can do, I thought, let's see, we'll scroll into Germany here.
18:04
And since a wrong goodie be is located in Cologne, I have a rough idea. I've actually gotten to visit. I really, really love Cologne, actually. Hopefully, someday I will hopefully even get to stay there for a bit longer. But I believe we are right around here.
18:24
This is a little bit of a guess, but from memory, I think I think here, but don't don't quote me on that. And so you see, as we drop to that point, now we have our type over here, which is a type feature.
18:40
And we have our geometry object, which is a point and then the coordinates for that. So if we wanted to, I'm actually located in Florida. And I know don't hold it against me and right around the Tampa area here.
19:00
And then our other location is in California. And that is a total guess as far as the location of that. And so you see, as I was doing that, we have these features and we're adding additional features that just contain geometry points. So one example of something else outside of a geometry point is this multi point line or line string.
19:29
And we could draw that from all of these. And now we can see what the line string looks like. So the shape of that line string, since it has three points, contains an array with three arrays within it, three coordinate arrays.
19:45
And so, you know, as you begin, you can you can play around with this. GeoJSON.IL, there's no sign up for it or anything. It's pretty cool. And it's a really nice way to get an idea of maybe just what some of these GeoJSON objects look like.
20:00
All right, and just doing a quick chat check. Looks like no questions, so I'll keep moving on. OK, so. OK, so getting back to the Orongo search, this is going to be a quick idea on, you know, for instance, kind of like how OrongoDB deals with data, the rough, rough idea of interacting with OrongoDB as a database.
20:26
And of course, we have a ton of videos and tutorial content on this. So if you are interested, I'm not going to go over what's basically already been covered a lot by us. So OrongoDB being a NoSQL database, we refer to our collections of documents or collections of data as collections.
20:49
And so that's what we have here. And the normal interaction you would have is you have you happy developer over here with your query. And then you were going to have some filters or whatever you might be trying to figure out
21:03
that you would be putting against the collection. So if you want to find all documents with, you know, if let's say our restaurant example, restaurant cuisine type of Mexican food, then you're going to be able to query the collections for that and get your results.
21:23
And you could potentially sort them. Where things get interesting here is these green sections. These are the Orongo search specific sections. So in Orongo search, we allow you to index data and attributes from multiple collections at once.
21:41
And so when you do a query, you query against this view. And now you're doing a fully federated search against multiple collections. And this analyze section here is actually what we refer to as analyzers. And so this allows for doing things like tokenizing strings or, you know,
22:02
if you need to convert all of your text to lowercase so that querying is a bit more reliable and consistent. We have things such as Ingram match is our Ingram analyzer, which allows for tokenizing it in a very specific way.
22:21
And this is also something that's multilingual. So it supports a large number of languages. And we have a lot more than I'm kind of going over as far as analyzers available. And one of the most recent analyzers that we added in version 3.8 was actually this GeoJSON analyzer.
22:40
So now you can index and have access to all of these Orongo search specific functions, your GeoJSON data. And this would be really beneficial, especially in a situation like the restaurant one, or like we're going to see with Airbnb where you might have coordinate information next to textual information
23:01
that you want to be able to run a lot of these Orongo search specific functions. And since Orongo search has access to ranking and similarity search functions, this can get really, really beneficial. So for instance, if you're doing like fuzzy search functions with our Levenstein match tools, this allows you to do things like autocomplete, or dealing with incorrectly spelled words,
23:25
or being able to try and like pin down more of the context of potentially what the user is searching for, rather than just doing like a one to one look up. There's going to be a lot that goes there. And then once you have those results, and once you have queried all these collections,
23:41
and maybe got exactly what your user was looking for, you still need a way to sort it. And so this is where our ranking feature comes in. We have built in support for TFIDF and BM25, which allows you to rank and then you can sort by the ranking. So hopefully, getting the most relevant contextual results for whatever you're searching for.
24:04
So this is really the super quick version of how we structured data in OrongoDB, and specifically how you're interacting with Orongo search through this Orongo search view. And this is just, yeah, at the end of the day, it just becomes a data source for your queries,
24:20
and allows for the multiple collections. We also have a lot of tutorials available on our site, if you're interested there. Okay, so as I mentioned, we did just add a geospatial analyzer. And so we actually have two, the GeoJSON one is going to be the more common one, allows for indexing valid GeoJSON objects.
24:44
We have another one called GeoPoint, which actually is for just raw coordinates. So maybe you're not specifically dealing with GeoJSON, you just have coordinate information, which is also common. You can actually use the GeoPoint analyzer for that as well. Whereas GeoJSON supports, you know, whatever geometry you're working with.
25:04
And so, yeah, as I said, I covered those. And so kind of just mentioned this in the last one, but again, the great benefit of this is that the Arango search indexing process makes access to this data, and especially large amounts of data, hyperfast.
25:22
And it keeps it consistent, it is eventually consistent. So whenever you insert documents, the view will update it on its own. And not only does the access to it become really fast, because it's indexed in this way, it also provides access to geo specific functions. So geo contains, geo distance, geo in range, geo intersects.
25:44
And we'll actually take a look at how we use these. But so this allows you to make use of your geospatial data by doing relevant geo queries. And this, of course, was really, really important and useful for our Airbnb example.
26:02
And this down here is just an example of how easy it is to create it. I think this might actually, if I recall, this is actually what we use. This is all you need to set up an analyzer in Arango DB is you do this from the what we call the Arango shell or ring gosh, and you can just create an analyzer like this, and this will work.
26:20
Especially with the geo JSON, there's really not a lot of tweaking you have to do. The most common options are definitely the ones that most people will use. You can adjust some features like the S2 max level cells and that sort of stuff if you're really familiar with it and that is important to you. But we found for the most part, you're probably not going to need to change much here.
26:45
Okay, yes, as promised, we'll go ahead and take a real quick look at the site itself. And I actually have an update that I thought was interesting from overnight. I was looking at this last night, but we'll see. Okay, so here it is. I have it running on my local computer, but I also wanted to show that it's available here
27:07
on the Arango DB community GitHub. And there's a full walkthrough of how to get started with it. Especially with Dr. Compose, we had one of our community members, I believe, actually set this up for us.
27:22
Really, really awesome. Really appreciate that. I really appreciated all the help from our community members in getting this project going. But so the general, you know, the current features, I guess, are the ability to drag around on the map, get results. And one thing that was especially interesting to figure out was getting these markers to work.
27:44
So for instance, you click on the marker, it gives you information about that listing. You can click on the listing to get more information. Right now, we're actually only using a Berlin dataset. But we retrieved this data from inside Airbnb.com, and that they have information for Airbnb listings across a large number of places.
28:10
So definitely feel free to check that out. And so this is this, or you can click this over here to also get the same thing, get information about it, property information space, accommodate amenities, and so on.
28:21
And I'll kind of leave it there. But one last thing actually is that this feature, the ability to search, we use the I believe it's leaflet. Actually, it might be OpenStreetMap as far as being able to do is called geocode searching. And this was just our initial implementation. But as of last night, this worked, and I have it set up in a couple of places.
28:44
And this is just one of those things I saw today that I saw today that they have updated the policy for interfacing with this geocode service. So I created a new issue this morning. So if you did want a way to get started, helping would be to take a look at this search functionality.
29:05
One last thing was the filters. So you can filter by like things like room type, amenities, as well as pricing. And this will continually update the available amenities for the markers that we have shown here. And so we can continue to like wind down and it returns a it returns a like 100 results, I believe by default.
29:27
So and then you can get results like that and get more information about what you're looking for. So, yeah, definitely feel free to play around with that. Always, always happy for any new contributions to that as well. I believe I'll need to keep moving on.
29:45
Okay, don't want to run out of time too much here. So the initial requirements, so our initial requirements for this project were was to search an Airbnb data set, which we which we do draggable map user on go search to keep everything fast.
30:02
Search the data set using the geographic coordinates that are provided within it and filter for things like I was showing you in the filters. And of course, use a QL for everything and having multilingual support. We do have that starting with the German data set. We can do things like acknowledging the symbols and that sort of stuff within the text.
30:25
Right now, we don't have a big use for that, but that's going to come along with some of the more keyword based searches. And one of the things on the list of things that would be cool or interesting to do would be to kind of crawl the description and try and find any additional amenities or features to provide.
30:45
We did all of this through GitHub. We used to GitHub projects, which allows you to get this sort of like Trello like board that creates issues for you. We also got to use the discussions feature was really cool. That's how we found some of our community members that actually ended up creating a react front
31:02
end in parallel to our view front ends that they both rely on the same back end. That was really cool. And the data prep, I'm not going to try and speak too much to this, but I am keeping an eye on the time. So but the data prep step for me getting started with this is I'm kind of a Python guy as well as a JavaScript guy.
31:24
So getting started in a Python notebook is easy for me. And so, for example, I provided a really, really basic example of getting started with doing data exploration with maybe some really simple data tools such as pandas.
31:41
I think it's pandas and of course, like matplot to plot them. I don't know if I ended up using that, but actually I take that back. So I think primarily just pandas and NumPy for this one. But for a lot of people, that's going to be how they're getting started. But something else that I did was also I wanted to really, really see what I could do with a QL and show that you can do this data exploration, data analysis.
32:05
And then since the data is already in your database, now you can do the data transformation that you might want to do as a result of that alongside it. So, for example, actually, I'll just take a look real quick at this article. If you are interested in the data modeling and data preparation side of it, I wrote up a full
32:25
article talking about some of the challenges that we went through and some of the queries that we use. So, for example, one query that I needed to start out with was just getting information about what data types do we have within this data. You know, for someone, if you're just, you know, someone just pop some data on your virtual
32:44
desk and say, you know, figure out what we have here, what can we do with this? And this is where a lot of these questions and a lot of this article covers is like, okay, do we need to clean up this data? Do we have issues, you know, that we can detect early on in the data? Which we did.
33:02
Some things, you know, maybe there's more null values than we expected or there's a formatting problem with some of the data. And so we can start doing some of these data cleanup tasks in advance. And this was also part of our kind of like a migration. We started out with a bunch of CSV data. And so, you know, going to be more similar to like a relational database, but we needed in our, you know, NoSQL multi-model database.
33:28
How do we start combining some of these collections to be a bit to leverage like the NoSQL features a bit more? And so all of these things. And so, for example, this query here generates the attributes that we had available in our data, the types and the counts for them.
33:47
And so we would find things like like I can remember one was a description value that one of them, you know, some of them at least had numbers instead of actual text, you know, streaming descriptions. And just really being able to do this cleanup all within AQL is actually
34:05
super easy and just really convenient because the data is already located on the database. So there is that. And yeah, we'll go ahead and hop back into it. So this is our breakdown of the front end tools that we use.
34:23
This was a Vue.js front end project. We used a Vue CLI to get started and Vue material for a lot of the styling, which was great. Really, really like that. And then, of course, Axios and then Leaflet and OpenStreetMap. That's the map that we're using. And for this part, I am going to kind of do something that I'm sure is foolish and try and
34:50
do a really, really quick live coding example of how to add a Leaflet map to your Vue.js application. I wanted to do this because it was surprisingly, once I figured it out, I feel like it was surprisingly easy.
35:04
So I'm going to bring over my terminal here and we'll go ahead. I'm going to make this a bit bigger for us. How's that look? Okay. Okay, so I'm in my terminal here. And so we use the Vue CLI, which allows for scaffolding a Vue project really quickly.
35:25
And it's really, really nice. So I'm going to do Vue create, you know, just do frost example. Oh, frost example.
35:43
Okay. And for our project, we just used Vue 2. It's still like the default sort of. You could use Vue 3, I think, and this would work just fine. But we started this, you know, many, many moons ago where Vue 2 was still definitely the like stable version.
36:00
And so I think Vue 3 is probably a more sensible thing to go with at this point. Go ahead and let this spin up. And if all goes well, this is actually a fairly, fairly quick process. And I just thought it was kind of neat to show how quickly you can get started with all of this.
36:21
And then we'll take a look after this at the back end. And I really wanted to show off like kind of there's like one primary query that we ended up building out that does the bulk of the heavy lifting for our project. This should go pretty quick here. Okay.
36:41
So example. And so this is what our project structure looks like. Very basic kind of what you would expect. And I already know I need leaflet. So I'm going to go ahead and install that.
37:02
Okay. Now that it's installed, I feel this needs coming on. So if I do, I'll try and mute. I can go ahead and do a VM run serve.
37:23
Okay, so here's our standard Vue JS app. And I've got the leaflet documentation up. And then also what I will do, I actually will open this in my code editor.
37:44
And I'll bring this over in just a second here. Okay. So here is the code editor with what we have. Here's all of our project files.
38:01
We can start with going to our hello world app. And we can get rid of all of this because we won't need that. And we also won't need props for our example. We will need a methods. And we're going to call this set up. I have one over here, so I'm not totally winging it just in case.
38:25
So set up a leaflet map. You don't have to name it this. It's just whenever you start up a leaflet project, you've got to have something that sets up the map. And we're going to change this to be map. And it assigns itself to a ID,
38:43
so a div container within your project. And I want to show where we're pulling this from just because I want you to be able to do this on your own as well. So this is the leaflet.com or leafletjs.com has the really basic like getting started. Here's how simple it is to put in. And we're going to just copy that over.
39:03
And then I'm going to put that in that method that I made here. There we go. Okay. And we'll need some import statements, of course. And so the import statements that we're actually going to bring is one, there's some for leaflet here.
39:22
So these are the two that you'll need, the leaflet CSS. And then this import L, this is just the most common thing. As you see down here, this is this L we're talking about referring to leaflet itself. And then we've got two assets I'm going to bring in to show how we did the orang-db marker. Keeps those in here.
39:43
And it's complaining because we're not using it yet. And so at this point, even what we can do is get rid of some of this styling. Just do map here. And let's do like width, 50 vertical width,
40:05
50 height, we'll do the same thing. And then we'll auto. So all we're doing is really just trying to center this map and we need to give the map a size. So that's why that was important. And let's see if this is working or if it's complaining.
40:24
This is the fun part where we can see if we haven't had it working yet. Okay, it is not liking something. And I'll try and not get stuck here too much if it doesn't look like it's going to work.
40:41
That's in style. I might even just need to restart this. But what I will do as well, oh, mounted, sorry. So yes, now that we have this, what the important part is, like once our Vue.js
41:03
is ready to go, once the actual map ID has been mounted, we need to actually call this setup function. It says, okay, now that we have the map ID up here available, we need to go ahead and set it up. And that's what we've got. So now we actually have a functioning map already
41:21
in just that quick. And I'm running out of time, so I'm not going to spend a ton more time I'm just going to go ahead and copy this whole function I have over here, make all of our lives easier. Just to show you at the end of the day how easy it is to make a custom icon as well. And so that's what we've got here.
41:43
So this is how you can set up the icon. It allows you for configuring the icon size and even where its placement is, you can have an offset for the pop-up that happens with that anchor. And so that's all this is, is you were extending the leaflet icon
42:04
to be able to add our own custom icon. And we do that by passing it through right here where we added it. And this add to functionality is the general flow for working with leaflet is the ability to add to. So a lot of the stuff we'll be adding to the map
42:20
or adding to a layer of the map. And then the open street map tying in here is that open street map is actually doing all the tiling for this project. So that's the actual like art basically of the tiles here. And now you can see our new text for the pop-up using DOJ, some of the Rongo search is awesome. And our little Rongo search marker. And this is a completely draggable.
42:41
So now you can start getting the coordinates and everything that you want throughout your project. And that's how easy it is. So I really, really enjoyed working with leaflet. And I did not blunder too bad here. So I'm pretty happy with that, but okay. So let's go ahead and keep moving on
43:00
just a real quick wrap up. The last thing I want to show you is using a RongoDB with KOA. KOA is a very express like app for creating your routes and this sort of thing. And we just found it was a lot easier. One of our community members was familiar with it. And then for those that were curious,
43:20
I use Nginx and Docker on the actual server itself. So super, super quick version of this is, this is how our backend structure works. So we structured our project in workspaces. And so we have our front end for Vue and our front end for React. And then we have our RongoJS client where we import using our RongoJS.
43:42
This is the official RongoDB JavaScript driver. And we're dealing with the database object of that. And we pass this database object around throughout our application, especially the API. And all we do here is set up the configuration, which is defined in our server config here,
44:00
which just has our RongoDB URL database. The imports for this are really, really small. Looking back at our server file, we're just dealing with KOA and RongoDB. And that's it. And we can set up error handling for all the routes here.
44:21
That was also kind of a nice convenience is the ability to reuse this error handling. We have a couple of routes set up. The main one I wanted to show you, and hopefully if we have time, would be this example of map results, where we actually get our map coordinates so that map box that we're working with
44:40
that I just showed you. And then we form that. It's not quite GeoJSON coming from Leaflet, but we just form it as a bounding box, and we pass that through to our query. This is allowing us to do some of the filtering. And then we build up our query, and this is our AQL query using our AQL template tag option,
45:03
which allows you to really, allows you to have the white space and everything that you'd want. And we pass that through, and we get our cursor, which allows for, we do all in this situation because we know there's only 100 results, but if you needed to, you could do batch processing and work through the cursor as you need it, just like you would expect.
45:22
And then finally, the last thing that I did want to show you is actually this query in action. And that will sort of tie into how you can get started on your own as well. So let me pull that back up. This is using Docker Compose. I'll make sure I stop this actually. I don't want, hopefully that stopped correctly.
45:43
Okay. So this is Docker Compose. This is set up by our community member. This allows you to start a Rongodb backend, the React and Vue frontends, all with one command. And it's really, really convenient. It's a really nice way to get started. So now if we go to our same port,
46:01
this localhost 8080, that's where we were at here. So I can just refresh that actually. And there you go. But now if we go to the default Rongodb port of 8529, we'll get that for some reason. Oh, it was trying to go somewhere else. There we go. So test, there's a default password I think of 123,
46:23
or not test, sorry. That's defined in the Docker container. So be sure to check that out. And then here is our Rongodb data and collections and everything. We have this calendar that we never ended up doing anything with, but I just, I'm kind of like zipping through this because I know I'm coming real up on time here.
46:42
So, but this is that query I was just showing you in the API where we are able to use our geo contains function. We pass through that rectangle that we received, and we're iterating through all of our listings and their location information. And here is where we're actually using that geo JSON analyzer.
47:01
So we say we've got this box. We want coordinates that are contained within this box. We need this geometry information. We provide geometry information, and we want to use the geometry type of point and we want to find those points within this rectangle. And then not only do we want to do that,
47:22
once we've found them, we want to make sure that they're aligning with our filters. So some filters, for example, that I put in here is that it can be a private room or it can be an entire room and the heuristic listing room type. And you can continue filtering down. It needs to have wifi and heating. And we want our price to be between 30 and 50 and so on.
47:43
And we actually even have a sorting feature we're utilizing of a Rongo search that does this. And then we get the filters so that we can update those filter check boxes. And we return that based on the results that we get from this. And this is the object that we return. And we run that.
48:01
And oh, this is in the wrong format. There we go. And so then we get our listings and our filters. And yeah, so that's the bulk of it. And if you are interested, definitely like I said, feel free to check out that project. Reach out to us on, you can even reach out to us on GitHub
48:22
since we have that discussions feature enabled. But then some next steps that we're looking at is a really cool use case that Airbnb actually posted on their engineering blog about doing entity detection. So taking in the images that customers provide and being able to pick out potentially amenities that they didn't list or create new features that they can list on the listing
48:44
based on the entities that they're able to detect in the images. So that's a really cool sort of tie-in to some of our ML work as well. We also just need some of this work building out user profiles and completing the whole rental process flow. And I have no false expectations
49:05
that this code can be better. So spaghetti removal is probably finding some of this spaghetti code that I have created along the way and making that better. And then to add one more to that is this geocoding search that overnight stopped working. So if anybody has any suggestions
49:21
or just really wants to contribute to a project like this, definitely feel free to reach out. And we've got full instructions on how to do so in the read me as well as we have a community projects channel on our community Slack. All right, I think that's it. I think I was able to get through it and hopefully not too over time here.
49:43
And yeah, I guess that is where I'll stop. Thank you again for your time and thanks, FrostCon, for having me.