We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Effortless Aerial Data Management and Sharing: The DroneDB Ecosystem

00:00

Formal Metadata

Title
Effortless Aerial Data Management and Sharing: The DroneDB Ecosystem
Title of Series
Number of Parts
351
Author
Contributors
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2022

Content Metadata

Subject Area
Genre
Abstract
DroneDB is free and open source software for geospatial data storage. It provides a novel approach to store point clouds, textured models, aerial images, orthophotos and elevation models via a dynamic filesystem index. In this talk we will present DroneDB's storage approach and how you can start using it right away. We will discuss the project's architecture and roadmap. We will also perform a showcase of the project and demonstrate its most effective use cases. We will cover how DroneDB's dynamic index can be published on the web using Registry, a cross-platform open source application which provides both a friendly user interface and a RESTful API. This enables users and GIS developers alike to access and manage the underlying data. We will showcase the ddb client, a command-line interface that enables power users to manage the index and can be used to sync, share and download remote datasets in a manner inspired by git workflows. We will show how WebODM and Registry can integrate together to create a powerful and versatile workflow for 3D reconstruction using a full opensource stack.
Keywords
202
Thumbnail
1:16:05
226
242
Open sourceData storage deviceSoftwareFreewareComputer-generated imageryMereologyOpen sourceHeat transferSet (mathematics)Perspective (visual)Level (video gaming)Computer animation
Element (mathematics)Computer-generated imageryPoint cloudSubsetSelf-organizationLink (knot theory)Directed setType theoryVideoconferencingData modelPoint (geometry)Endliche ModelltheorieAdaptive behaviorView (database)MetadataFile viewerDigital rights managementSoftwareComputer fileDigital photographyExterior algebraSoftwareView (database)Point (geometry)VideoconferencingMedical imagingLevel (video gaming)Set (mathematics)Web browserEndliche ModelltheoriePoint cloudData typeOpen sourceSelf-organizationGeometryReal-time operating systemLink (knot theory)Data storage devicePolygon meshSubsetProjective planeData structureSource codeComputer animation
Link (knot theory)Polygon meshPoint cloudDynamical systemWeb browserElectric generatorComputer animation
Computer-generated imageryTesselationPoint (geometry)Digital photographyView (database)SoftwarePoint cloudOpen sourceWeb browserComputer animation
TesselationComputer fileTesselationMereologySoftwarePolygon meshData structureVirtual machineModulare ProgrammierungWebsiteInstallation artComputer animation
Enterprise architectureCommon Language Infrastructure.NET FrameworkWindows RegistrySelf-organizationData structureProcess (computing)Library (computing)Price indexGraphical user interfaceComputing platformServer (computing)File systemInterface (computing)DisintegrationVisualization (computer graphics)Type theoryMetadataKeyboard shortcutVertex (graph theory)World Wide Web ConsortiumProjective planeComputing platformSet (mathematics)ResultantDiagramCloud computingWeightClient (computing)BenutzerhandbuchCartesian coordinate systemComputer fileLevel (video gaming)Web 2.0MereologyOpen sourceSubject indexingWindows RegistryInterface (computing)Library (computing)Line (geometry)Cross-platformComputer animationProgram flowchart
Common Language InfrastructurePrice indexElectric currentInformationQuery languageComputer-generated imageryInterface (computing)Local ringVideo game consoleDatabaseMetadataMeta elementRepository (publishing)Directory serviceComputer fileWindows RegistryFile systemPhysical systemSet (mathematics)ThumbnailBuildingComputer networkAttribute grammarCloningPoint cloudElectronic mailing listLoginAuthenticationComputer configurationServer (computing)PasswordOpen setConfiguration managementComplex (psychology)Self-organizationInformationComputer fileComputing platformDatabaseSubject indexingLine (geometry)WindowInstance (computer science)Windows RegistryComplex (psychology)Configuration managementDefault (computer science)SoftwareData storage deviceCache (computing)Medical imagingCuboidSharewareCloud computingPoint cloudCartesian coordinate systemCompilation albumSet (mathematics)PasswordDiallyl disulfideMobile appComputer animation
Windows RegistryComputing platformLine (geometry)CoprocessorLink (knot theory)Computer fileWindows RegistryFunction (mathematics)Process (computing)CASE <Informatik>Source codeComputer animation
Personal digital assistantCommon Language InfrastructureSynchronizationWindows RegistryInstance (computer science)Local ringComputer fileServer (computing)Configuration managementControl flowInformationComputer networkDisintegrationUniform resource locatorComputer-generated imagerySelf-organizationBoundary value problemFunction (mathematics)Data modelCASE <Informatik>Instance (computer science)Self-organizationLocal ringComputer fileUser interfaceSet (mathematics)INTEGRALInterface (computing)MathematicsSoftwareData storage devicePoint (geometry)Polygon meshDiallyl disulfidePlug-in (computing)LoginPoint cloudFunction (mathematics)PasswordSynchronizationCloud computingProcess (computing)Remote procedure callComputer animation
File formatLevel (video gaming)Point cloudPositional notationComputer animation
Computer filePoint (geometry)Point cloudMetadataPhysical systemCommon Language InfrastructureMeasurementInterface (computing)Goodness of fitPoint (geometry)SoftwareProjective planePhysical systemCloud computingDescriptive statisticsPoint cloudMetadataCommon Language InfrastructureSet (mathematics)Entire functionSpacetimeComputer fileComputer animation
Plug-in (computing)ImplementationInterface (computing)Software frameworkFile formatProcess (computing)SoftwareVisualization (computer graphics)Data conversionOperations researchEndliche ModelltheoriePoint cloudProjective planeProcess (computing)Software developerMagnetic-core memorySoftware frameworkPoint cloudMedical imagingSet (mathematics)Plug-in (computing)SoftwarePoint (geometry)Computing platformComputer animation
Musical ensemblePresentation of a groupMultiplication signComputing platformSlide ruleComputer animation
Transcript: English(auto-generated)
Hello, everybody. I'm Luca. I'm the co-founder of Digipa and the IT director. I'm the second active contributor to OpenDroneMap, so many of the tools that you are using comes from the global effort part of me. What is DroneDB and why are we here? We were trying
to find a need and we tried to address this problem from the open source perspective.
What are we doing today to transfer geospatial data and, in particular, large datasets? We are doing this. We are uploading to Google Drive or Dropbox or putting on GitHub or even worse on WeTransfer. These usually end up like this, like seven hours of wait for
an enormous dataset and I only wanted to check a single image or I wanted to check if my drone gave me the right georeferenced images or was my overlap sufficient to process?
All these questions get really annoying when you don't have a way to inspect and verify your dataset without actually downloading it. Moreover, you have very large files like
point clouds and meshes and orthophotos that should be able to be viewed and inspected directly in the browser without downloading like 20 gigs of the file. So this is the
need that we identified. We have all the aerial data types that we actually support. There are images, videos, orthophotos, elevation models, 3D models and panoramas point clouds and what we want to do is browsing data, inspecting this data and be
able to view the orthophotos and the meshes directly in the browser. We looked up for commercial solutions and there were many commercial solutions but we hate commercial solutions. We need to develop a tool that was completely open source and
accessible to everybody. So we started developing this with the organisation and dataset structure in mind. So somebody can create an organisation, upload a dataset, set it private or public and you can just share a link to the dataset or a subset of the dataset and just
view it without any problems. So what's DornDB? It's all of these things. It's open source. We are actively developing it every day. It's well supported so if you need some
support of it, we have commercial support and open source support without any problem and we have a very comprehensive documentation and it's really really close to be the alternative to the commercial software. You can go to dronedb.app and read about it. It will be a really
really fine piece of software. So this is what we developed. I had to embed the video because I cannot go online but let's see what you can do with the map for example. We have an autofoto
and point cloud. You can select the images. You can see them on the map. The path that you took with your drone and this is a really interesting thing. You can actually
see the geo projection of your autofoto from the camera that was on your drone. So you can inspect the autofoto and your flight path really well. You can do that in real
time. This is dynamically generated. It's not stored anywhere so it's really flexible approach. We can do the same thing. It's a dynamic tiler. You can zoom in and load more details about that and then you can do the same with the autofoto. So you can
zoom in and increase the details. You can do the same with the mesh for example using
Nexus. This is a 3JS browser and it's all dynamic generated. So you just give somebody a link and he will be able to browse your mesh and your point cloud. We actually
have a point cloud example here. I think this is the first one. This is the autofoto and this is the point cloud. Can you find out what's the tool? It's pottery. It's a
really common browser and you can browse the point cloud and the dynamic tiler will download just the chunks that you need to put in your view port and optimize it for online consumption. This way you can actually browse like 40, 50 gigabytes of point cloud
without breaking a sweat. You can do that from your phone. This is not revolutionary but it's amazing to be able to do that with just open source software. This is a
same thing with the dynamic tiling. The great part of this is that you don't need to have
anything installed in your machine. You don't have to install large software packages or pay licenses. You just drop in the file. The tool will build the structure and you can view it like this. How this works? It's magic. There is something really
interesting behind it. The three main projects that our ecosystem is composed of are the actual DDB lib that contains even the CLI, that's the common line interface to interact
with this index. So you take a data set, you index the data set with this tool and that generates all what's needed behind it to be able to be visualized. We have the registry cloud application. It's a .net open source application with cross platform. It
has the client application written in Vue.js and it contains the map and the files explorer
and so on. This is the three key projects of our ecosystem and they work together to offer you the results that you saw. This is a more in-depth diagram of what's behind
the curtain but we have like many moving parts but it works like a charm. What should we do to get started with the drone DB ecosystem? It's actually really simple. We can start with the CLI. We can download it, read the documentation. It's just a few comments.
You go inside your data set folder, you initialize it, you add your files to the index like Git, for example. You can tag your data set with an organization slash data set so to give it a name and you can query info about the files that were added to the index.
After you do all these things, you can share it with a public instance of the cloud applications registry or with a locally hosted one. There are really few comments but really
there are many more comments to do all these things and it's a community effort to bring you this kind of software but we focused on making it really simple to use. How can you run the cloud application with just one common line? If you have Docker,
you can just run it from the Docker images, the front Docker image and give it a storage folder when it will store all its data and it's okay. It will run, you will be able to log in with default password, default credentials and you will have all the features
that were shown in the demo. You can even run it natively because thanks to Microsoft, we have .NET that is able to be compiled natively across many platforms. So we can
actually compile it for Apple M1 to Linux to Windows. Every platform is supported and you can run it just like this. There is a possibility to add a more complex configuration, for example to add a cache layer with Redis or to connect to a MySQL or MariaDB database
or Postgres changing the configuration and you can do pretty much everything editing the config but out of the box it works with just one common line. This is a startup
output of registry and it will give you some useful links that will help you develop for this platform. For example, it will give you a swagger endpoint to document the API
and there are health monitoring solution and the background jobs processor because when you upload for example a last file, it will get built with EBT and this process is monitored
by our back-end solution. We can actually see a use case for this. For example, this happens really often. We have a local data set we just took from our drone and we need
to see and inspect it on the cloud. You can just add the files and tag it with a remote tag, organisation tag. This will make you able to push it to the remote endpoint
and inspect it at the remote endpoint. It's great because you can edit your local data set and then push the changes to the remote one and vice versa. You can remove and edit the files in the cloud application and pull those changes from your CLI. It's
useful because sometimes you have a large data set, you push it, you do some modifications for example, you move folders, you delete bad files, you find out that a couple of files were broken or there were some problems, you can do that in the cloud application
and then pull those changes directly on your local data set and this can be done from many people, so you can actually sync the data set across the entire organisation
both remotely and locally. We are really glad to have this and this is the integration with WebODM. I don't know if you know it. I think you know it. It's a software and it's the web interface for ODM. We have a plugin for drone DB. You can configure the login
and password of the instance and you can import the files directly from drone DB. It will put them, work on them and after that is the pull interface. You choose the organisation,
you choose the data set, you choose the folder and you go verify and import. After that, after the process, you can share to drone DB. So you don't need to have very large storage on your WebODM instance because you can just share that to drone DB and let him
take care of it and that's the output. This is the output of ODM with the orthophoto, the mesh and the point cloud. We have a really tight roadmap because we received many requests about support of file types and data notation and many other things,
but we are focusing on a few of them that we consider really important for our software. The first one is a metadata system. So you can in the cloud application tag both a file,
a folder, the entire data set, a point in the space with data. Here is an example about writing who flowed the drone, for example the pilot or if there was good weather or bad weather or for example to add some kind of notes about what we are doing and the description of the
project. Half of this feature is already present in the CLI. We are just writing the cloud interface to use that. But I think this will be really useful for our software.
This is another thing that we were thinking about. It's a plug-in framework. The software alone is not sufficient. We need a way to expand it without taking inside the core, for example proprietary things or not crucial things. So you write the plug-in framework,
you allow other developers to attach to your software and improve on it. This is the final one. It's the processing pipeline. It will be amazing if you can do
everything inside drone DB. You upload the images directly and you process them in a pipeline. You can actually convert the point cloud, you can cut the point cloud, you can export
everything from the data set directly in drone DB. So adding a processing pipeline capability, we could achieve this and it would be amazing to leverage projects like GDAL, PDAL and ODM integrating them in the platform. I'm sorry if I went so fast on this presentation but
the time is really tight. The first time I tried to roll the slides there were like 100 slides and now there are 30. So I think I missed many things. But if you have any questions, I'm here to ask them. Thank you very much.