Effortless Aerial Data Management and Sharing: The DroneDB Ecosystem
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69138 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
FOSS4G Firenze 2022106 / 351
1
7
13
22
25
31
33
36
39
41
43
44
46
52
53
55
58
59
60
76
80
93
98
104
108
127
128
133
135
141
142
143
150
151
168
173
176
178
190
196
200
201
202
204
211
219
225
226
236
242
251
258
263
270
284
285
292
00:00
Open sourceData storage deviceSoftwareFreewareComputer-generated imageryMereologyOpen sourceHeat transferSet (mathematics)Perspective (visual)Level (video gaming)Computer animation
00:54
Element (mathematics)Computer-generated imageryPoint cloudSubsetSelf-organizationLink (knot theory)Directed setType theoryVideoconferencingData modelPoint (geometry)Endliche ModelltheorieAdaptive behaviorView (database)MetadataFile viewerDigital rights managementSoftwareComputer fileDigital photographyExterior algebraSoftwareView (database)Point (geometry)VideoconferencingMedical imagingLevel (video gaming)Set (mathematics)Web browserEndliche ModelltheoriePoint cloudData typeOpen sourceSelf-organizationGeometryReal-time operating systemLink (knot theory)Data storage devicePolygon meshSubsetProjective planeData structureSource codeComputer animation
05:19
Link (knot theory)Polygon meshPoint cloudDynamical systemWeb browserElectric generatorComputer animation
05:42
Computer-generated imageryTesselationPoint (geometry)Digital photographyView (database)SoftwarePoint cloudOpen sourceWeb browserComputer animation
06:45
TesselationComputer fileTesselationMereologySoftwarePolygon meshData structureVirtual machineModulare ProgrammierungWebsiteInstallation artComputer animation
07:20
Enterprise architectureCommon Language Infrastructure.NET FrameworkWindows RegistrySelf-organizationData structureProcess (computing)Library (computing)Price indexGraphical user interfaceComputing platformServer (computing)File systemInterface (computing)DisintegrationVisualization (computer graphics)Type theoryMetadataKeyboard shortcutVertex (graph theory)World Wide Web ConsortiumProjective planeComputing platformSet (mathematics)ResultantDiagramCloud computingWeightClient (computing)BenutzerhandbuchCartesian coordinate systemComputer fileLevel (video gaming)Web 2.0MereologyOpen sourceSubject indexingWindows RegistryInterface (computing)Library (computing)Line (geometry)Cross-platformComputer animationProgram flowchart
08:50
Common Language InfrastructurePrice indexElectric currentInformationQuery languageComputer-generated imageryInterface (computing)Local ringVideo game consoleDatabaseMetadataMeta elementRepository (publishing)Directory serviceComputer fileWindows RegistryFile systemPhysical systemSet (mathematics)ThumbnailBuildingComputer networkAttribute grammarCloningPoint cloudElectronic mailing listLoginAuthenticationComputer configurationServer (computing)PasswordOpen setConfiguration managementComplex (psychology)Self-organizationInformationComputer fileComputing platformDatabaseSubject indexingLine (geometry)WindowInstance (computer science)Windows RegistryComplex (psychology)Configuration managementDefault (computer science)SoftwareData storage deviceCache (computing)Medical imagingCuboidSharewareCloud computingPoint cloudCartesian coordinate systemCompilation albumSet (mathematics)PasswordDiallyl disulfideMobile appComputer animation
11:23
Windows RegistryComputing platformLine (geometry)CoprocessorLink (knot theory)Computer fileWindows RegistryFunction (mathematics)Process (computing)CASE <Informatik>Source codeComputer animation
12:07
Personal digital assistantCommon Language InfrastructureSynchronizationWindows RegistryInstance (computer science)Local ringComputer fileServer (computing)Configuration managementControl flowInformationComputer networkDisintegrationUniform resource locatorComputer-generated imagerySelf-organizationBoundary value problemFunction (mathematics)Data modelCASE <Informatik>Instance (computer science)Self-organizationLocal ringComputer fileUser interfaceSet (mathematics)INTEGRALInterface (computing)MathematicsSoftwareData storage devicePoint (geometry)Polygon meshDiallyl disulfidePlug-in (computing)LoginPoint cloudFunction (mathematics)PasswordSynchronizationCloud computingProcess (computing)Remote procedure callComputer animation
14:52
File formatLevel (video gaming)Point cloudPositional notationComputer animation
15:06
Computer filePoint (geometry)Point cloudMetadataPhysical systemCommon Language InfrastructureMeasurementInterface (computing)Goodness of fitPoint (geometry)SoftwareProjective planePhysical systemCloud computingDescriptive statisticsPoint cloudMetadataCommon Language InfrastructureSet (mathematics)Entire functionSpacetimeComputer fileComputer animation
16:03
Plug-in (computing)ImplementationInterface (computing)Software frameworkFile formatProcess (computing)SoftwareVisualization (computer graphics)Data conversionOperations researchEndliche ModelltheoriePoint cloudProjective planeProcess (computing)Software developerMagnetic-core memorySoftware frameworkPoint cloudMedical imagingSet (mathematics)Plug-in (computing)SoftwarePoint (geometry)Computing platformComputer animation
17:19
Musical ensemblePresentation of a groupMultiplication signComputing platformSlide ruleComputer animation
Transcript: English(auto-generated)
00:00
Hello, everybody. I'm Luca. I'm the co-founder of Digipa and the IT director. I'm the second active contributor to OpenDroneMap, so many of the tools that you are using comes from the global effort part of me. What is DroneDB and why are we here? We were trying
00:27
to find a need and we tried to address this problem from the open source perspective.
00:41
What are we doing today to transfer geospatial data and, in particular, large datasets? We are doing this. We are uploading to Google Drive or Dropbox or putting on GitHub or even worse on WeTransfer. These usually end up like this, like seven hours of wait for
01:09
an enormous dataset and I only wanted to check a single image or I wanted to check if my drone gave me the right georeferenced images or was my overlap sufficient to process?
01:27
All these questions get really annoying when you don't have a way to inspect and verify your dataset without actually downloading it. Moreover, you have very large files like
01:46
point clouds and meshes and orthophotos that should be able to be viewed and inspected directly in the browser without downloading like 20 gigs of the file. So this is the
02:02
need that we identified. We have all the aerial data types that we actually support. There are images, videos, orthophotos, elevation models, 3D models and panoramas point clouds and what we want to do is browsing data, inspecting this data and be
02:22
able to view the orthophotos and the meshes directly in the browser. We looked up for commercial solutions and there were many commercial solutions but we hate commercial solutions. We need to develop a tool that was completely open source and
02:43
accessible to everybody. So we started developing this with the organisation and dataset structure in mind. So somebody can create an organisation, upload a dataset, set it private or public and you can just share a link to the dataset or a subset of the dataset and just
03:08
view it without any problems. So what's DornDB? It's all of these things. It's open source. We are actively developing it every day. It's well supported so if you need some
03:23
support of it, we have commercial support and open source support without any problem and we have a very comprehensive documentation and it's really really close to be the alternative to the commercial software. You can go to dronedb.app and read about it. It will be a really
03:46
really fine piece of software. So this is what we developed. I had to embed the video because I cannot go online but let's see what you can do with the map for example. We have an autofoto
04:05
and point cloud. You can select the images. You can see them on the map. The path that you took with your drone and this is a really interesting thing. You can actually
04:20
see the geo projection of your autofoto from the camera that was on your drone. So you can inspect the autofoto and your flight path really well. You can do that in real
04:42
time. This is dynamically generated. It's not stored anywhere so it's really flexible approach. We can do the same thing. It's a dynamic tiler. You can zoom in and load more details about that and then you can do the same with the autofoto. So you can
05:13
zoom in and increase the details. You can do the same with the mesh for example using
05:23
Nexus. This is a 3JS browser and it's all dynamic generated. So you just give somebody a link and he will be able to browse your mesh and your point cloud. We actually
05:42
have a point cloud example here. I think this is the first one. This is the autofoto and this is the point cloud. Can you find out what's the tool? It's pottery. It's a
06:06
really common browser and you can browse the point cloud and the dynamic tiler will download just the chunks that you need to put in your view port and optimize it for online consumption. This way you can actually browse like 40, 50 gigabytes of point cloud
06:32
without breaking a sweat. You can do that from your phone. This is not revolutionary but it's amazing to be able to do that with just open source software. This is a
06:54
same thing with the dynamic tiling. The great part of this is that you don't need to have
07:04
anything installed in your machine. You don't have to install large software packages or pay licenses. You just drop in the file. The tool will build the structure and you can view it like this. How this works? It's magic. There is something really
07:29
interesting behind it. The three main projects that our ecosystem is composed of are the actual DDB lib that contains even the CLI, that's the common line interface to interact
07:44
with this index. So you take a data set, you index the data set with this tool and that generates all what's needed behind it to be able to be visualized. We have the registry cloud application. It's a .net open source application with cross platform. It
08:14
has the client application written in Vue.js and it contains the map and the files explorer
08:23
and so on. This is the three key projects of our ecosystem and they work together to offer you the results that you saw. This is a more in-depth diagram of what's behind
08:42
the curtain but we have like many moving parts but it works like a charm. What should we do to get started with the drone DB ecosystem? It's actually really simple. We can start with the CLI. We can download it, read the documentation. It's just a few comments.
09:05
You go inside your data set folder, you initialize it, you add your files to the index like Git, for example. You can tag your data set with an organization slash data set so to give it a name and you can query info about the files that were added to the index.
09:26
After you do all these things, you can share it with a public instance of the cloud applications registry or with a locally hosted one. There are really few comments but really
09:40
there are many more comments to do all these things and it's a community effort to bring you this kind of software but we focused on making it really simple to use. How can you run the cloud application with just one common line? If you have Docker,
10:05
you can just run it from the Docker images, the front Docker image and give it a storage folder when it will store all its data and it's okay. It will run, you will be able to log in with default password, default credentials and you will have all the features
10:26
that were shown in the demo. You can even run it natively because thanks to Microsoft, we have .NET that is able to be compiled natively across many platforms. So we can
10:42
actually compile it for Apple M1 to Linux to Windows. Every platform is supported and you can run it just like this. There is a possibility to add a more complex configuration, for example to add a cache layer with Redis or to connect to a MySQL or MariaDB database
11:10
or Postgres changing the configuration and you can do pretty much everything editing the config but out of the box it works with just one common line. This is a startup
11:25
output of registry and it will give you some useful links that will help you develop for this platform. For example, it will give you a swagger endpoint to document the API
11:48
and there are health monitoring solution and the background jobs processor because when you upload for example a last file, it will get built with EBT and this process is monitored
12:04
by our back-end solution. We can actually see a use case for this. For example, this happens really often. We have a local data set we just took from our drone and we need
12:24
to see and inspect it on the cloud. You can just add the files and tag it with a remote tag, organisation tag. This will make you able to push it to the remote endpoint
12:45
and inspect it at the remote endpoint. It's great because you can edit your local data set and then push the changes to the remote one and vice versa. You can remove and edit the files in the cloud application and pull those changes from your CLI. It's
13:08
useful because sometimes you have a large data set, you push it, you do some modifications for example, you move folders, you delete bad files, you find out that a couple of files were broken or there were some problems, you can do that in the cloud application
13:25
and then pull those changes directly on your local data set and this can be done from many people, so you can actually sync the data set across the entire organisation
13:41
both remotely and locally. We are really glad to have this and this is the integration with WebODM. I don't know if you know it. I think you know it. It's a software and it's the web interface for ODM. We have a plugin for drone DB. You can configure the login
14:04
and password of the instance and you can import the files directly from drone DB. It will put them, work on them and after that is the pull interface. You choose the organisation,
14:21
you choose the data set, you choose the folder and you go verify and import. After that, after the process, you can share to drone DB. So you don't need to have very large storage on your WebODM instance because you can just share that to drone DB and let him
14:41
take care of it and that's the output. This is the output of ODM with the orthophoto, the mesh and the point cloud. We have a really tight roadmap because we received many requests about support of file types and data notation and many other things,
15:07
but we are focusing on a few of them that we consider really important for our software. The first one is a metadata system. So you can in the cloud application tag both a file,
15:22
a folder, the entire data set, a point in the space with data. Here is an example about writing who flowed the drone, for example the pilot or if there was good weather or bad weather or for example to add some kind of notes about what we are doing and the description of the
15:47
project. Half of this feature is already present in the CLI. We are just writing the cloud interface to use that. But I think this will be really useful for our software.
16:04
This is another thing that we were thinking about. It's a plug-in framework. The software alone is not sufficient. We need a way to expand it without taking inside the core, for example proprietary things or not crucial things. So you write the plug-in framework,
16:29
you allow other developers to attach to your software and improve on it. This is the final one. It's the processing pipeline. It will be amazing if you can do
16:45
everything inside drone DB. You upload the images directly and you process them in a pipeline. You can actually convert the point cloud, you can cut the point cloud, you can export
17:00
everything from the data set directly in drone DB. So adding a processing pipeline capability, we could achieve this and it would be amazing to leverage projects like GDAL, PDAL and ODM integrating them in the platform. I'm sorry if I went so fast on this presentation but
17:24
the time is really tight. The first time I tried to roll the slides there were like 100 slides and now there are 30. So I think I missed many things. But if you have any questions, I'm here to ask them. Thank you very much.