From the field to the desk - end to end mobile data capture in an enterprise environment.
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 351 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/69153 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produktionsjahr | 2022 |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
00:00
UnternehmensarchitekturProgrammierumgebungTopologieStandardabweichungRechnernetzSingle Sign-OnW3C-StandardSpezialrechnerServerInformationsspeicherungBefehlsprozessorDefaultVolumenSynchronisierungUmwandlungsenthalpieOffene MengePunktPolygonAttributierte GrammatikRelation <Informatik>LastART-NetzWeb SiteDigitalfilterOrdnung <Mathematik>WidgetMultiplikationNumerisches VerfahrenDialektMini-DiscCachingTypentheorieSchlüsselverwaltungVideokonferenzDatenfeldStetige FunktionPerspektiveMotion CapturingEDV-BeratungBenutzerbeteiligungProjektive EbeneSoftwareART-NetzPasswortServerBitSynchronisierungDatenfeldMAPBildgebendes VerfahrenDatenbankCachingMapping <Computergraphik>TabelleMengeMathematikKonfiguration <Informatik>TopologiePolygonUnternehmensarchitekturPerspektiveEinfach zusammenhängender RaumElektronische PublikationZusammenhängender GraphSkriptspracheMini-DiscWeb ServicesBandmatrixUmsetzung <Informatik>Inverser LimesRechenschieberSchlüsselverwaltungSystemplattformWald <Graphentheorie>Minkowski-MetrikVideokonferenzPunktwolkeRepository <Informatik>VersionsverwaltungUmwandlungsenthalpieMereologieEinfügungsdämpfungApp <Programm>StrömungsrichtungMomentenproblemNotepad-ComputerInformationArithmetisches MittelData-Warehouse-KonzeptFlächeninhaltWeg <Topologie>Prozess <Informatik>Überlagerung <Mathematik>DifferenteMultiplikationsoperatorEinfache GenauigkeitGüte der AnpassungAutorisierungAttributierte GrammatikEDV-BeratungInformationsspeicherungBefehlsprozessorRechter WinkelPunktTypentheorieFunktionalSondierungGebäude <Mathematik>DefaultQuick-SortTLSModallogikTUNIS <Programm>BimodulURLChecklisteCASE <Informatik>Mobiles InternetOffene MengeQuaderStandardabweichungKonfigurationsraumProgrammierumgebungMotion CapturingEinsWeb SitePlug inE-MailGamecontrollerVerkehrsinformationSpezifisches VolumenElektronischer ProgrammführerFokalpunktFehlermeldungQuellcodeWidgetStellenringMailing-ListeLateinisches QuadratDiagrammSchreib-Lese-KopfEin-AusgabeBildschirmmaskeFamilie <Mathematik>Vorzeichen <Mathematik>DatentypTelekommunikationSprachsyntheseSkalenniveauInstantiierungRelativitätstheorieMultiplikationSelbst organisierendes SystemKlassische PhysikFormale SpracheSoundverarbeitungTrennschärfe <Statistik>ResultanteThumbnailZahlenbereichArchitektur <Informatik>Digitale PhotographieKartesische KoordinatenElementargeometrieComputeranimationFlussdiagramm
Transkript: Englisch(automatisch erzeugt)
00:03
Is this on? OK. Hello, everyone. I'm not even allowed the TV remote control at home. So now I've got the mic. I feel like I'm the king. So this is from the field to the desk. It's a kind of project run through of what we've done recently for a customer using a mobile data capture
00:24
project, no surprise, using merging maps as part of the architecture. My name's Andrew Bailey. I work for Aston Technology. It's a UK company. And my Twitter handle is, this is so regretful,
00:43
it's AndyBMapMan. That's kind of OK when I talk to my family, when I'm in front of you guys. It's not good. It sounds like I'm a crap superhero that works in 2D only.
01:00
Is it? OK. I'm not sure. Is that working? It's not working either. Oh, hang on. Did I press the right button?
01:25
So yes, this is a run through of what I want to talk about. So self-hosted merger maps. So this is very much a dummy's guide without the instructions, because the instructions are so good that Lutro have provided. So I just want to talk about, I suppose, the pain
01:41
points of what we've done for this particular customer. It's set up within an enterprise environment. It's regarding a tree survey project. So there's a QGIS project involved. And whilst we're doing the project, the forest agency in the UK released a tree, really pacific, an individual
02:06
tree standard for a survey whilst we're doing this. So there's a side project to that at the end, which I'll cover, and a bit about the grumbles and lessons learned. So yeah, Aston Technology, we're based in the UK.
02:22
We're all kind of sitting at home now due to the shock of COVID. We specialize in obviously geospatial and metadata consultancy to those who want it. Our focus for customers is predominantly government bodies in the UK, and that would include about 60 local
02:42
authorities, municipals, of which this project is about one of them. So a typical customer or common customer is this local authority. A point of contact for them is sometimes head of IT or typically the GIS lead for that. In this particular project, that wasn't the case.
03:02
We were dealing directly with the environmental service department, the tree survey people. They put out the piece of work for Tender. We caught it and delivered. So hopefully you can see a very simple enterprise architecture diagram.
03:20
At the top, we've got the customer network. At the bottom, we've got our virtual private cloud, which we host in Amazon AWS. There's no real surprises. At the top, we've got an ETL piece of software that uses OGR under the bonnet. That pushes data into the spatial data warehouse from internal data connections and so forth.
03:42
And the customer, the GIS people in the organization, they'll access that via QGIS or MapInfo version 8, as it turned out. And we also provide for them for internal use a web GIS with single sign-on, and they can access that.
04:05
So they can do reporting on that. They can do feature editing, which is quite nice. And they'll read from the spatial data warehouse. So Mergin, that's in a self-hosted environment. We've plonked it in there. It's an Ubuntu server.
04:21
The specs will come up at some point. And the idea is that people out in the field are using 3G on a 3G network. I think I was told they're using iPads, but I can't quite believe that. But anyway, mobile devices of some kind, they'll download the project we've developed.
04:40
And they go off and do a tree survey, synchronize. That data goes back into the spatial data warehouse via Mergin's DB sync, which we'll cover in a minute. And the people that are making use of the web GIS, they can see that data as it comes in. And both parties can edit. And both Mergin project and the spatial data warehouse are
05:06
kept in sync with each other. So here's a little bit more detail about the Mergin server. When you follow the instructions on Mergin Maps
05:23
GitHub, the instructions are great. And I had no prior Docker experience to this. If you follow them, you'll end up with three Docker instances, the Mergin, the application, Mergin DB, which is the internal database.
05:40
And that just keeps track of users and projects and permissions on those projects. And Redis, which I believe is a caching function for Mergin. And you'll also want this DB sync module. So all that needs is a work with one project and one
06:07
Geopackage in that project. And that will keep sync with one schema in the PostGIS database. So if you've got more than one project, you can have more than one DB sync. So that's continually monitoring synchronizations.
06:22
And on the right-hand side, we've got the web GIS. So the people over there, they can edit data as well. And DB sync is going to push that back to Mergin's project. What's nice about that is that the web GIS can make
06:43
use of a web processing service for editing. So we've got that plugged into Ordnance Survey's topographic area master map data set. Essentially, it's polygons that cover the UK if you don't know, and some of those are going to be woodland classified polygons.
07:01
So they might do a rough drawing of a polygon out in the field. And they'll come back in. They can use the web GIS to replace that or to combine it with the master map topographic area plug-in. And that's just going to be, in the UK talk, it's the bee's knees.
07:20
It's the polygon that they want to capture, really. Also, because they want to extract information. They might want to run reports from the web GIS based upon the trees that they've gathered in the field. They need to have the images. So when you click on a tree, you'll get information, all
07:41
the attributes stored about that, plus the image. So we have a Python script that routinely just queries Mergin's API, extracts the entire project, including the images, and then runs another script just to convert those to thumbnails, just to keep down on bandwidth. And I think the next slide just demonstrates nothing
08:04
surprising about the left-hand side, really. Nothing surprising about the right eye, though. It's just that's our web GIS. You can see I've clicked on a tree. And up comes the information about it. And they can further click to see the full picture.
08:21
This is the current server specification we've gone for. The project's not currently live at the moment. We're just in the process of signing it off. So we're not entirely clear about their actual demands. We've got an idea of the size of the project. I think this is probably overkill.
08:42
But when we have a bit more data about CPU performance, we might reduce this. So it's certainly not under-qualified to perform its duty. I suppose the good advantage about self-hosting is that you can whack on the storage size that you want.
09:03
One of the downsides is that you have to create the user accounts yourself. So actually, this is a bit of a unnecessary slide. Just really a checklist. That's the URL that you need to go to to get hold of the repo, and that includes the fantastic instructions from
09:23
literature on how to get yourself sorted. In this particular project, the people out in the field, they're not using a VPN. So we have to open the doors for 443. Make sure you've got disk space alarms, all the normal stuff, including the necessary site certificate.
09:43
When you download the repo, the first thing you need to do is look at the Docker Compose YAML file. So there's lots of config options in there. I just put some sort of key ones on there. The first one that you need to do is turn off user self-registration.
10:02
The main merging website has that enabled, and any user can go on, create an account, and they get allocated some storage space. Well, we don't want it in this case, so we can disable it. We can also increase the default storage size. That's on a project level. And set up some mail server settings, so in case they
10:23
forget passwords. Map your folders to volumes in case everything goes a bit belly up and the containers go down, you might have to restart them. You don't want to lose all your hard effort.
10:41
So the db-sync is even easier. As I said, it's one container, one project, one geopackage, one schema. If you have lots of them, then name them accordingly, the container name. Ours is called like tree survey db-sync or something like that.
11:01
Because you're going to end up with however many projects you've got in your set up, and you'll have db-syncs running for each one. And it just contains merging connection details. In this case, it's localhost straightforward, the geopackage in question, and where does it go into the post-gis connection and schema.
11:22
So as I said, it's a tree survey project based in the UK. So for those who don't know, any merging project starts with a QGIS project. So in this case, it's British National Grid. We use the grid file for conversion between last and long
11:42
to get the high precision conversion into Eastings and Northings. That does cause an issue in merging maps, the app. You do get an error message. I think Lutra are aware of this. You just skip through it, and you only see it once.
12:00
So it's not a heartache. The base mapping is all WMS. It could otherwise be offline, but the guys that we're working with, they've all got grand 3G connections, so they're all good. They're using Ordnance Survey's WMS that we happen to host, but they can also flip to aerial
12:21
WMS using map themes. That one comes from Airbus, I believe, as part of their local government agreement. And no surprises, trees are points, woodland areas are polygons. The other thing that they wanted was a means of looking up a location.
12:41
So in emerging maps app, you can search against all attributes of all layers. So we put in essentially an invisible layer, acting as a gazetteer, and that is a data set from the Ordnance Survey called OS Open Names.
13:00
And that's actually really fit for purpose in this case. It has road names, postcodes, and prominent buildings. This particular customer does surveys for highways, parks, and particularly schools for insurance purposes.
13:20
So having the schools listed as prominent buildings is a really nice feature. So they can type something in, and off they go to the right location. So when you're controlling designing your form, that's
13:41
going to appear in Merging. You can make use of these widgets in QGIS, which are hopefully that you're all aware of, but they're really great for fine tuning and making that input of data less manual. So the first two I've got here are combo box widgets. So essentially to create an option box in Merging or
14:02
Qfield or others. There's one called value relation, and that will essentially use a non-spatial table in the project that you can assign as the source for that particular field. So tree species is the one here that we've got. Because it's such a long list.
14:23
So that's really useful. If it's pouring down in rain, and you're in the north of England with lightning coming in, you don't want to remember how to spell the Latin name for some spruce tree or something. So straightforward. The other option is to use a value map widget, which you
14:42
can self-define the values. Typically, you'd use that for a smaller selection option, such as the health, bad, good, or medium, that kind of thing. And away you go. So the one on the right, value relation, that's obviously linked to the table. If the table changes, then the options change where
15:00
value map is more of a static one. More widgets, new numbers, dates, photos, multi-line. Multi-line's important for being in the field. Issues experienced. Multi-polygon's crept in somewhere, and that really upsets DbSync, because it is expecting polygons to be in
15:23
the PostGIS table. So that tripped it up. So we converted all to multi-polygon. And that was fine. Just before I went on holiday two, park asterisks, which I recommend, the Redis cache component failed. I've yet to look at the details, but that happily then
15:42
resulted in filling up the whole disk space. So that's worth further investigation, but also put on disk alarms and perhaps Docker monitoring. And the modification of polygons is a bit limited with the app at the moment. You can split into two features of woodland-polygon, then delete one.
16:02
That's the way that you would fine tune something. So we found the Web GIS quite a useful tool for them, so it was a good work around. The lessons learned. Really, if we're looking from a profit and loss perspective on this project, because we're a commercial company, a huge
16:22
amount of time was spent on communication, because this customer isn't a typical one of ours. Normally, we're speaking to a GIS person. They know about primary keys. They know about data types. And this team's expertise lay in environmental sciences.
16:40
So we had to be aware of language and terminology, the differences within them. But we found that providing personalized documentation, though that's quite costly to do, videos, knocking out videos to indicate a solution or a work around to something
17:02
or how to show off a feature or something, is actually really easy and time-effective for someone in a consultancy to do. It takes five minutes, and it's out the door with documentation and worrying about particular spelling and sentences and all that kind of stuff.
17:22
It's straightforward. So the end result. They are reasonably happy. It kind of goes up and down, to be honest. But at the moment, they're happy. And they can edit via mobile, Web GIS, or QGIS. They won't, but they like MapInfo.
17:40
Two minutes, right. And they do see the opportunities for mobile data capture beyond that. So it's a good example within the authority. Hopefully, we'll see some more projects come in. As I mentioned, the forest research, they released a standard.
18:02
Was it last year, I think, at some point. It's all COVID-y. So I've essentially sort of brought that into a project which is publicly available on the Merging Maps site. So if you want to go off that, you can clone it, make use of it. You can download it.
18:20
You can put it into Qfield Cloud or other competitor mobile data platforms. And it's based upon, as I say, this individual tree data standard. Resources, I've mentioned before, Merging Maps Docs. It's a great place to start. Also, the instructions in the repo are thoughtless.
18:41
And they also have Lucha provide a Slack channel to provide assistance. And I'm pretty grateful for them. And that's it. Thank you to Lucha, and thank you to forest research.