Elephant in the room
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 266 | |
| Author | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/66390 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSS4G Prizren Kosovo 2023204 / 266
10
17
23
44
45
46
47
48
49
50
52
53
80
84
85
91
110
116
129
148
164
167
169
173
174
181
182
183
186
187
199
202
204
206
209
215
241
248
265
00:00
TwitterFile viewerWeb 2.0Coefficient of determinationDigital photographyArithmetic meanNeuroinformatikWorld Wide Web ConsortiumPhysicalismObservational study
00:52
Interface (computing)Uniform resource locatorBoss CorporationInformation technology consultingData storage deviceTwitterOrder (biology)Decision theoryInformationWebsiteClient (computing)World Wide Web ConsortiumComputer animation
01:38
World Wide Web ConsortiumLine (geometry)World Wide Web ConsortiumCodeOpen sourcePresentation of a groupUsabilityCrash (computing)Software developerInformation securityCartesian coordinate systemAlgorithmClient (computing)Point cloudMultiplication signComputer animation
02:57
Uniform resource locatorNetwork topologyAnalytic setCartesian coordinate systemWorld Wide Web ConsortiumMeeting/Interview
03:13
Computer hardwareGoodness of fitWater vaporGrass (card game)
03:49
MassExpressionPoint cloudOffice suiteMereologyNeuroinformatikIntegrated development environmentComputer animation
04:45
MUDBackupComputer programmingBackupProfil (magazine)Point cloudPoint (geometry)Computer animationLecture/Conference
05:20
Demo (music)BackupSound effectServer (computing)DatabaseMultiplication signMeeting/InterviewComputer animation
06:02
Universal product codeGeometryDifferent (Kate Ryan album)MathematicsServer (computing)Mathematical optimizationSpacetimeStrategy gameInformationRevision controlDatabaseClient (computing)Cache (computing)WordTable (information)Form (programming)MereologyReplication (computing)Web pageCartesian coordinate systemUsabilityTesselationAssociative propertyMultiplication signLevel (video gaming)Position operatorIntegrated development environmentBit2 (number)Computer animationLecture/Conference
08:19
2 (number)Decision theoryCache (computing)Cartesian coordinate systemClient (computing)Plastikkarte
09:02
Hard disk driveSpacetimeScaling (geometry)NeuroinformatikDatabaseExecution unitServer (computing)Process (computing)Data miningComputer animation
09:44
Shared memoryLastteilungData storage deviceUsabilityExecution unitProcess (computing)Queue (abstract data type)NeuroinformatikComputer animation
10:29
BitControl flowExterior algebraProduct (business)Computer animationLecture/ConferenceMeeting/Interview
10:56
TesselationVector spaceComputer configurationElectric generatorPoint (geometry)File formatType theoryComputer animation
11:18
Query languageType theoryPoint (geometry)Multiplication signOpen setCache (computing)Electronic visual displayNeuroinformatikTesselationVector spaceWorld Wide Web Consortium
11:54
Shape (magazine)Computer fileFlow separationSpacetimeData storage deviceBlock (periodic table)Relational databaseFile formatDatabaseFile systemObject (grammar)Execution unitMultiplication signExterior algebraQuery languageComputer animation
12:47
Query languageGoogolMultiplication signComputer fileMultiplicationInformation securitySubject indexingGeometryData storage deviceExecution unitUsabilityQuery languageBlogObject (grammar)Software frameworkBlock (periodic table)NeuroinformatikDatabaseFlow separationPoint (geometry)Self-organizationPlug-in (computing)Lecture/ConferenceComputer animation
15:00
Water vaporWater vaporTraffic reportingData centerServer (computing)Point (geometry)Execution unitVirtual machineNumberControl flowAverageSlide ruleDifferent (Kate Ryan album)Integrated development environmentVideo gameMetropolitan area networkComputer animation
16:33
Data centerWater vaporLocal ringField (computer science)Lecture/ConferenceMeeting/InterviewComputer animation
17:32
Computer animation
Transcript: English(auto-generated)
00:08
Hi, yeah, my name is Dennis I'm a web solutions engineer at geolitics at geolitics is our Twitter handle Would be really happy if you find out more about the company who?
00:22
Affords me to come here and what we do and also I'm a poodle dad and have a Doc's Twitter account which I manage and Geolitics would be happy if they can catch up with the viewers my dog gets I'm also a geographer not the social kind meaning. I'm a study physical geography
00:41
For I don't really do that anymore. I'm just working on the computer Hardly do any geography these days, but I'm still very much concerned about our geography. Please come in Well my own Twitter handle the bosses and at geolitics, so what's geolitics? We are location intelligence company, so we actually do mostly
01:03
Social geography we consultant for retailers Franchises where to put the store so In order to do this we have a couple of very smart people mostly data scientists and a lot of data and I
01:24
Write the interface for our findings to communicate information to our clients, so Make better decisions where location matters is our mantra as you can find out on our website So what do I do? What what does the web cloud solutions engineer do?
01:44
Well, I code code as it is because I believe in no no code and I promise you there will be no code in this Presentation because there was already a lot of code for coding yesterday in my workshop. So On a good day, I write maybe ten lines of code
02:03
So that's not actually a lot of coding on a very good day. I delete ten lines of code That's that's that's the best day. So so so what do I do all the other time? Well, I don't call it and well, I worry mostly I worry. I'm very about worry about performance worry about security security is paramount for everything we do because
02:24
An application can crash that that happens Algorithms crash that that's not a disaster But if you have a security breach that could be it for the company for the career So yeah worry about my career as well. I'm very about usability not just about the clients
02:42
Usability but also our developers usability of all the code. I'm doing so open source. So Do people use it? Well, will they use it in the future? I mean this just tons of stuff to worry about always looking about my shoulder, it's
03:00
So many other brilliant applications here and not just location analytics, but like web solutions There's a shaker tree another web solution falls out. It's just so much on the market So but If the grass is green on the other side, you can bet the water bill is higher. Well, what does it mean?
03:21
Well a lot of other solutions are really really good because they have they put a lot of money in that a lot of people there that they have Good hardware good infrastructure You can't do it all alone. It's just whether it's documentation whether it's design whether it's performance So all of these things are worried about it. I
03:43
Look at what other people are doing I mean, that's why we here like we want to learn when we learn about solutions from other people and That's that's where work That's a graphic but I work at home I hardly ever go to the office and Massive benefits their massive environmental benefits mainly and you can work at home
04:03
you generate solar energy on your roof and you you're working environmentally friendly like you have a zero carbon output and a lot of us do it and that's in my opinion a part why the cloud is so successful because We we don't want to have our data on our computer so in our office we want to have it in a cloud
04:25
We want to have it. We want to have access wherever we are whenever we are And so by concur vehicle, that's a German expression. It's basically if you have to solar panel and Your counter goes backwards so you can also work at night and use the electricity for the same price
04:44
It's not legal You See panorama had recently a program it's the cloud damaging the planet. I found it was a really excellent program and the main point is Data size. There's just there's just so much data of it. It's that we keep everything
05:01
Myself, I looked at to my my Google profile the other day and it was like 10 gigabyte of stuff Like I don't need that stuff anymore So, but I don't want to delete it all and I can't go through all of it and go Yeah, I need this. I don't need this I delete it It's just so much backup everything has a backup back in backups have backups. It's just
05:24
Just piles up So so so what's the elephant in the room by the elephant in the room is the Postgres elephant and the problem is like all the data which accumulates has to be stored somewhere and has a negative effect on the environment and and why is that I mean, it's
05:41
Postgres is not the problem. Like we are the problem how we using it by why we use Postgres because it's so good It's so easy. So talking about backups. How do we make backups of Postgres? Why we create another Postgres server we create another Postgres database. So they are they're running all the time Yeah, maybe they're running all the time. Should they be but it's disgusting
06:02
I says it's a lot of geometry like we have a table with from OAS or postcodes and they have information and then we have a different table with different information, but the same geometries and We might end up like on a different server in a different region and we have the same geometry. So there might be one geometry
06:23
Optimized not optimized maybe multiple versions of it and we have the same geometry hundreds of times just Associated with different information. So we should look at this If there's one geometry, then we should only keep one form of geometry Ideally the optimized version and then the other information we just associate with the geometry when when we need it
06:44
So it's a lot of geography. So active geo replication like we have clients in China We have clients in the u.s. Sometimes the same client has a different position Do they need the active geo replication? Do we need the data in China Australia and the USA at the same time or is it?
07:00
Okay, if I access the data and it's get cached in the edge, maybe it takes a little bit longer But I mean, what's three seconds, what's four seconds, it's not the end of the word Unlike climate change, which is quite literally the end of the word. So yeah Cashing strategies we heard about it earlier. And do we need to cache everything? Do we do we need to?
07:21
You ain't gonna need it Like maybe if you zoom very far in you don't need to cache for that we can create that data on the fly The top one. Yep. Yep Definitely should cache that because you open the application your web page your map and everybody sees the same tile at the beginning So definitely that should always be cached
07:41
When data changes do we immediately have to cache it again? Do we store all the cache do we store it indefinitely? I mean, it's that's all questions Which not just concern performance and usability but but also the environment and yeah, don't just throw money at it So that's that's back in a day. We
08:02
Run out of space and we throw money at it. We buy we don't delete old stuff now We just buy a new hard drive or buy a new server Like anything can be solved with money employ people who maintain the database. It's The easiest way just making money. That's that's a difficult part It's not always the fastest who wins like as I said three seconds. That's that's not a disaster
08:26
Same as performance like and maybe the application is some not the fastest application Maybe not the best looking application and still people use it. Hopefully they do Because it's got to be smart smart caching
08:41
We we need to get a good package we need to think about all the all the things associated with the work we do and Hopefully our clients feel the same way and make a decision that we want to work with you not because it's it's The best but when maybe because it's the other concerns
09:04
See database always on how do you scale so that's worry of mine like how we go from 10 to 100 users So we go from 100 to a thousand users. Like you can't really think like this You just need to think how we do end users like the scaling it shouldn't be
09:21
You don't want to have a database always on but you want to scale it so users come and Processing computing units are scaled up and I need more space and then the space is scaled up I don't want to have like a server running with a terabyte or petabyte hard drive But I still have hard drives and it's mostly empty. It's third uses electricity
09:45
This is serverless postgres really love it. I love from neon. It's it's not fast, but it's great it's it's great usability very easy to use some and Three and you start with three but you're on the back of the load balance
10:01
She basically works like this the people who pay for it They say how many computational units they use and the people who don't pay for it when? In the feature preview on the hobby accounts and they use the computational units you pay for but you don't use so that's sharing sharing
10:21
computational units and processing and also in storage, that's So it's a great way forward. So yeah, we love it. Oh Yeah, as I said in the notes, you might have read about bit IO and why I'm not talking about bit IO But I got bought by data bricks just what I heard about this one week ago two weeks ago
10:43
So then I had to move from bit IO to neon tech which is basically the same product, which is properly the reason why? Why data bricks stopped using that also because maybe they fought their better alternatives. So What our turn it is our formats the different formats?
11:01
Not everything needs to be vector tiles. So that's mainly we use vector tiles It's quite easy to generate vector tiles with Postgres but if you have raster data now visit that's not an option to use vector tiles, but the issues are the same and But what's our options our options is a points layer so for example
11:21
If you have a point layer, but don't necessarily use vector tiles don't cache it like and use aggregation like You go in and instead of multiple tiles you're requesting which are multiple requests Which have to be processed multiple times and have to be multiple types have to be cached
11:41
There's a lot of computation just one query like one query for an aggregation of points at your viewport and Display it and for example web GL and open layers fantastic technology Alternative storage format so on our relational database Postgres, but we have file databases
12:01
But it's not the database a file file storage, so don't don't use a shape file, please but Also file storage might be a misnomer because lots of us think about s3 as a file Storage space while indeed is a block storage upload a file
12:23
But then the file is stored in blocks or object storage like MongoDB quite popular benefit for these and especially like s3 is the separation the separation of the computational units and the actual storage, so That's it. That's great. Well. Sorry very on time
12:41
plenty All right, it's okay fine. Yeah, so It's actually not that much to go as I said a big query So that is one thing you pay for it, and you have like one query and also big query. What's big query? How does it work is it a block storage is an object storage?
13:00
Actually really know what it is, but if I just before this talk I check BigQuery How does it work under the end and they said but the same things durability usability? performance security and the main thing what they said on top is separation separation of Computational units and storage the data at rest where it is, and that's how it's work, so
13:24
Multiple requests what one request is split into multiple requests going to different computational units Which work all at the same time? But they share the same storage and that can be one person or multiple person accessing the same data at the same time and they access the data and just where they need it, so
13:42
Same principle is its dual pocket. It's this new format many of you know it It's basically packet that can which can be geo, so it's it could be file It's like you don't need to database It's a file can be geo packet file, and it's a column storage so again You access what you need so could put the file and block storage and say okay. I only want the data
14:08
In my viewport and queries the data uses to indexing it's Fast it's not as fast as post person, but it's good. It's it's It's necessary fast enough like
14:23
It won't hurt you waiting and who's involved in that, but it's big companies like everybody It's involved in it these days like literally everybody which could put our name on there But I also wanted to talk about more about a geo packet, but just then at the time to Then at the time to wrote an adapter plug-in and our framework to to use geo packet, so I know about it
14:48
But I don't use it, so maybe I should in the future, but Main thing what we see here. They don't know charitable organizations like a few if you go to them You gotta have to give the money and which is pretty much my last point of what's a good environmental metric
15:04
Money, that's that's a good environmental metric because you you pay for something you pay either for men or women or metal Like a server a computational unit. That's that's not magic That's somewhere server less is still a server which runs somewhere which takes electricity which takes water
15:20
And that's what you pay for so and it's a hard market as I said data bricks Different solutions which offer the same so where do you decide to go like don't ask them? Which is the most environmental friendly company? No, you go to the cheapest like we put the less money and said they don't use the magic so what?
15:41
They're using their money for using their money for metal Actual metal dug up on in the roof and processed into machines and water man used to create the machines and running them and Electricity so it's so that's all summed up here water and electricity why why because
16:01
Very expensive to heat pools like and data centers use use a lot of water like in that BBC panorama report which Circuits us back to the beginning is a new data center gets on average An allowance of more than a billion litre a year now opposite billion is a mind-boggling big number
16:22
That's enough to fill an Olympic sized pool every day and Coming to my last life and Putting that forward forward it's so what's what's the best solution again from the BBC was an article about deep green which runs tiny data centers and
16:41
Uses not the water you swim in from the pool now They're using an oil and the oil is used to heat the pool So if you like to swim in the winter and you like to pour me water like me then that's a fantastic idea like you have the data center and you have nice warm water and climate and society friendly way to compute and just yeah, I want to end on this because I think that's
17:07
One of the best ideas like if they they open a data center in my local London field swimming pool I would Go more often swimming pools. Don't have to close down and I would give them money so that we can store our data
17:22
locally and Environmentally friendly and thank you very much. That's my talk