Bestand wählen
Merken

iGUESS – The Good, The Bad And The Ugly Of Developing A Distributed Modeling System For European Cities

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
Ts of the session and the missing chance for doing a ranch I'll try to make this a good presentation and thanks to Michael for letting me user's computer at the of us go OK so my hands because I can't I would be honored to the Institute in Luxembourg and here today to talk about I guess which is distributed modeling platform the son of open Web services immortality bit about what ideas is how it works and have a section on the lessons that I learned that we learned programming and some ideas for advancing the web systems that was web-services ecosystem for to make products like this a little easier and of course will be cowboys so up music is
a European project are intended to help cities reduce you 2 emissions primarily through in the areas of energy production and the energy consumption the a partnership 5 European cities and to research agencies and I guess is the modeling platform a decision support system that's the primary component of the music project and the types of questions that I guess is intended to answer things like how much solar potential that we have in the city where can we install solar panels most effectively there is a geothermal potential here of the mainly at a lot of questions around the production energy actually within the city
so we had a couple of primary design also we set out to work on I guess so the 1st thing was it was was interoperability between partners 1 upon partners to build provide data about the cities and perhaps other partners to provide models and we needed to make sure that there was a way that the data and models could work together we wanted to simplify access to modeling tools of if you've done much modeling is sometimes models can be very complex to configure and run a similar complexity is just inherent in modeling of the complex business but we were looking for ways that we might go to simplify the process a little bit and we want to create reusable framework we want to use this this code for other projects 1 of our partners ability you for other projects we wanted to have at the open source it's anybody use it and so that the code had redistributable so we have all these things together we decided that idea should be designed around open Web services and that ecosystem so this is a list of a schematic
view of how ideas fits into the the services that we use our on the to select select the right this side of the far side and because of the diagram you see these were the model services of model service so the the the the piscivores they can live anywhere on Internet we data over here on the right that are also US data provided by people on the internet and I guess is in the middle I guess doesn't host any of processes I guess of imposing data it is acts as a middleman and of a matchmaker help a match at data services with modeling and then it will it helps about the results of the modeling process something I like to point out in this diagram is that the use a gray box around these different services and the reason that the box encompasses all the services together is because we see these data services is is is components of a really of a single aggregate composite data service we don't think of WFS as different than WCS minute is different but we we also what we really think of as the the data service so for the modeling perspective that we don't really differentiate between those 2 so this is a
screenshot of ideas and this is just always a showing here this is the list of the different models that have been registered with the system currently and it doesn't matter if these are all 1 the media server and different I the EPS service the user care where they are of the user just have these registered models they can they can use and run so what I do now is is still open on 1 of these experiments have showed you what's on the inside so
here we have a description model of the top and we have a list of all of the inputs that the model requires and down the bottom is a list of words all this is just a formatted view of the WPS servers get capabilities strain and it's a request you can make to to the media server will tell you what it can do and and this is also this is just a review of that response but before we can can run a model we have to add data some and show you how you did and I guess so this is
the important thing or it's more data registration thing because we don't actually import it all we're doing is registering where the data lives in enough to put you on the line and that's up at the top and the look clients will go off and make requests using the VMS and that the PCs and BFS to find out what data is available at the Urals holes that responses against these things up and treats a little of little like that and I think this is an index card showing what the data is this stature the data and you can see if the data is available for a for mapping if if the DBMS servers responded for the data is available as a model and the problem most important thing on the screen is the title so when you register dataset that you can assign tags and these
tags are drawn from the identify input identifiers of the different WPS processes so when you start hanging datasets new where data set aside tag your essentially associating that dataset with the input to model and we'll see how it works the same so this is
the actual model configurations frame this is essentially the same information here that we saw earlier that the model description the inputs to the outputs but in this case a gaseous assigned datasets to each model and so for example the top 1 is a digital service model and any dataset that we had DSM is available as a as an importance drop down box so we can better so what it is that of ideas but if you've only tied 1 2 as the and so the only ones we have to worry about deciding when you're when you're configuring us so you can go through signed input data sets to all the model inputs but if the model in this case there a place where the model requires a string entry can take that and basically configure the model by providing nudists specifying what all the inputs are and on the bottom you can say what you want the output data sets to be called so we get that figured
everything's happening provided all the inputs that's a we need it and we have the option to run a model so here the models running it says running a sighting of throwing the and it goes and WPS the the protocol says that you can you what we do here it's a hole in the the media server and the media servers well behaved it will tell us it's promises a go so for example and this case of the model is 25 per cent completed and we get that this that status indicator in the browser so model runs away runs runs runs and finally hopefully it finishes and then what we can do is we we traits this case to new datasets we can view those datasets we can use them as inputs and other processes or download them and and analyze it further on a test of and so this
is just a quick review of our map the or the date of your there's nothing particularly interesting here for people this conference introducing plenty of Mafia as it doesn't do anything special so I want to know that is
what we learned while we're developing system so 1st we have the figure that
so the system is it's easy to deploy updates if we fix a model fix about a model a dataset gets updated these of you know by by any of the data owners these changes are propagated immediately to all users of the system you don't need to install offered on to download the data it just happens automatically the on chain controller data and that's important for some people they don't wanna give the dataset away that 1 field this still retain ownership so by using Web services this way are we can use other people's data but we don't have to take it from and we can use data models of develop other people and perhaps but then the other way that we had anticipated some of the registering a model the system that we haven't seen before and use it with data that we might not seen before and come up with a result that we had anticipated so this so this type of system gives flexibility to be creative and allow other people to be creative and we we find that to be an important thing is based on open standards is no vendor lock-in I come from a long history other so this is a really new thing for me I like it and the system is flexible by plugging in different models and different datasets can suddenly do entirely new things so these are all great great aspects of architecture so
not everything is greater so we have 5
partners solicitor and each of them had been what to me was an interesting problem so the first one I call the w at meteor problem and this is basically an observation that we dealt with a lot of very professional very educated talented yes managers and almost none of the herd of Web services so if you're going to do do play model with people in the real world who were maybe not the people this conference they're not necessary to know what what services are how to set up so the VTS and so on no another problem we had was that 1 city had well apparently contracted although I functions to a private company I'm not sure what to do that but that is another company was very happy to sort of Web services for as long as it was by mice in the contract appropriate change fees were paid to change the contract and it's it's to go over here did everything renegotiated the money sorted out and get get this all working and this is set of server sometimes in-house IT can be a problem to that 1 of our partners had an IT department that was not very happy to open new ports on the firewall have servers that were not under their control running in some IT places and in in in some parts of some some places maybe work in places like this don't really do they want control everything and so if if you're dealing with people men who were in that kind of environment and that's it's a difficult issue to overcome I did a security privacy concerns are a big issue here so 1 of our partners have DEMO GSM dataset they purchase from a vendor they couldn't put that service so that data on a web public Web services because it violated the contract and copper and the data set was it was important issue they also had data that would tell utility data is aggregated someone to the block level but then there were some blocks only have 1 house alone so you even when aggregated data you could still get information about individuals utility consumption and there are some really legitimate concerns of publishing data that can be tied back to individual so that's another issue with putting data into this kind of ecosystem and finding that some of our partners suffered has or support for web services for example 1 of our using an old version of MAP guide which just doesn't support that PCs the data that from start to conclusion from this was that sometimes the non-technical issues are bigger than the technical ones and you dealing with partners who are you not discern the cutting edge this is used to the relation the the
it gets there by this is primarily a comment
on my feeling stressed the BCS theory says that there is this is a is the service providing rested and it puts in itself it's a fine thing is is that nobody supports it and not too many people seem to be using so there's a lotta library a good week we use open layers Ngô we actually wrote the BCS of services for these and I think sometime next week a clue for open letters anyway the BCS code be rolled into the main app I mean really the Judases actually supporting by Islamists yesterday's this will be supporting the BCS maybe as of today but they release that quite until today they can support easier so if you need if you're going a system that relies on the BCS as we did this is this is a problem it will get better over time but right now there's still a lot of packages transport unstable there's a
lot that can go wrong in this ecosystem as a lot of ways things can failure dealing with servers all over the network you know what issues that server issues service go down people take datasets the way things happen there that doesn't happen on the desktop be known seeks in your office in the middle and I've probably and still urea datasets so there's a lot of things that can go wrong in this environment others will slows the system will be under control right so you you might be using I guess is your modeling platform but of somebody's has changed the version of software somewhere or it is something the remote cases may not support the same versions of protocols are hoping supported this a lot of a lot of stuff that can happen and this because sorry different pieces of software that people can plug into this ecosystem it's very difficult to actually test and verify the system works and so often will happen is you know somewhere we do have some good ways of propagating error back to the user but oftentimes there's looked like this shows to user who's using ideas channel model on solar panels and no they're not aligned with what all idle charter came from our actually him service of but you this is a problem because the BCS server somewhere was misconfigured positive EPS server fail and analysis it's difficult
so the undead it is of the cowboy the presentation will be compared with others on
so that is the idea the finally a comment on the fact that the the piscivores more now let me that this is actually more of a crutch over time but the what I want in me that this is that's once you start running in the BPS process you can't kill these women particle I mean you can of the server but there's no way within the WPS service vertical because I know we have our solar model that if this if somebody uploaded a reasonably large datasets this thing run for 2 or 3 weeks it's a huge processed it consumed a lot of resources a lot of power at the expense of the runners somebody started flying concern about may actually put into an instead of 3 they start crying well of AIDS as final process's gonna forget about it but the dubious process process just keep running so weeks go by while this thing is it is in the eye and its well approximated wants another 1 and then decide you want that and you know so we can end up with a lot of of a lot of processes going it's expensive there's no way to kill them and this is this is a real problem so the future of these ideas
for how the ecosystem could be improved in the in coming years and hopefully so the first one is this
idea of of if it interview you worked in radio a greedy hill servers but but WFS and the BCS which of the vector and roster of providing services have and encoding capabilities that describes what the server has know currently you have to really make that request separately for WKS and W episode of PCs and this is like this seems to me it's like that of Web services so they can use that to you data goes for the bed as in the BCS and then you have to maybe you describe covered a described feature requests you get all these may also request the data back in and you have to try and so all loud and and a whole there's nothing conflicting and hold everything looks good in literal hand would be great if there is a way to say get me all the metadata for all the data on the server and if 1 request so that we my 1st years to to simplify the way to simplify the process of getting data getting metadata about the data that on server so 2nd 1 is w kiosk and this is the model running service requests that we have you know that you have to the data to run a model but the WDS process is not actually describe what it needs very well you can provide style sheet that the data is very detailed but processing this in passing is an understanding the style sheet is very very complicated and we have a funding good tools for doing it what would be great is of a model the safe use of data that I need and you can take in and described in detail and you could send that description off to a data service and say hey data service what did you have that matches this description and the data services say well OK this dataset looks like you you you use this data set is input and there's a way that the WGS to cooperate better with the data providing services that would make life a lot easier for everybody of WPS also you should have some call accident right now but when you start running a model in this case it's there three week solar zombie our model the model it runs in your head depend stopping the servants ahead done high you are you done we do this every minute now this thing is 1 for weeks and work were calling every minute that seems to me to be a little bit wasteful I would really like to have the when the server is done running it could call your own say hey I'm done now and then you carry on and it is also applies to very very short passes your process that runs in a 2nd but you're appeared only checking out once a minute that means the user could have could have an almost instantaneous feedback from or in the model but instead might have to wait a whole men until your polling cycles through so if the the media service has a way of announcing its status somebody that would be great 4 words as molecular using your favorite on the control method is needs to be a way for the recast to to be terminated and the last 1 is it would be great to augment XML what I mean by that is you do a fictive those requests server tell you about itself and tells you about itself an XML format of brothers don't understand externality doing this not something on the client side get back x amount may need to interpret that acts on all night usually requires a fairly be library and if you're running a phone or a tablet there's a lot of job description to download in order to make this make this work and if if any of you have looked into the open the source code a lot of it is just axonal passing however if we get a response adjacent browsers to understand Sunday that way we have a bunch of stuff from open the simplify make life easier for developers make life easier for users and I think that make everything work will so ideas for how things can be improved and we're now
at the end of the presentation I thank you for your attention
the the the right that was to the in of the the the but I as he could use that I think that mostly right because repeat that again quickly you of of the OK the catalog of data comes not from Baker that's comes from the registration process the user enters the URL of the data service and and kind builds a catalog on ideas of so there is not the I never had know how of well of course the the processing servers need to data in order to run of course but it doesn't come through I guess what I guess there's the sense the URL that data to the WPS server and then you get several on its own download of the dataset from the from the data provide is at the bottom of the pot yes the yes so I mean what I want it it's great it that what works and usually works it's great but if somebody has remove the dataset since you last saw or uncle but the network is not working and with the size of the the the word which so I people the yes yes that's true I know what this is unavoidable if your processor processing service is different than the guy who has the data and there's no way to avoid the cell you can the fact that use we could but by doing that you reduce the modularity the system unit so people are now starting to install software and we have fun of problems is getting our customers or partners data providing services but they had also have now processing services that are going to be sitting there processing for weeks at a time it's going to be too much but yes I think your pointer will really fast the Yoccoz the strategy that the the easiest of all the time this and it is standardized and I also came across the the is the natural world as color organizations in the the and that is basically running of the lecture the network models with the the the parents of old that but it is designed to be a generic model it's very similar to the architecture of the house and so I think about this is that you can take this thing and this is what we are have the and it is always the case you know doing that and the but I think that this really is the future of our new model of of modeling or yeah I I agree wrenches been like from bacteria in the form set of and the effect of using the BCS basically in its simplest form we just grab everything and what the server figure out what it what it means to do it because if you now if if you wanna start adding bounding boxes including and stuff like that the new face becomes much more complicated and so yeah what is using and in the most basic raw form because of the the I am so the noise and the and all of what do you of the of the of the of the of the of the of the
Physikalisches System
Bit
Benutzerbeteiligung
Web Services
Medizinische Informatik
Offene Menge
Gruppe <Mathematik>
Güte der Anpassung
Garbentheorie
Physikalisches System
Modelltheorie
Optimierung
Kombinatorische Gruppentheorie
Biprodukt
Systemplattform
Vektorpotenzial
Bit
Prozess <Physik>
Systemplattform
Komplex <Algebra>
Code
Framework <Informatik>
Computeranimation
Physikalisches System
Web Services
Gruppe <Mathematik>
Datentyp
Entscheidungsunterstützungssystem
Modelltheorie
Open Source
Systemplattform
Mailing-Liste
Biprodukt
Energiedichte
Framework <Informatik>
Flächeninhalt
Komplex <Algebra>
Komponente <Software>
Offene Menge
Energiedichte
Entscheidungsunterstützungssystem
Projektive Ebene
Modelltheorie
Resultante
Subtraktion
Server
Prozess <Physik>
Quader
Desintegration <Mathematik>
Computeranimation
Internetworking
Web Services
Perspektive
Modelltheorie
Web Services
Sichtenkonzept
Matching <Graphentheorie>
Einfache Genauigkeit
Mailing-Liste
Physikalisches System
Stochastischer Prozess
Diagramm
Angewandte Physik
Komponente <Software>
Rechter Winkel
Hypermedia
Server
Modelltheorie
Sichtenkonzept
Mailing-Liste
Ein-Ausgabe
Computeranimation
Chipkarte
Deskriptive Statistik
Client
Polarkoordinaten
Registrierung <Bildverarbeitung>
Automatische Indexierung
Hypermedia
Minimum
Endogene Variable
Server
Wort <Informatik>
Modelltheorie
Gerade
Touchscreen
Quader
Konvexe Hülle
Ein-Ausgabe
Computeranimation
Eins
Deskriptive Statistik
Web Services
Digitalisierer
Minimum
Identifizierbarkeit
Information
Modelltheorie
Konfigurationsraum
Funktion <Mathematik>
Zeichenkette
Mapping <Computergraphik>
Protokoll <Datenverarbeitungssystem>
Browser
Hypermedia
Server
Modelltheorie
Indexberechnung
Ein-Ausgabe
Computeranimation
Stochastischer Prozess
Konfiguration <Informatik>
Resultante
Offene Menge
Subtraktion
Kontrolltheorie
Mathematisierung
Besprechung/Interview
Datenmodell
Varietät <Mathematik>
Physikalisches System
Computeranimation
Web Services
Verkettung <Informatik>
Datenfeld
Standardabweichung
Datentyp
Modelltheorie
Modelltheorie
Softwareentwickler
Figurierte Zahl
Unternehmensarchitektur
Standardabweichung
Kontraktion <Mathematik>
Firewall
Mathematisierung
Versionsverwaltung
GSM-Software-Management AG
Datenmissbrauch
Computeranimation
Übergang
Eins
Benutzerbeteiligung
Web Services
Polarkoordinaten
Computersicherheit
Luenberger-Beobachter
Modelltheorie
Elektronischer Programmführer
Web Services
Lineares Funktional
Datenmissbrauch
Kontrolltheorie
Computersicherheit
Relativitätstheorie
Softwarewerkzeug
p-Block
Quick-Sort
Design by Contract
Software
Menge
Mereologie
Identitätsverwaltung
Server
Information
Programmierumgebung
Offene Menge
App <Programm>
Software
Server
British Computer Society
Web Services
Offene Menge
Programmbibliothek
Vorlesung/Konferenz
Physikalisches System
Code
Computeranimation
Subtraktion
Ortsoperator
Versionsverwaltung
Nichtlinearer Operator
Systemplattform
Computeranimation
Systemprogrammierung
Physikalisches System
Web Services
Software
Vorlesung/Konferenz
Modelltheorie
Kontrolltheorie
Analysis
Fehlermeldung
Protokoll <Datenverarbeitungssystem>
Kontrolltheorie
Datennetz
Prozess <Informatik>
Physikalisches System
Office-Paket
Angewandte Physik
ATM
Ein-Ausgabe
Server
Programmierumgebung
Versionsverwaltung
Fehlermeldung
Prozess <Physik>
Prozess <Informatik>
Kombinatorische Gruppentheorie
Computeranimation
Stochastischer Prozess
Gotcha <Informatik>
Knotenmenge
Web Services
Server
Protokoll <Datenverarbeitungssystem>
Vorlesung/Konferenz
Partikelsystem
Leistung <Physik>
Bit
Prozess <Physik>
Browser
Familie <Mathematik>
Service provider
Computeranimation
Metadaten
Deskriptive Statistik
Client
Web Services
Lesezeichen <Internet>
Polarkoordinaten
Prozess <Informatik>
Endogene Variable
Vorlesung/Konferenz
Greedy-Algorithmus
Modelltheorie
Softwareentwickler
Erweiterte Realität <Informatik>
Kontrolltheorie
Schreib-Lese-Kopf
Videospiel
Kontrolltheorie
Güte der Anpassung
Systemaufruf
Quellcode
Vektorraum
Ein-Ausgabe
Mechanismus-Design-Theorie
Hill-Differentialgleichung
Stochastischer Prozess
Menge
Hypermedia
Dreiecksfreier Graph
Tablet PC
Server
Dateiformat
Ordnung <Mathematik>
Kiosksystem
Message-Passing
Prozess <Physik>
Quader
Selbst organisierendes System
Natürliche Zahl
Zellularer Automat
Geräusch
Online-Katalog
Kombinatorische Gruppentheorie
Computeranimation
Methodenbank
Netzwerkdatenbanksystem
Bildschirmmaske
Einheit <Mathematik>
Web Services
Software
Minimum
Modelltheorie
Zeiger <Informatik>
Figurierte Zahl
Soundverarbeitung
Datennetz
Physikalisches System
Stochastischer Prozess
Generizität
Menge
Registrierung <Bildverarbeitung>
Rechter Winkel
Strategisches Spiel
Server
Wort <Informatik>
Kantenfärbung
URL
Ordnung <Mathematik>
Unternehmensarchitektur

Metadaten

Formale Metadaten

Titel iGUESS – The Good, The Bad And The Ugly Of Developing A Distributed Modeling System For European Cities
Serientitel FOSS4G Nottingham 2013
Autor Eykamp, Chris (CRP Henri Tudor)
Sousa, Luís de (CRP Henri Tudor)
Leopold, Ulrich (CRP Henri Tudor)
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben
DOI 10.5446/15544
Herausgeber FOSS4G, Open Source Geospatial Foundation (OSGeo)
Erscheinungsjahr 2013
Sprache Englisch
Produktionsort Nottingham

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract The integrated Geospatial Urban Energy decision Support System (iGUESS) was conceived as a way to help urban planners explore renewable energy and energy savings potentials to make cities more sustainable and self sufficient. Models that calculate solar, wind, and geothermal energy potential can be complex to build and run, so we felt we could simplify the process by creating a web-based tool that a planner could run from their browser. To maximize interoperability with existing models and data sources, we decided to build the system using existing OGC standards and protocols. iGUESS is a web-based system for connecting data, modeling, and visualisation services distributed across the Internet. Users can leverage data and processing services offered via standard OGC protocols such as WMS, WFS, WCS, CSW, and WPS. iGUESS helps users match data with models, launch model runs, monitor progress of execution, and visualize computed results. iGUESS does not store data or host computation services, but instead relies on data and modeling web services provided elsewhere in the project, by our partners, and by third parties. Developing iGUESS has given us a better understanding of the strengths and weaknesses of a distributed modeling system based on OGC services, and some of the inherent limits of these protocols. The interface allows users to interact with services in real-time, using minimal caching, so it always presents an accurate reflection of what data and modeling services are available. This design has presented us with interesting challenges related to intermittent and unpredictable availability of distributed data and process services that live beyond the bounds of the system. The primary advantages of this distributed modeling system is its modularity and flexibility. Users can run models using input datasets they (or others) may have published for different purposes. Models can be upgraded and improved by their publishers without requiring users to install new software. Finally, running models via WPS can be easier than configuring a local desktop model, and the processing is offloaded onto a computer presumably more suited to handling large, complex calculations. Many of the specific challenges we faced have been related to the limitations of the WPS protocol. It is difficult to precisely specify inputs or describe outputs, and there is no mechanism for prioritzing or terminating a running process. The lack of process control is particularly relevant to the sorts of large, processor intensive models that iGUESS was designed to run. Also, very little of the data our partners need to use is actually available online, and they have encountered a wide range of logistical and institutional barriers to providing it themselves. Lastly, we are still trying to cope with issues related to exposing computationally expensive processes to the Internet. This talk will present a technical overview of the iGUESS system, how it works, alternate approaches we considered (distributed architecture vs. traditional “desktop” approach), and the lessons we learned building it (managing complexity and the risks of oversimplification). It will also explore some of the “real world” hurdles mentioned above, and will offer some ideas and insights into the type of applications that are best suited for the WPS protocol.

Ähnliche Filme

Loading...
Feedback