We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Get over the boundaries between client and server in web app development

00:00

Formal Metadata

Title
Get over the boundaries between client and server in web app development
Title of Series
Number of Parts
160
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Get over the boundaries between client and server in web app development [EuroPython 2017 - Talk - 2017-07-11 - Anfiteatro 2] [Rimini, Italy] The practice in the development of advanced web applications is to break it into two main areas where the effort on the server, with the typical WSGI environments, focuses on data access configuration whereas the application logic is delegated mostly to the client through the use of JavaScript frameworks. The reason for this separation into two roles lies in the need to use JavaScript on the browser and the fact that the communication channel (HTTP) and the way in which the application state is handled on the server side is still that thought to serve full web pages. In this talk I'll show a framework and an application where the line between the two worlds is blurred and where it is possible to think of the application in terms of unity, with the two components that cooperate equally and communicate without thinking in terms of URLs or HTTP verbs . This is made possible by the asynchronous/reactive management of the processing from the database driver to the user interaction, the use of websocket for communication and the use of Python (optional) for the development of the client part. Some technologies used are: PostgreSQL, the ""asyncpg"" driver, the ""aiohttp"" web server, Crossbar (http://crossbar.io) for the websocket comms, pkg ""metapensiero.sphinx.patchdb"" for schema upgrades, the package ""metapensiero.reactive"" and ""metapensiero.signal"" for the management of reactive event streams,the package ""Javascripthon"" for Py3 to ES6 JS transpiling
Server (computing)SoftwareIntelTable (information)Content (media)Context awarenessWorld Wide Web ConsortiumMobile appRelational databaseInterface (computing)DatabaseRenewal theoryIntranetHome pageSingle-precision floating-point formatData structureSoftware frameworkObject-relational mappingSoftware developerIterationSocial classElectric generatorElement (mathematics)Hecke operatorSoftware developerPoint cloudData dictionaryWeb 2.0InternetworkingSoftware frameworkCartesian coordinate systemSet (mathematics)Multiplication signString (computer science)Object (grammar)Information securityDatabaseLibrary (computing)CodeElectric generatorObject-relational mappingWeb browser10 (number)Level (video gaming)Server (computing)Point (geometry)Bookmark (World Wide Web)Vulnerability (computing)Turbo-CodeStandard deviationWeb pageLogicWeb applicationKey (cryptography)LoginNumberLink (knot theory)Stress (mechanics)Relational databaseSystems integratorDataflowPhysical systemData structureForm (programming)Single-precision floating-point formatTable (information)Home pageRevision controlFormal languageInteractive televisionType theoryFlow separationCuboidSega Enterprises Ltd.Query languageMappingRight angleScripting languageMenu (computing)Musical ensembleNeuroinformatikCross-correlationBasis <Mathematik>Pairwise comparisonQuicksortThomas BayesInfinitySocial classWave packetComputer animationLecture/Conference
Category of beingInformation securitySoftware developerKey (cryptography)Information securityObject (grammar)Vulnerability (computing)Level (video gaming)XML
Information securityCategory of beingSoftware developerInterface (computing)Phase transitionType theoryBlock (periodic table)Information securityCodeCategory of beingBlogInheritance (object-oriented programming)Arithmetic meanInterpreter (computing)Formal languageSubject indexingLink (knot theory)Row (database)Object (grammar)Single-precision floating-point formatComputer animation
Data typeScripting languageServer (computing)Data structureComplete metric spaceDatabaseWorld Wide Web ConsortiumRootServer (computing)LogicMultiplication signDatabaseFlow separationData structureCartesian coordinate systemWeb applicationSoftware developerProcess (computing)Domain nameType theoryRepresentational state transferMereologyScripting languageComputer animation
Data modelDependent and independent variablesSoftware frameworkKolmogorov complexityClient (computing)Content (media)Object (grammar)Floating pointCodierung <Programmierung>Process (computing)World Wide Web ConsortiumHost Identity ProtocolServer (computing)Schmelze <Betrieb>Element (mathematics)Different (Kate Ryan album)Server (computing)Interface (computing)Standard deviationInteractive televisionCartesian coordinate systemSoftware frameworkWeb 2.0Software development kitWeb applicationStreaming mediaLogicPatch (Unix)Query languageContext awarenessNormal (geometry)Dependent and independent variablesPhysical systemDecision theoryDatabaseWeightClient (computing)Social classDampingForm (programming)GUI widgetMultiplication signGame controllerWeb-DesignerObject (grammar)Mobile appXMLComputer animation
Server (computing)UsabilityPhysical systemClient (computing)Element (mathematics)Event horizonLogicQuantumTheoremServer (computing)Software maintenanceConcurrency (computer science)State of matterClient (computing)Cartesian coordinate systemPhysical systemXML
UsabilityPhysical systemServer (computing)Event horizonElement (mathematics)ModemPhysical systemComputer animation
Patch (Unix)DatabaseRouter (computing)Event horizonHost Identity ProtocolSynchronizationDigital rights managementObject-relational mappingQuery languageGenetic programmingClient (computing)ImplementationCommunications protocolConfiguration spaceString (computer science)Address spacePhysical systemRemote procedure callSoftware frameworkLevel (video gaming)Client (computing)CodePhysical systemFormal languageHeat transferObject-relational mappingServer (computing)Different (Kate Ryan album)Library (computing)DatabaseError messageQuery languageMereologyEvent horizonSquare numberDirected graphDevice driver
Client (computing)Library (computing)Communications protocolImplementationRouter (computing)Configuration spaceString (computer science)Address spaceError messagePhysical systemRemote procedure callArrow of timeSource codeHeat transferClient (computing)Error messageXML
ImplementationCommunications protocolRouter (computing)Configuration spaceAddress spaceError messagePhysical systemRemote procedure callEvent horizonVertex (graph theory)Asynchronous Transfer ModeNetwork topologyReading (process)MereologyObject (grammar)Social classTelecommunicationComputer animationXML
Event horizonVertex (graph theory)Asynchronous Transfer ModeSocial classMereologyObject (grammar)Network topologyTelecommunicationCartesian coordinate systemLogicLevel (video gaming)Hydraulic jumpCore dumpWeb 2.0Bit rateConnected spaceEvent horizonLine (geometry)Relational databaseDeclarative programmingSoftware testingComputer animation
Execution unitEmailVertex (graph theory)Maxima and minimaEvent horizonIRIS-TMassSoftware testingLine (geometry)TheoryOrder (biology)CausalityData structureLine codeSystem callComputer animation
Computer virusVertex (graph theory)Annulus (mathematics)Limit (category theory)MereologyForestSoftware testingTask (computing)Line (geometry)Multiplication signDepth of fieldAddress space2 (number)Computer animation
Electric generatorAbstractionJava appletCodeReduction of orderSource codeSynchronizationServer (computing)Cursor (computers)Raster graphicsShape (magazine)IRIS-TSoftware testingDemo (music)Cartesian coordinate systemCommutatorServer (computing)Core dumpLine (geometry)Multiplication signSoftware frameworkAbstractionDemosceneCodeThermal radiationWeb browserShape (magazine)Goodness of fitSlide ruleDemo (music)Computer animation
Multiplication signRepresentation (politics)Lecture/Conference
Transcript: English(auto-generated)
Hello, so I am Alberto, I am co-founder of a small company that offers development and system integration services and today I will present you a framework and an application
that we built to bring back fun into the development of web applications. So, what I mean with web application, usually web application today means many things but
for the purpose of this talk I mean those applications that are usually developed today to replace desktop application that used to assess and relational databases and show data
to the user, let the user filter, query, sort, they usually show many data at once.
They usually offer advanced filtering sorting capabilities and so on and usually those applications are used for data entry and so they show very complex forms and they show
master tables and so on. I think that such application have a narrower user base than
public web page publishing and so on. Usually those applications are called single page applications. Many times they are installed on premise or they are installed on cloud
and then used inside an internet or so on. So the framework we developed, its goal is to serve these kind of applications. It's not to serve 10,000 web pages to anonymous users,
usually the user of our application is always a known user so the first thing he does is to
enter his credentials and log in to the system. So today how it's built an application using Python tools. Usually we start with developing the database structure and so on and then we
pick our favorite server framework like Django, pyramid, turbo gears, flask and so on. Maybe we
develop an object relational mapper on top of our database but it's not always needed. Usually we expose those data using REST services or get and post entry points and so on. But
then because they are web apps then we pick a JavaScript framework and we develop all the
application level logic with JavaScript. So all the user interaction, all the flow of the user inside the application and so on is coded in JavaScript. How many of you do that, for example?
Is it fun? Is it still fun? So I don't think that it's so fun. And yes, dealing with
JavaScript is inevitable because there is nothing else on the browser. But it's stressful because it has many circumstances, many things that you maybe learn one time and
then you forget something you know about its base types, not a number and so on. And every time you approach the development of JavaScript web application, usually the
scenario of available toolkits is changed because it changes with a very fast piece and usually the JavaScript community reinvents the wheel every time in a fancier, more
quick way and so on. But for us, that we want to code in Python, it seems not so good
to always learn a new JavaScript framework, maybe the version 2 of the JavaScript that we learned some months ago that changed the language from JavaScript to TypeScript
and so on. And many times the libraries of JavaScript libraries have compared to those available in Python and to the standard library do have very poor quality. You end
up installing tens of megabytes, hundreds of megabytes of JavaScript code just to let your toolkit work and so sometimes you don't understand why you have to install
all those things. So many of you can say that ES6 is better because it's classes, promises, iterators, generators. Some of these features come from Python and also
it has map and set that are implemented natively that for the first time you can put not just strings into dictionary keys but you can use any kind of object as key
and this is surely an advantage. But then, for example, when I learn about weak map and weak set, I thought that they were comparable to those available into the Python
standard library and so I said fantastic. They were what I was waiting for because then I can keep weak links to objects without worrying to free those links and so
it's very nice feature but then you learn that I discovered that, for example,
weak maps, you cannot know what's inside a weak map. Just if you know what's inside a weak map is because you have put it there. So it's not possible to ask a weak map for its keys.
It's not possible to iterate over its keys or its values and so on. It's just possible to give an object to check if it's a key in that object and it seems incredibly wrong to me
but the JavaScript community found that a feature instead. So one of the more prominent JavaScript developers said that it is a security feature,
something that you can, that it's really useful because you can have an object that
no one can look into but I think that it is a misunderstanding of what a weak map should do. It should firstly give you a way to use weak links, weak references to other objects
and then maybe a being used a security property or security object because you cannot look into it. So JavaScript, even EmmaScript ES6 has many, many things that seem wrong from a Python
Python side and some, if you read blogs and articles around the internet, some suggest
that you should use, for example, TypeScript which is a very, very good language that has interfaces, types but for example, a few days ago I found this example in a Reddit blog,
actually when they were explaining why they chose the TypeScript and you see that code where a foo is declared as an array that can contain just birds and the next
row they add an animal which is the superclass of bird and the TypeScript interpreter and checker is okay with that. So while having types may help maintaining the code
and so on, the language that has type in its name fails to check a very single simple
block of code. So again, I would say what it seemed very, very strange to me and that's because I know that we have to deal with JavaScript, TypeScript, CoffeeScript, what you want script
but I will put it in a cage and control it from very, I will like to at least. So
it's enough talking about JavaScript so let's get back to Python and what's the role of Python in modern web applications. Usually when you have such a separation between application logic
and data access logic, one the first developed in JavaScript and the second developed in Python, the Python server that you develop in one of such application servers that I told you before
is used like a data hub, you expose your data in some way and really for me and for us, the fun part of developing application stops when we develop our database structure, we confront
with the customer about the domain data and so on and really what comes after the database
structure, the exposing, the create a REST API, create a get or post API, it's boring. We do really need that. When do we really need the REST APIs and so on? We think that we need
the REST APIs just when your service as to your application as to interface with other services
by using standardized interfaces such as REST and so on. Or if you give an API to your user to use and to interact with the, for interaction with your application
instead of using the GUI. This is the normal way of how Django, Pyramide, the Flask, you name it, do they work more or less? More or less, they all wait for a request and then
they will, the request contains all the data, the context data that your application, your
controller, your handler needs to understand what it should do and then it retrieves the data from the database. It builds up a response and stream it back to the client, maybe it saves
some data in a session that usually is used like a poor man dictionary and then your objects are destroyed and your system is waiting for the next query, next get, post or
patch and so on. How are the things because there are still, those toolkits are still used
today to do desktop application. How is developing an application with desktop toolkits? Those that are now supplemented by web application and so on. Usually developing an application
with PyQT and PyGTK, the role of the GTK or QT toolkit and Python,
their interaction is much more tighter than with web apps. In web apps we have the application logic completely moved to the client but instead in normal desktop application we have
the application logic is completely driven by Python with the app of classes and UI widgets
that the toolkit provides us. So they are very different from today's web application. Our, we found ourselves at the end of last year in the need of
decide if we were maintaining an old application developed with one of those frameworks. It's much older indeed. It was developed in ZOOP or we have made a decision
instead of replacing it with a new app and we wanted to bring back the fun into web development.
So we developed a framework from scratch and the idea was to use an asynchronous system not just to achieve better concurrency but also to ease the maintenance on the server of the state
of the application. We thought to have two equal systems on the server and the client and to have them freely talk to each other using modern RPC systems that can use
HTTP as a back-end protocol but that they abstract you from the notion of having an HTTP and even they work without HTTP at all. So we developed
this framework Raccoon with Postgres at the bottom level. All the framework has to be
asynchronous and so we chose asyncpg to drive to the data access to the database
and SQL alchemy to construct the queries. Crossbar has been chosen as the intercommunication backbone between the two pieces on the client and the server
and AEO HTTP is used as an HTTP publisher for the stuff that cannot be transferred via for the initialization part and so on. Then we use the SQL alchemy but
not the ORM because the object relation mapper cannot be used with async code and the RPC we chose crossbar because it allows your system to interface not just between Python and JavaScript but between many, many different languages and as both a twisted
and asyncio client library and as such nice features like error transfer between the caller
and the callee. If the callee raises an error, this error is brought back to the caller and so on. It's really nice and has many features. So we designed an API that
or API as composed by a tree of nodes object that where the initialization is split in two parts because the underinit cannot be an asynchronous method so we have to split the
initialization in two parts and node is an mixing class that just deal with the communication part of the problem. The rest of the object is supposed to do, it's your application
and so on. What the node class gives you is a way to declare callable things and signals and handlers at the class level with the decorators and so on and it does that in a way
that it's possible to construct your object trees with relative references to each other
and then attach them to the crossbar connection machinery and have them published on the have them published. Here you see some the nodes and the methods and the signals. Signals are
events that the nodes allows you to declare to use. So this is, let's jump to an example of a test and you see that at the line 12 we have the first node which is just a node
that's used for structure and then we have two other nodes, the second and the third
where we customized our and we added some handlers, some signals, an handler, a signal, a call and so on. The node is automatically instrumented to expose those methods
and you see that for example the line code third, line 17 that calls the third node with a relative
path. It depends on where those nodes are mounted into, are bound to, there is a base
path, this is test and then first is bound to base plus first that will become test dot first and then third is bound to base dot third and then the first node adds the second
as with the name second and so its node path will become test dot first dot second and so you saw before that the handler on line 25 listened for the signal on first, second
and all these are relative addresses that when they are realized when the node is connected to
the machinery which is in line 37 and 38 and so on and this is a variable also in JavaScript,
this is actual JavaScript code and it is in JavaScript code because we use a tool we made to compile Python to JavaScript and we use the same syntax, the same abstraction
and then we compile them to JavaScript, to a JavaScript that also a very old browser can run
and that has no asynchronous, no promises and so on. This is actually an anatomy of our session of our application user session, how it starts, there is a service that is the only thing that
exposes both HTTP and crossbar and connects to crossbar and then each other side can talk to each other very freely. I have many more slides but I am out of time so if you
are interested we have developed the application is nearly complete, I cannot show you it now but if you are interested we will, I can show you, we can give you an assess and a demo
to the application. We will probably publish this framework when it's ready, it's in good shape, we need to completely separate it
from the application but it's going very well, it has more 4,400 commits so if you want or see the code just ask me or drop me a line, thank you.
Thanks Albert for your presentation, questions for Alberto? No? Thanks again, thanks Alberto.