Concurrent Programming Made Simple

Video thumbnail (Frame 0) Video thumbnail (Frame 1000) Video thumbnail (Frame 2183) Video thumbnail (Frame 3374) Video thumbnail (Frame 4611) Video thumbnail (Frame 5571) Video thumbnail (Frame 6830) Video thumbnail (Frame 10643) Video thumbnail (Frame 14423) Video thumbnail (Frame 15572) Video thumbnail (Frame 16635) Video thumbnail (Frame 18720) Video thumbnail (Frame 20080) Video thumbnail (Frame 21795) Video thumbnail (Frame 23218) Video thumbnail (Frame 25063) Video thumbnail (Frame 26136) Video thumbnail (Frame 27113) Video thumbnail (Frame 28601) Video thumbnail (Frame 31254) Video thumbnail (Frame 32402) Video thumbnail (Frame 35936) Video thumbnail (Frame 38277) Video thumbnail (Frame 39573) Video thumbnail (Frame 41033) Video thumbnail (Frame 42242) Video thumbnail (Frame 46261) Video thumbnail (Frame 48088) Video thumbnail (Frame 51268) Video thumbnail (Frame 52279) Video thumbnail (Frame 54102) Video thumbnail (Frame 55261) Video thumbnail (Frame 56697) Video thumbnail (Frame 58262) Video thumbnail (Frame 59667) Video thumbnail (Frame 60834) Video thumbnail (Frame 62790) Video thumbnail (Frame 64304) Video thumbnail (Frame 65507) Video thumbnail (Frame 66636) Video thumbnail (Frame 68675) Video thumbnail (Frame 70121) Video thumbnail (Frame 72920)
Video in TIB AV-Portal: Concurrent Programming Made Simple

Formal Metadata

Concurrent Programming Made Simple
The (r)evolution of Transactional Memory
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
This talk will present Transactional Memory, a programming abstraction for managing concurrency, both in multi-threaded programs running on multi-core processors as well as in distributed cloud infra-structures. This talk will present Transactional Memory (TM), a programming abstraction for managing concurrency, both in multi-threaded programs running on multi-core processors as well as in distributed cloud infra-structures. TM allows programmers to declare which parts of a program need to run atomically (i.e., as indivisible steps), while a generic implementation involving compilers, runtime libraries, middleware, and hardware support for TM takes care of ensuring this atomicity at runtime. The two presenters will give an overview of recent advances, standardization efforts (e.g., for C++), and open-source tools providing support for TM (e.g., GCC).
Concurrency (computer science) Multiplication sign Total S.A. Database transaction System call Computer programming Demoscene Word Arithmetic mean Computer animation Data storage device Semiconductor memory Right angle
Group action Context awareness Computer animation State of matter Semiconductor memory Synchronization Atomic number Multiplication sign Shared memory Database transaction Associative property
Message passing Group action Computer animation Atomic number Computer hardware Website Set (mathematics) Database Software testing Right angle Computer
Area Programmer (hardware) Implementation Computer animation Semiconductor memory Code Database transaction Sequence Abstraction Computer programming Machine vision
Implementation Group action Observational study Code Real number Modal logic Tape drive Workstation <Musikinstrument> Virtual machine Mereology Semantics (computer science) Computer programming Revision control Atomic number Semiconductor memory Single-precision floating-point format Computer hardware Logic programming Energy level Functional programming Task (computing) Social class Programming language Focus (optics) Assembly language Consistency Constructor (object-oriented programming) High-level programming language Shared memory Database transaction Perturbation theory Instance (computer science) Compiler Vector potential Arithmetic mean Computer animation Software Data storage device Personal digital assistant Statement (computer science) Website Right angle Quicksort Procedural programming Pressure Abstraction Sinc function Tuple
Slide rule Concurrency (computer science) Information Execution unit Database transaction System call Semantics (computer science) Attribute grammar Number Web 2.0 Latent heat Pointer (computer programming) Computer animation Synchronization Functional programming Compilation album
Programming language Implementation Multiplication Theory of relativity Database transaction Sequence Product (business) Arithmetic mean Latent heat Word Computer animation Semiconductor memory Synchronization Personal digital assistant Order (biology) Automation Energy level Right angle Endliche Modelltheorie
Implementation Computer animation Bit rate Semiconductor memory Computer programming Database transaction Computer programming Resultant Compiler
Default (computer science) Insertion loss Database transaction Cartesian coordinate system Limit (category theory) Metadata Computer animation Atomic number Synchronization Generic programming Functional programming Associative property Arithmetic progression
Thread (computing) Block (periodic table) Interface (computing) Electronic mailing list Database transaction Element (mathematics) Compiler 2 (number) Computer animation Integrated development environment Right angle Functional programming Stability theory
Programmer (hardware) Implementation Computer animation Atomic number Code Website Database transaction Generic programming Exception handling Compiler
Point (geometry) Implementation Run time (program lifecycle phase) Arm Code Database transaction Mechanism design Computer animation Semiconductor memory Data storage device Right angle Functional programming Resultant
Default (computer science) Implementation Algorithm Overhead (computing) Run time (program lifecycle phase) Mapping Multiplication sign Set (mathematics) Bit Database transaction System call Compiler Computer animation Software Synchronization Term (mathematics) Personal digital assistant Semiconductor memory Different (Kate Ryan album) Computer hardware Website Functional programming Modem Exception handling
Point (geometry) Group action Overhead (computing) Computer animation Channel capacity Semiconductor memory Hybrid computer Computer hardware 1 (number) Database transaction Office suite Quicksort
Programmer (hardware) Slide rule Concurrency (computer science) Computer animation Personal digital assistant Chemical equation Right angle Usability
Point (geometry) Implementation Overhead (computing) Code Multiplication sign Numbering scheme Protein Power (physics) Revision control Programmer (hardware) Structured programming Semiconductor memory Oval Computer hardware Code refactoring Resource allocation Mathematical optimization Arm Shared memory Database transaction Limit (category theory) Cartesian coordinate system System call Compiler Computer animation Order (biology) Right angle Pattern language Arithmetic progression Abstraction
Overhead (computing) Thread (computing) Code Real number Sheaf (mathematics) Scalability Element (mathematics) Revision control Workload Mathematics Estimator Semiconductor memory Atomic number Single-precision floating-point format Data structure Endliche Modelltheorie Tunis Form (programming) Multiplication Scaling (geometry) Constructor (object-oriented programming) Physical law Database transaction Limit (category theory) Variable (mathematics) Computer animation Right angle
Point (geometry) Support vector machine Implementation Run time (program lifecycle phase) Graph (mathematics) Arm Sequel Civil engineering Multiplication sign Feedback Hidden Markov model Cartesian coordinate system Programmer (hardware) Latent heat Mechanism design Computer animation Term (mathematics) Different (Kate Ryan album) Synchronization Hybrid computer Blog Extension (kinesiology) Traffic reporting
Point (geometry) Latent heat Group action Implementation Computer animation Observational study Lecture/Conference Code Order (biology) Virtual machine Graph coloring Tunis
Point (geometry) Dynamical system Group action Computer animation Software Semiconductor memory Virtual machine Database transaction Price index Abstraction Position operator
Point (geometry) Distribution (mathematics) Multiplication sign Electronic mailing list Virtual machine Device driver Set (mathematics) Database transaction Cartesian coordinate system Rule of inference Computer animation Term (mathematics) Telecommunication Logic programming
Point (geometry) Server (computing) Scaling (geometry) Consistency Reflection (mathematics) Virtual machine Translation (relic) Bit Water vapor Database transaction Fault-tolerant system Replication (computing) Rule of inference Subset Programmer (hardware) Facebook Computer animation Personal digital assistant Natural number System programming Game theory
Programmer (hardware) Computer animation Multiplication sign Sheaf (mathematics) Skewness Database transaction Sequence Position operator
Point (geometry) Computer program Direction (geometry) Database transaction Replication (computing) Flow separation Scalability Word Bootstrap aggregating Computer animation Personal digital assistant Software framework Communications protocol Partial derivative
Domain name Programming language Game controller Context awareness Mapping Java applet Code Multiplication sign Expert system Online help Bit Instance (computer science) Programmer (hardware) Latent heat Goodness of fit Computer animation System programming Right angle Social class
Domain name Medical imaging Focus (optics) Computer animation Computer file Euler angles Personal digital assistant Multiplication sign Demoscene Modeling language Social class
Domain name Computer program Distribution (mathematics) Theory of relativity Code Interface (computing) Interactive television Cartesian coordinate system Computer Structured programming Process (computing) Computer animation Software framework Social class
Domain name Implementation Data management Computer animation Computer file Code Data storage device Configuration space Counting Database transaction Musical ensemble Cartesian coordinate system
Domain name Programmer (hardware) Distribution (mathematics) Process (computing) Computer animation Database transaction Cartesian coordinate system Mereology Computer
Point (geometry) Computer program Group action Concurrency (computer science) Computer file Consistency Cartesian coordinate system Replication (computing) Software bug Category of being Computer animation Personal digital assistant Telecommunication System programming Software framework Reading (process) Control system
Distribution (mathematics) Sine Open source Code Structural load Multiplication sign Boilerplate (text) Database Cartesian coordinate system Computer animation Personal digital assistant Term (mathematics) Hybrid computer Universe (mathematics) Website Software framework Social class
Point (geometry) Domain name Mapping Code Student's t-test Line (geometry) Instance (computer science) Mereology Arithmetic mean Computer animation Universe (mathematics) System programming Object-oriented programming Functional programming Endliche Modelltheorie
Axiom of choice Complex (psychology) Computer program Server (computing) Context awareness Run time (program lifecycle phase) Code State of matter 1 (number) Virtual machine Drop (liquid) Code Scalability Attribute grammar Atomic number Oval Operator (mathematics) System programming Software framework Task (computing) Theory of relativity Subject indexing Uniform resource locator Computer animation Vector space Personal digital assistant Formal grammar Local ring
Complex (psychology) Open source Code Multiplication sign Projective plane Expression Line (geometry) Computer programming Mechanism design Computer animation Software Computer programming Universe (mathematics) Software framework
Computer animation Different (Kate Ryan album) 1 (number) Website
Domain name Computer program Programming language Service (economics) Sequel Connectivity (graph theory) Consistency Workstation <Musikinstrument> Independence (probability theory) Database transaction Instance (computer science) Mereology Rule of inference System call Scalability Data management Computer animation Code refactoring Right angle Abstraction
Implementation Group action Sequel Observational study Multiplication sign Virtual machine Electronic program guide Set (mathematics) Mereology Event horizon Computer programming Lecture/Conference Semiconductor memory Logic programming Software framework Compilation album Programming language Interface (computing) Database transaction Flow separation Compiler Computer animation Visualization (computer graphics) Thermal conductivity Resultant
Point (geometry) Distribution (mathematics) Consistency Multiplication sign Virtual machine Set (mathematics) Database transaction Number Front and back ends Subset Lecture/Conference Personal digital assistant System programming
um total rendering an end on its thank you very much not what can
and my name is what legal I got for edits to changing there's things concurrent and I'm new media said the DC them from Portugal and will tell you something about transactional memory to the so
before I start to get to the let's actually I set the scene here right so this talk is about concurrent programming and concurrent means that things happen at the same time but they're not independent right so this is slightly so different to call programming words say they also happen at the same
time but they usually independent you know like alliance and then you have concurrent actions but usually you know 1
conceptual problems state you need to make
sure that the current actions actually synchronized right so that it can make sense out of state transitions and when we look at Malta costs for example we have shared memory associates stage and we need to
synchronize on a shared memory so we have this thing called synchronization then you have
transactions so then you get the main transactional memory where it comes from on another topic
another concept that is very important it's atomicity and in the context of shared memory synchronization atomicity means that something
happens as an indivisible spend indivisible step right if you're database person think about this as atomicity isolation combined and this is why we have instructions such as x 86 compare exchange and to illustrate a little further where to messages are important let us at that on the small r O situation at everybody of you will have experienced already right so sometimes you have something like and now a hallway right just people need to pass each other so we're working to each other and usually somebody will step to decide but sometimes the people like this you know and they demand into each other we can't let computers do that and so what is the problem here the problem is that you know we are facing each other and I see the site and I think
it's empty right you know and sees the same side and also sees the things it's empty and then we do the both of the same sex right the the same situation and this is what happens so what's going on the problem here is a lack of atomicity if these 2 steps of seeing decide and stepping over would be in the visible we wouldn't have a problem right i see if I go to the site and then you know make sister right on the other way around so if we had this the 2 actions are atomic bomb than we would not have any problems so this shows you how atomicity is important right so during idea something like a test and set that you have hardware as well so this is how I can remember remember why atomicity is important
but and on the new
so coming back to the transactional
memory on if you want to remember 1 thing about transactional memory and please let it be this right transactional memories prodding abstraction and it allows programmers to declare which those sequences the
code should be atomic and instead of requiring to actually implement all this stuff this is the underlying vision this is how all the different kinds of the area where they give you and instead of the program and we have a generic implementations that
ensures atomicity so this is not specific to particular problem or something like that but what are its generic right and I can be pure software can be new hardware can be makes of hardware and software In our focus in this talk will we g and for high-level programming languages so what we're gonna do is that we have the 1st part in which I will talk about him for shared memory on a single machine I'll give you an overview of the sort proposed of of of language constructs are I'll give a peak in the Jesus use implementation of that and I'll make some comments and performance next of and we'll talk about the TM for distributed shared memory so multiple machines but the shared memory abstraction on he will discuss stronger
consistency and annotation of of news that our at work we have a short hearing a if there's something very short quick you know the alleged pressure that you have created this short all Absolute everything else goes at the place and warning know that also went to the house and now it is that the analyst or new the idea to the 1st idea of the proposed 20 years ago but real research didn't start until about 10 years ago and it's still ongoing heavily on standardization procedure + started about 5 years ago people got together informally and we have license cyclical study groups in summer 2012 this Jesus is support a since Apostol sites since version 4 . 7 and following also getting a real hot agenda plementations others so it's all has been out there for a long time but special-purpose audibly EEG carnallite under referrals real means instances that house transactional memory hardware on our this happens all task right if you have a new a lot of you actually might have a new workstation about with chip so how the unseen tuples of language constructs look like so we have this transaction atomic construct there and this allows a program which declared that the following compiled statement must execute atomic so in this case you will to X and the potential to lie in the store to buy all the executed as 1 indivisible steps all that clear our 0 specified give you explain sorry our specific semantics or there are no data annotations necessary on no special data tapes that approximates necessary right and then those transactions you there are and existing sequential code you can the function calls you can call other classes of you can have nested transactions and stops however the cold and transactions must be what we call transactions safe and among on safe code for example is the use of logical level tonics are assembler instructions volatile memory accesses or simply functions not known to be saved are however the compiler will actually checked whether you grow to safe right so you get a compile-time checking guarantees for that and because of functions are known to be safe on
for cross compilation unit called for and I
of France as a function of like for the calls of function pointers in each annotator functions with this on transaction safe attribute for example you say here if you make a claim that the food is actually transactions safe a further information have a look at the signal of specification best just web search for the paper number there and thing now how this concurrency and so there's a nasty little details so here's a slide about synchronization semantics
transactions extends this
11 suppose also level memory model so the memory model is the thing in the language specification that defines the execution of multiple products right which behaviorally actually allowed to have the transactions are is a matter of transactions are integrated into the memory model and the basic guarantee is that all transactions of poorly ordered and this border contributes to the happens before relation happens before it is basically the relation of models what happens on after what other thing in your problem if you have a multi-threaded execution and he TM ensure some valid orders to the GMM plementation shows how the what order that is consistent with happens before for every execution In other words this means that 1 transaction happens at the other end and make sense with respect to the other synchronization going on in the proper so if you have the intuitive meaning active actually pretty easy to understand that note over this does not employ sequential execution right this is a correctness guarantees if the transaction is actually independent of the implementation is also freed you execute them on parole for example but it allows to reason about the transactions as if they would be executed strictly 1 after the other however there's a gotcha and that is that they there is freedom is still required that is the case throughout all seen sepals plus right whether using large pool of Automics whatever that means that if you problem has data is you undefined behavior so I have a small example where I about publishing data and you have 1 correct on doing initially initialization of the data
and then end users in transaction to make the
data by setting a flat and the other of that uses a transaction to use data if it has been published this year is correct usage of doing something like that it's not because if you do the temporary reads in front of it you have the data rates new program and it was result you get on the undefined behavior
but as I said this is something that you have to consider anyway I went to see in support of programs and the reason why we have that is that this actually allows efficient implementations for the compiler 1 of the biggest benefits of transactional memory is
that it supports model for and so that means this is because farmers don't need to manage the association between the shared data and the synchronization in so shared data here application data things that you actually need to use our synchronize on the finer problems to right and a synchronization metadata something like a loss and the reason for that is dead because we have a generic term of limitation that takes care of ensuring the atomicity so that's 1 thing for example promised need to have a look convention on the default of the the following the 2nd thing that kind of follows from that their
functions containing only transactional our synchronization compose without that so the nesting or of transactions for example doesn't matter and however you cannot expect another threat to make progress in an atomic transactional concurrent
an atomic transaction this would make sense right you requiring that that uh the compiler environment to make you transaction to 1 indivisible so that you can go that 0 this is 1 indivisible step at all by the way in between I want this other ProPred to to make another cell right so this this would be conceptually incompatible and at the bottom I'm giving you an example for the for that right comparing transaction and ritualists so the example is that we want to move 1 element from 1 place to another and I have very little need small genetic function here where removed from runless indeed into the other synchronizing never making the save are with an atomic transaction is easy you just put in atomic transactions they move maybe an element 23 if you do this the blocks try to do this with blocks it becomes difficult 1st of all where do you put the locks right on the in the list i-th strongly to the list so if they're in the list you need to add an interface out for the new functions that you can create
locks stability lists actually have in the right if you have done that the next problem is that I use the how do you require lots and consists of or somebody's calling move baby in another thread is calling you move BA right if you just acquired a lot for the list on your 1st and then was 2 seconds will run into that what you have the problem of transactional
only because the implementation will take care of this if you for try to do something as generic like that but lots of becomes messy switching gears a little so how do we do it actually in the implementation so are his overlooking of our peak into GCC's implementation and it has 2 sides to it 1 is to compile a site and the others that have run some libraries
and there are 3 steps essentially that they do in the compiler except pollen and stuff like that so 1st year ensure the atomicity guarantee and we don't answer shortage your programmers data race-free but if you can assume that you did or did your problems that arise free then we can ensure at compile-time atomicity for a transaction and this is why this works because we check all transactions have code whether it is indeed safe and transactions that code is the code it sees implicitely transaction because you call quote from a place that should be transaction
safe or whether you have annotated functions accordingly once we have checked all this so we create an instrument that clone of the scope and so this code is that the transactions functions and so on and what they do essentially is that you take all the memory stores and rewrite them into this is that they are call into the TM runtime library that will then perform the memory also so we hope introduced Lotus or mechanisms such similarly redirect function calls to clones and as a
result we have both an instrument that call path you know where the to store so intercepted and an arms and instrumented 1 where things are just like before and then finally we generate begin and commit code for each transaction and 1 node and the 1 thing that's worth noting is that we allow the runtime library to decide on every transaction start with an instrument of the instrument of code to be executed and the whole point of delegating to a runtime library is really that get implementation of flexibility right so we
can we don't need to generate synchronization called in the compile time why at compile-time about a compiler but we redirect we have a little more overhead in terms of 2 function calls but full flexibility granted to do something you know different and better potential so on the runtime library site and the runtime library is a whole
by Chairman case of due to see this is the thing that and actually forces the atomicity under around her and the by 2 and
currently contains different our software only implementations told us here so these are quality and implementations that do not need any special and the default here is the 1 that it doesn't right through essentially to memory with under looking and use a set of intron locks to protect concurrent accesses to the same piece of memory between different transactions so manage the delivery of magic couple lots and automatic automatic mapping from the memory to the locks and then run a kind of a a little bit of both algorithm to actually make the 6 yeah answer this uses the understand the cold the currently in live channels make use of hardware can implementations so HTS and 1 thing that you might not be aware of that is that all the experimentations out there with the with the exception of of kind of special modem the parties the stuff is that they all best efforts in the sense that
they do not guarantee you that you they can run all transactions but some of the transaction I hope you try to accessible memory in the proper transaction then there's sort the capacity to track your excessive then it will simply fail to tell you know can do so you always need to have a fallback for your market transactions and that means that for example a new twofold back locking unity need to fall back to nasty and so on so the H T M by itself is not really that is so little thing and live by can only uses HTM what the global office forward that's very simple but it's also very non-scalable of fall like obviously on the good thing however is that allows
us to use the UN instrumented coal hot the instructions uh hardware but transactions sorry so that the hydrogen actions really don't have any overhead because of instrumentation currently I don't have any hybrid as GM-HMM minded yet and the point here but this is something that will hopefully have at some point the idea there is that we have hardware transactions they're very fast and then you can fall back to something that is actually still scalable so to
on the slides about performance and performance is difficult topic for 10 because GM has seen quite a bit of high and so let me 1st all the like everything else that's a pool it's not magic right and the performance goal of TM already performance delivery same person implementing shared would give you is that it tries to provide a useful balance between ease of use and performance it does not claim to provide the most useful violence on the only useful balance right on that it tries to make concurrency suitable for programmers and still deliver decent performance the goal is always to make it simpler you know except by you concerning some corner cases and then we see how good we can get and probably looks like performance can be decent the and the other
point is that right now I don't think it's are meaningful to try to draw conclusions from yeah formants and this is also 1 reason why I wouldn't give you any performance lots of such here because it would mostly be misleading and tell you why so 1st of all the implementations of work-in-progress solely by TMS looking progress has not seen a lot of tuning for example the H T M so first-generation implementations of a hot implementations I would claim that they also work in progress for example they like features that would be very useful for high-potential from marketing implementations and 2nd performance heavily relies on a lot of factors many that might be of many modern might be obvious so for example it relies on the hardware obviously relies on a compiler what kind of code to generate the actual channel with and use the HDM implementation for example you know how many memory accesses can attract for transactions so users what kind of cash or a dedicated offered used to actually try excesses How which other instructions instructions can handle right it was the spark order chairman limitation there was so restricted that you couldn't do function calls that's not useful for general purpose Chairman right useful if you use it in a very very tiny in a controlled way but it's not useful for you know the high-level TM abstractions the allocated matters because the allocator decides you know how you end up moving data around and very a data gets located memory and its full sharing issues of all that stuff whether use time optimizations or not doesn't matter because it affects the overhead and these are all just
all the factors that the implementation actually has control over on only the application programmers control over the actual transaction conflict probability how how long the transactions or but old story choose whether that many read-only transactions and so on memory access pattern data layout power allocation schemes again other code experience transactions and so on so there's a lot of stuff that's going on and the 3rd point I want to make is that I mentioned at the arm is still fairly new that means because the protein structure that people actually have to use right actively then we are still in a kind of chicken and egg situation so I'd
like to lower left to humans live ITN for real workloads but the currently have very few of them right so any tuning will be guest nonetheless
on limit tried to give you a few rough estimates that hopefully still hold in the future for TM forms and TM performance can be characterized by both models typically called a single clutch performance of a single thread overhead and the multiple threads performance so scalability essential regarding the former we can say that estimates slaughtered sequential code you have to synchronize so there will be more stuff right on the other hand how much overhead that will be can vary a lot from doing just the 1 1 critical section to you know this is something we examine Memory as will probably use lower or equal to course walking this is because his synchronized to release need to have some kind of atomic construction of all of these different will a change of something like that so that's like a lot R H will be aureus I about as fast as on continent critical sections or even faster because you don't need to change memory at all and depends on how it's implemented and so on but it sharing can be very fast provided the hospital transactions body decoder can actually run your transactions right if you do this is called a new transaction it's not a good probability that you know we will actually be able to do it depends on HDM again but this example regarding scalability on we can say that most STM with insult they're typically scale variable right you say you have a shared data structure you move elements there's no real logical conflicts in a transaction it scales well but this is less and less likely in these new law the single thread overhead is right because if you want to do have a lot of scalability usually do more you can just go coarse-grained lock or something like that HTM also scales well but again the full because a problem so if you for work there's a scalable and you have to use the full they often then and so this is why I
said that a hybrid SVM HMM applications will hopefully help to to get
ht and performance of most of the time but still have full like that doesn't talk in terms of performance and 1 point that is worth noting is that the and runtime libraries at that runtime the currently do this to little extent but it's a lot more that they can do this means that of
all the possibilities you get a good hold that automatically the implementation will pick the right thing to do right which would be very hard to do for programmers nobody wants to maintain and it's not a general library maintained 3 different or 5 different synchronization mechanisms just to be able to be high performance when it's used in different ways and if you still on worried about that didn't give you actual performance data that's a very easy way to get it you just need to use it right of graph C 4 . 7 or a more recent used that happened gesture and then measure performance in the applications right I read the Civil Hospital specification that's out there are report about the findings of blog about it if you thing doesn't a bark on and inefficiency in g Shoshie's implementation reported Barclays if you have feedback for arm in to construct a specified there it might be a good idea to get involved with the spices sequel Boston
specifications of this study group fight and if you really are more interested than the color please dive into just but implementation see what's there Our the commons and
little IT and called our fairly extensive so even though this is concurrent code it should be relatively easy to follow and there are many interesting things to work on for example improving the order tuning and so and with that will us you know discussed
something we that but and so as you can
see now we're going for a different world but it will still be addressing the simple and he was point we're looking at a single machine where we already understood that we need to make sure that these concurrent accesses the data are regulated that's we typically
like to realistic indications and there we have many many of these machines and the point here is that would be very interesting if we could explore these independent memories but look at them as a single shared abstraction and so what I will show you basically is how to do this and we
will use the same abstraction of transactional memory but in a different environment and so the
approach that we will refer to is basically distributed transactional memory and similarly to what we saw before is that we are bringing actions to the top of the software stack so there are no longer very down deep in Europe that the store in but actually they are out there closely application and so the cells for dynamic transactions meaning you do not have to specify a priori how long they are which positions they will axis and
this will be straight embedded into the application logic which is also rule for long lease transactions that could be different from what we saw before from Torvalds is that now we have to worry about these 3 things on dual so namely persistence because people industry and obligations enjoy this and the distribution because this raises other concerns in terms of 46 the cost of communication and of course because this is the main driver of why you're doing this reciprocation in most of the time was in 1st place and so just to give you create the list of concepts imagine having a set of data appear any have URIs to be consistent what to to be what we have until this point from Pavel is that we have a single machine but now we basically
just replicate these data because this is not
fault tolerance and so this is what we call a full
really fully replicated system however if you're going to change a bit of your data you have to communicate this to everyone and if you're gonna scale to water machines the stars become highly costing and so basically what you do is that you only replicating even that item in this subset of the machines which is a partial replication and so considering these I mean just try to
recalls convince you of why you should use strong consistency in the scenarios and for that let's consider what has been quite popular until some years ago and started perhaps 15 years ago which is eventual consistency so if you have these kinds of pure which says that the server i want to change is that i've been is no longer green it's yellow and then after a while it says I want to read the that I found them again is gets forwarded to a different machine and actually sees what was there before and not the most up-to-date and this is a reflection of what we typically say in popular uh saying that eventual consistency is no consistency at all because it's not formally defined you have to dig into your system and understand what he's providing to you to then realize what you can do with useful idiots and this varies from system to system and it's terrible for programmers and so in this case the problem is that the replication was the 1 going and so this is a very similar problem what you might have witnessed on Amazon phrases where you so you want to buy something an while the woman you're buying leads and then it pops up again or even your Facebook posts you you write something and after a while it's not there and other people can see and so this has convinced a lot of people and the most recent years we are now getting lot of popular systems which actually give you the extraction of a transaction that you might still have transactions which are stronger than others and 1 example is if you can use natural translation intersection there is still not perfect and let me present you an idea of why that is the case to have here a game where you have 2 points and they can never be into to use so the slots so what they do is they look around this see what's there and they choose the next move such that they will not collide with the boy it and that is what people do then they coherently move and they move next to each other breaking the rules basically
what happens is that the positions that they wrote did not intersect at snapshot isolation will only important section if it tried to something that was written concurrently and this is what is known as a right skew anomaly but notice that if they had moved 1 at a time as we presented to you work with a notion of sequentially
sequentiality that comes from the ethnicity
these would not have happened and that's what you get utilizable transactions is that every execution is allowed is equivalent to sequential 1 and so that's very easy for programmers to reason about and basically that's the
approach that we try to take is that we embrace serializable transactions which might be costly but for overcoming that we try to explore several things such as partial replication that I showed you earlier and we can will scalable protocols on top of that and you might have word of recent proposals such as Google spanner which are trying to move into this direction and so what we are doing here is not trying to save the soul the hunger pain world we are aiming for a simple use case but
hopefully the common 1 and for that we try to give framework which is easy to bootstrap has enough fast enough scalability and performance for most use cases and and has a lot of because which are hidden from the programmer of course this will not be the deal to make performance if you really need to have something critical and the approach that we take starts from a very key point
features that you specify the domain of Europe occasion in a domain specific
language and all these you know object-oriented way and through this we will hide from you concurrency control persistence and data placement in your distributed systems for some things you might still need your help as a developer and for that we give you API is an object-oriented manner for which you can do distributedexecution of code and ensure that all cultures and then we still have some things only for expert programmers if they want to go and so for the rest of these this guiding example where we have phone books which have context and the context may be shared among phones and this typically maps to some Java classes or whatever you your preferred language and then they might have uh they will really these relationships over there uh the uh some and collections or references and which have to be abated bidirectionally answer for and if you consider a typical approaching the industry which is used for instance I wouldn't you will basically rights encoding resembling bits and so you have to put these 2 collections on the 2 sides which you have to manage every time you change 1 side you have to remember to go on the other 1 annotated accordingly we have to worry about these annotations and manage the unique ideas of the classes and so forth and this actually this approach we believe has a lot of good things and and underlying it's and so we will be quite close to the so that's why I'm showing this example but will try to clear up
the review and focus on what's actually and
so the DSL I talk to you about we call it domain modeling language and for the example that I provides you basically you only have to specify the classes that you you will have your domain in this case it's very simple have phone book a their attitudes and then I say that they are related and their related the Indies many-to-many fashion and the is reading in a separate file the main file image of each scene that it should be easy for most to understand and this is all its you've already understood and they just so it for the 1st time and so a all it's of
DSL and basically the framework that we have and then uses these CSL to generate gold and it generate code with the well-known interface and these will basically boiled down to mapping the entities to classes we get the sense that there's hiding all the things that I talked to me about and then making the relations automatically managed in a bidirectional manner and so the framework will starts from your DML over there so the application has the the domain written by the programmer then do some code generator will be triggered generates a classes and they are placed in your application such that the programmer can extend these these of the these domain classes with the behavior because the only tells us I have a book with the structure is not held the behavior that's his job do the premise that needs to something like and then in the end these interacts with the rest of the application and so an example for the class that I presented to you are basically these is that you have a common gather centers for all the things in all the computer seen Persistent distribution I'm even in the so the interface will always be
the same regardless of the actual code implementation breeder much should be like what we just saw for and so the 2
then what you can actually do is choose
different transactional managers and their counts for persistence and you don't really have to change only have specified in a configuration file a for the rest of the talk I will use the guiding example of using Infini-D's vanities is that agreed open open-source from red and so what we actually do in this particular example that we have to map your domain to this persistence sense if things by music you felt store this with 1 and so we will some halted this mapping such that you don't have to worry about it and then you run your transactions Europe application
will map the 2 transactions in things by and and will ensure the seal viewed the huge but what matters that programmers are
not aware of these spots they might even be able to see if they want but they don't have to and that's what's important and the sees regardless
of the back-end that you're choosing whether it's infini span are any of and so if you were
still using the you know the process so the overall like hibernates phrases I told you we wanted to clear up the the the landscape and so this is what an application would look like using the domain and specify issue go with the amount you can clear these and in the end you only have the accommodates here which are related to what we saw in the 1st part of the talk but nothing else is related to computers your persistence it's completely human and even distribution and so if you want
to write an act an application we did which resembles what I just described with that's this file properties file and then you have 2 points the confusion files which are back independently in this case I'm using this example
In this example we need to specify some groups filed for communication with the system just use a pre-built 1 and we
have to configure doing things by balls rather easy we just say we want strongest consistency if performance X then we might want to delve evidence that the most cases this should be enough and does not bother the programmer with anomalies and you can write it in a safe way so you what you can see here is basically that we are asking for still as ability we use an optimistic concurrency control mechanism it's distributed using bar for replication and user multi-version written and all this is therefore free and independent of framework but OK you can also feeding other ways Frank might want repeatable read because you want to shoot yourself in the foot but in reality it's very easy
so you just below the application with a simple maven command specifying the code generator you might use another code generator besides if his by or others of our
this case I put example for hybrid to jam and the then it just it's we there yet another simple commands and all dealing all with all the boilerplate of the database distribution and so forth it's even from from and so for a sin for quite a long time so this is a website of my university and the therefore I think over 10 years this has been used there and this is serving go around 15 hundred people and do what is actually cool about it is that it started there it was made open-source so they will get a load of stuff on top of this framework which uh matters to the university itself in terms of the diplomas registering for classes you know all that
stuff and the and and actually this was made the inside the become used in other places as well and so it's running in many universities in India and actually 1 cool thing about it is that at some point the
created these map of the object the domains and basically what you're seeing is the instances of the domain model of the University caller by the part of the university belong to whether it's students professors and so on and these system which is in about over 1 million lines of code from correct uh we busy means of functionalities for universities it's running in several places so that's the cool part about and it's using the approach that I just described the risk and to so there are plenty
of advanced features that I could talk about but I'll just briefly go over a couple of them and in this case of just talk about the index relations which fission of these designed for technology about and you know want to disable a contact from your phone book evening might have to write code somewhat like this which basically you need to go over all the complex and then you say OK this is the contact I want to visible but this is crappy if you have a lot of course and so what happens here is that you can simply say 0 I want to index my contacts by given atoms and then we'll give you a very efficient index that will solve the problem and you can just say OK I'll get my contacts and this should be efficient independent of the vector so of course to get these features there I was just trying to think of fighting the beginning medical things about the location so we don't want the grammar no where things are and it's important because you might drop servers at runtime bring new ones to have provisioning how we cannot let the programmer know exactly where things up but he can try to to to play between its and here's what you can say is that he wants to point
out that this contact should have should be placed using these attributes what the cells view is that if you want to run some codes we can create a task to run in a distributed system which will use these contacts and then will use the locality of the contact to be placed and what this means is that these codes these this task will writing a machine where the contact is guaranteed to exist as so these also have co-location of data and codes which is important in the 3 seasons for voids fishing remote operations to other servers medical things that can co-locate states and the and that simply using the same technique where you can place these 2 context in the same machine by using the hints of course ultimately the framework has the choice of not doing that because you might just with everything the same machine and that defines the scalability the of the of the of and so in summary we know that the the cations are hard to develop and it's
in particular if the programs are exposed to low-level mechanisms that they have to
unions configure and so on think here is that the eyes should hide the complexity every time it's possible but still allow expressiveness whenever they need needed by the program and so that's exactly what we tried to do here are some references so this was done in the scope of a new European project the framework I described to use open source here you the corresponding recommendations and the universe stuff that I told you about over a million lines of code using several neighbors these is there with lots of the documentation and nice manuals and so if you want to get involved in these there is a and research network called the rhodium sponsored by European Union and
it's basically bringing unity new together it has a lot of countries belonging to it
and you can participate in namely workshops which are coming soon in April in Amsterdam The your attempt and fund their participation in these workshops also training school which will take place in offline and France also soon air where will have the speakers from Intel red hats and Zisserman other even Oracle so you've and other industry members and you can also collaborate with other institutions and just check the website for more from and that's the thing different
with the got questions just raise your
hand rule of you and I give you my phone at the end and I have a question regarding the clock part so I see I got police or everyone please the court didn't understand the question I have a question regarding the call on uh I we have seen that the uh why consistency is needed so what I don't understand what is the proposal the solution do I introduce a 1 central of management station would say that that there is going to handle this for how likely the distributed so the question is are basically whether the proposed solution is using centralized components to be alleviated distributed data management and the answer is no uh but that might be in that on the back and teachers so we have backend which manages data in a completely decentralized way which is favorable for scalability and we have used these with over 100 machines but if you choose the which of the instance we have support from my sequel and you can run it through a centralized database and in that case you're
asking for it and you get its centralized components so it's orthogonal and independent and your questions over the think I don't know answers when they were kind of latency on the transaction management when is introduced like this OK so once again that might on the back and that you're using and so your the question was what kind of latencies we we notice that the service right yeah you know kind of factors involved right uh whether so 1 of the main problems is the data placement so by providing the programmer with the simple abstraction of the domain language where they can specify the mind and not actually not asking the programmer to place the data you might have intersection which is spending a lot of
different machines kind if you have my sequel shot at and I are talking with a lot of shots but they're hopefully these can be fixed and actually there is an internal part of the framework which is right notices tracks accesses and moves data to be co-located to make sure that if you axis x
and y together a lot of time then x and y will be together at the there are several efforts regarding implementation there are several efforts to guide implementation of conduction memory as far as I understand it the standard but there are several efforts from of from GCC for and from microsoft I assume also is there any effort to synchronize cells so 1 of the efforts is definitely the study group flight from ISO C + + so that all that is standardising the language constructs that you have so the interface the program receives and there are people more from Oracle red-headed are Microsoft currently not IBMs involved HP so quite a set of logical people and academia as well on the API are Intel in red that have been standardizing or have a document out there that our specifies the API that's used by dint of apologies to see this is answered questions the you mean the language for event or transactional memory part is that the SVM is implemented in C + + years results what compiler is it to be implemented invisible suppose that's compilable Visual C + + no I don't know about democracy sorry OK
1 last question so in case of a distribution distributed to the back end of could deliver uh example How is it solved to this problem that will demonstrate that with of the eventual consistency pockets of the main problem is that there is this very non problem industry systems called consensus which is that you have different machines that need to arrive to the same conclusion to anti-Semitic transactions you basically have to solve Consensus and basically what what happens is that if you want to use that to either be the you always have to solve Consensus and if you're set of machines the lot you'll be paying a high cost every time so the main 3 Claire these to make sure that every transaction only such as allele number of machines and that's why it's bleeding get the data in a smart way the papers a long way to make peace efficient so if I have to say that the main key point that makes it possible is what i presented earlier as the partial revocation so dividing and making sure that you only replicated taking a subset of the machines and that in the smart way such that you are resulting consensus between a small number of machines it that's because again