Hyper Hyper Space: In-browser p2p applications
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 287 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/57048 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Data typeScripting languageLibrary (computing)HypercubeEntire functionInformationObject (grammar)Process modelingJava appletMessage passingHash functionData modelString (computer science)Boolean algebraContent (media)Set (mathematics)Element (mathematics)Revision controlInternet service providerOnline chatRule of inferenceData storage deviceSpacetimeLibrary (computing)Semiconductor memoryPower setUsabilityObject (grammar)Set (mathematics)Message passingPeer-to-peerSoftwareInformationDifferent (Kate Ryan album)Order (biology)Web browserCASE <Informatik>Virtual machineCodecHash functionPhysical systemMathematicsData typeBefehlsprozessorNeuroinformatikData modelCartesian coordinate systemOperator (mathematics)Inheritance (object-oriented programming)Server (computing)Web pageWeb 2.0SpeicheradresseCommutatorRule of inferenceMultiplication signAdditionResultantGroup actionContent (media)Revision controlInternet forumDatabaseAuthorizationEndliche ModelltheorieBitThermal conductivityIdentity managementUniverse (mathematics)Type theoryState of matterNumber1 (number)Table (information)Validity (statistics)Functional (mathematics)Instance (computer science)Social classVideo game consoleSubject indexingComputer programmingState observerSystem callSynchronizationClient (computing)Polygon meshConnectivity (graph theory)Commutative propertyConnected spaceLevel (video gaming)Online chatAbstractionSlide ruleDirected graphFigurate numberDoubling the cubeTouch typingRandomizationConsistencyMatching (graph theory)Source codeProcess (computing)Pearson product-moment correlation coefficientCategory of beingLattice (order)Ocean currentService (economics)SequelDiagramComputer animation
09:39
Message passingHypercubeOnline chatRule of inferenceData modelOperations researchCommutative propertyInternet forumElement (mathematics)Validity (statistics)Invariant (mathematics)AerodynamicsObject (grammar)Hash functionIdentity managementType theoryPeer-to-peerBroadcasting (networking)SynchronizationStructural loadSource codeCore dumpPower setConstructor (object-oriented programming)Operator (mathematics)Library (computing)Category of beingAdditionSoftware testingBitCausalityPoint (geometry)Internet forumMultiplication signData storage deviceMessage passingRevision controlSynchronizationSpacetimeType theoryCartesian coordinate systemHash functionIdentity managementSet (mathematics)Object (grammar)Rule of inferenceGroup actionVotingPeer-to-peerRight angleCommutatorHypercubeLink (knot theory)CodeVermaschtes NetzState of matterOnline chatServer (computing)Data typeCore dumpBellman equationStructural loadRootDirected graphEmailMereologySoftwareElement (mathematics)Computer fileFunctional (mathematics)Invariant (mathematics)SummierbarkeitDifferent (Kate Ryan album)Position operatorConnected spaceKeyboard shortcutData modelFinite differenceSource codeComputer programmingComputer configurationField (computer science)Information technology consultingInformation securityQueue (abstract data type)Touch typingFitness function2 (number)ReliefWebsiteWorkstation <Musikinstrument>Musical ensembleDivisorFocus (optics)Dilution (equation)Graph coloringLocal ringRoutingComputer animation
19:10
HypercubeSource codeCore dumpComputer animation
Transcript: English(auto-generated)
00:08
Okay, welcome! We are here to talk about Hyperspace, which is a library to create peer-to-peer applications. The idea is most of our devices have lots of
00:23
memory, lots of storage space, they have good CPUs that can do all the computations that most of our applications need, and more importantly most of our devices have web browsers, so the idea of this library is using web browsers as kind of a universal virtual machine where we can run
00:44
peer-to-peer applications virtually everywhere. And what we're trying to do is to create data types that are very, very, very easy to replicate, so we will when using hyper-hyperspace we will never do an RPC call or make a call to
01:04
another machine to try to get that machine to do something for us. The way we do things in hyper-hyperspace is that we always replicate whatever information we need locally and then we work on this information on the device where our application is running and then whatever changes we make will
01:22
be replicated back to the rest of the people using this application, to their devices. So ideally we will make one data type that will represent the entire state of our application and we will give this data type to hyper-hyperspace and it will keep it synchronized with whoever else is using the application. So essentially we will be creating a new layer, our
01:47
application, I mean in a client server application we would deal, this layer would not exist, this information mesh layer, and our application would be dealing directly with network abstractions like hosts and
02:00
connections and maybe HTTP verbs like post and get. However in this model where we have this information mesh between our applications and the network we will be using high level of abstractions. We will have a store, the store will be local to each device where we will save objects. These objects will be
02:20
created using hyper-hyperspace modeling library that we will see in a bit. And then there will be this sync component that we will configure using something we call peer groups, which is the group of devices that must keep some of the objects we create synchronized. So let's take a look at how that would work. So basically this store will just store typed immutable objects
02:46
which are shown in this table in the slide and this is for example if you go to a web page that is powered by hyper-hyperspace and fire up the console you will see an indexing DB database with things that look like this
03:00
where you will have JavaScript literals that contain object contents and then you have the hashes of these objects and then one tag that indicates which type each object has. And then when we go to our data model we will see that hyper-hyperspace exposes a base class called hashed object and all the
03:25
objects that we want to store and that we want to share must derive from this base class. And this class will give us two things. It will give us a consistent way of hashing the contents of the object and it will give us a way to
03:40
transform the object contents into a JavaScript literal. And it will ask us to do one thing that is to create a validate function because later on when we are synchronizing we will receive instances of this object over the network from untrusted peers so this validate function will enable the library
04:02
to check whether the information we are receiving is valid. Finally one observation if one object references another for example here we have a message object that has an author which is another object which is of type identity and this reference that in the running version of our program
04:22
will be a memory reference a memory address will be replaced by a hash based reference. The hash of the identity which would be another immutable object will be used to internalize this reference inside the store. So we will just create these objects as we usually do using new and allocating memory in
04:45
in our running program and then we will call a method called save that hyperspace store will provide and this will automatically create the entries this immutable entries inside our applications store this again if we are in a browser this would be indexed DB and if we if we are running in a
05:04
server this would be some database probably SQLite which is the one that we support currently. Of course this presents some challenges for example if we have mutable information in this example the author is trying to change the contents of his message so he replaces the text that used to say hi for
05:24
hello maybe he wanted to be a bit more formal I don't know but this doesn't work because when you save this to the store the contents of the object change so you get a new hash so there's no relationship between the message that you saved before and the message that you have now so this as a
05:43
way of mutating information this will not work so we need to come up with something else. So what hyper hyperspace does is using something called operation-based CRDTs. CRDTs are very well documented data types using these kinds of settings where you have several devices modifying the same data
06:06
types. Here's an example with a set so we create a set using new and save it in the store and this will create an object just with the immutable ones we were looking at and create a hash value and there's the technicality here all
06:24
this even this is if this is a mutable object it will be generated in a known initial state in this case in this case the empty set and of course if we hash the empty set we will always get the same hash number so that there will be a single set across our whole universe so the system adds a random seed
06:44
to each new set to get a different one and then as we make changes to the set in this case we are adding apple and orange and then saving the set to the store this will create new objects new immutable objects that are the operations are adding this set and the operations are just like any other object that we
07:04
have been creating and they have references that are also hash-based indicated which is the set that they want to modify so here's well following on with the set example we say we want to delete the Apple so the essence
07:25
of this CRDT data types which means conflict free replicated data type is that no matter in which order these operations arrive because different peers in the network may receive the operations in different order the result
07:40
must always be the same so in this case this is what is called an observed removed set and when we delete the Apple if you look at the operation object that is being generated you will see that it doesn't delete the Apple but it actually points to the addition operation that was adding the
08:01
Apple so if someone else added the Apple at the same time we were deleting it there would be no ambiguity about whether we are deleting that addition or not I mean either we received it and we will create a delete up for that one or we didn't so eventually everybody when everybody
08:23
receives these operations the result will be the same for everyone this is important so here okay we're just restating that this is a commutative property between the all the operations the type supports and well
08:41
hyper-hyperspace provides some types you would expect for example a mutable set a mutable reference but if you want to create other mutable types there's a mutable object and you can derive that and define your own operations and as long as they if they respect this commutativity rule everything will be fine so moving on here we have a say we want to create a
09:03
chat group and we want to create a data type that will support this chat group through hyper-hyperspace so this is maybe this is a simplified version but it will be something like this we have an owner that created the chat group and then we will have a set of moderators that are not responsible for enforcing that whatever rules I know rules of conduct is this
09:25
group has are being respected by everyone and then we have the set of members which are the participants of the chat and then we have the set of messages right maybe in a more we could be more ambitious and we could try to do away with the owner and have some kind of voting but just to keep it
09:43
simple let's imagine it works like this then we have some rules that we want to enforce through our data type for example that only members can post messages that the moderators are designed by the owner and that members can delete their own messages while moderators can delete other people's
10:02
messages okay and if we imagine this this for example this latter rule later rule it couldn't can generate situations where commutativity would be broken for example say we have a moderator that's called Alice and at the same time in two different peers she is removed from the set of
10:21
moderators and concurrently she uses her moderator rights to say remove a message from Bob right so depending on which one of these messages you receive first the deletion of Bob message would be valid or not because if it comes after Alice is removed from the moderator set Alice would no
10:43
longer be a moderator and she wouldn't be able to to do this so hyper hyperspace adds some some more constructs to to deal with these kinds of situations and we have a notion of operation invalidation so we can say that operation we've seen a bit can become invalid and then that we have
11:03
explicit causal dependencies between operations we can say that one operation is dependent on another and we have some special data types to use these properties one is called for example a causal set which instead of just addition until a deletion has an additional operation that's called a
11:23
test and a test would what a test will do is that it will say that at a given point in time one element is a member of that set and there's an asterisk in delete because the deletion in a causal set is a bit more sophisticated and it will mark the point in the causal history where the
11:42
deletion happened in the local store of the peer that is creating the delete operation this is kind of complex just bear with me so when Alice creates her deletion up to remove Bob's message because she found it offensive or it was
12:01
against the rules or whatever she will create a causal dependency first of all she will attest that she belongs to the set of moderators and then she will so this is these are two different operations the attestation is an operation and then the deletion of Bob message will be another operation and this operation will have property saying that it depends on the attestation
12:25
being valid and now since the deletion that Alice the delete operation that removed Alice from that set has marked a position in the history in it's the local history of that beer but either the attestation got there before he or
12:46
she deleted Alice or not and that will be the same for everybody because this picture of his local history will be shared alongside the delete operation so everybody will agree on whether this attestation is valid or not
13:00
and if it is valid then the deletion of Bob message will go through and if not the attestation will be undone and since we know that there is a causal relationship between the deletion of Bob message and undo will cascade and that will be undone again so this is this this gives you a feeling about how we can compose CRDTs to create more sophisticated rules using the
13:26
hyperspace engine so just to recap our immutable data objects are hushed and are literalized for sharing and we need to provide a valid function so
13:41
when they are replicated the engine is able to the sync engine is able to know if what we are getting is right or wrong we will use operational CRDTs to get mutable types and finally invariants will be modeled if necessary using explicit causal relationships and invalidation and of
14:00
course everything will be persisted as immutable typed objects and we will use hashes for referencing and for the identity of these objects so this sums up our data model and I want to quickly go through how sync works so basically sync each dot in this drawing is a peer which
14:23
would usually be different device and peers in hyperspace are a pair they are a network endpoint and an identity the identity usually will have a keeper inside and then we'll have a second concept of disappear group which is how each device will initialize this mesh network and this peer group is
14:46
also a pair one is a local peer indicating who we are in this mesh network and the second one is called a peer source and its role it has basically two functions one is giving us random peers to connect so we can get
15:01
some neighbors in this mesh network and the second one is if we receive a connection request from a potential peer it tells us whether we should accept it or reject it and so once this this mesh network is configured is it will basically do two things first it will gossip about the state of all the
15:21
mutable objects here's an example someone found a new add up and it's sharing this with the rest of the network and the second is once through gossip we find out that we are behind some that we are missing some updates there will be a sink this is not how the sink actually works this is just for for show and so basically yeah you you will see that you need
15:45
an object that hashes to something and someone will send you a JavaScript literal you will hash it you will see if the hash is right you will probably see that you are missing some references so you and you will have their hashes and you will ask for more literals eventually you will be
16:00
able to reconstruct the object and then you will be able to call the value function to determine whether your node should receive it or not it's not like this actually it's more like it what it does when when you pull or push stuff on git and it's more like you get headers for all history and it
16:20
finds the places where it forks and they're optimized things but you get the feeling it's something like this and finally there's one more thing because I mean we got a lot of objects in our model but but usually there is one root object in our example it was a chat group object that had all the containers inside so usually something called a space is
16:43
generated from this root object and spaces can be broadcasted and looked up in this mesh networks so that's we that's how we for example how we would join chat group usually in hyper hyper space and this is a bit say
17:01
it's a bit like a file space is a bit like a file but designed for for working on a network setting and what you can broadcast and synchronize later all the objects that are within your space this is usually part of what the
17:22
chat group would do automatically for you I mean whoever designed the chat group will will design its space and how its synchronization should work okay so we're basically done now we've got an idea about how hyper hyper space application would work we will define some data types and we will configure a
17:43
mesh network that should usually be like set and forget we will just do it that at the beginning and then we will work on this local store we have I know save operation a load operation or where we give the story hash and it will give us back an object and then we have a watch operation for the
18:01
mutable objects so we can ask the store to keep one of our objects the objects we have in our running program and up-to-date and since the store is being kept synchronized by by hyper hyper space just by keeping our version in sync with the store we will have the latest version well then there are
18:21
other things that I know they are they are react bindings so you can feed these objects directly into the state of your UI and things like that but that's basically it I just wanted to give you a feel how this works and okay there will be a Q&A session in a bit here are some links about hype happy
18:41
space website the core library that does all the thing and oh and there's a link to the chat group example we have been discussed so you can look at the actual code and I think there's also two more things a discord server in there that you can join if you want to discuss things and there's a
19:01
white paper discussing the things I have glossed over in greater detail okay so thank you very much see you in the QA session