Cost-Efficient Virtual Petabytes Storage Pools

Video thumbnail (Frame 0) Video thumbnail (Frame 1205) Video thumbnail (Frame 2667) Video thumbnail (Frame 6501) Video thumbnail (Frame 7571) Video thumbnail (Frame 10245) Video thumbnail (Frame 11427) Video thumbnail (Frame 13616) Video thumbnail (Frame 15022) Video thumbnail (Frame 16392) Video thumbnail (Frame 17887) Video thumbnail (Frame 18809) Video thumbnail (Frame 19849) Video thumbnail (Frame 21354) Video thumbnail (Frame 23252) Video thumbnail (Frame 24291) Video thumbnail (Frame 26276) Video thumbnail (Frame 27182) Video thumbnail (Frame 28361) Video thumbnail (Frame 29452) Video thumbnail (Frame 32412) Video thumbnail (Frame 35126) Video thumbnail (Frame 36284) Video thumbnail (Frame 39253) Video thumbnail (Frame 40106) Video thumbnail (Frame 41181) Video thumbnail (Frame 45024) Video thumbnail (Frame 45968) Video thumbnail (Frame 47311) Video thumbnail (Frame 49274) Video thumbnail (Frame 50454) Video thumbnail (Frame 59959) Video thumbnail (Frame 63530)
Video in TIB AV-Portal: Cost-Efficient Virtual Petabytes Storage Pools

Formal Metadata

Cost-Efficient Virtual Petabytes Storage Pools
using MARS
Title of Series
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Background data migration via MARS on sharded local storage is the key for massive cost savings and even better total performance, compared to big cluster architectures using expensive dedicated storage networks.
Keywords System Administration

Related Material

The following resource is accompanying material for the video
Video is cited by the following resource
Slide rule Goodness of fit Computer animation Multiplication sign Bit Theory Digital video recorder
Point (geometry) State observer Service (economics) Multiplication sign Source code Virtual machine Design by contract Set (mathematics) Repetition Code Event horizon Number Computer architecture Centralizer and normalizer Internetworking Hierarchy Square number Energy level Cuboid Error message Partition (number theory) Address space Physical system Moment (mathematics) Debugger Data storage device Directory service Variable (mathematics) Flow separation Category of being Cache (computing) Computer animation Personal digital assistant Formal grammar output Spacetime
Pulse (signal processing) Slide rule Optimization problem Data storage device Set (mathematics) Student's t-test Web 2.0 Computer architecture Computer animation Different (Kate Ryan album) Search engine (computing) Search algorithm Cuboid
Point (geometry) Trail Source code Execution unit Similarity (geometry) Open set Client (computing) Dimensional analysis Computer architecture Internetworking Operator (mathematics) Computer network Computer architecture Stapeldatei Structural load Data storage device Denial-of-service attack Volume (thermodynamics) Storage area network Flow separation Human migration Computer animation Logic Video game Quicksort Cycle (graph theory)
Slide rule Dataflow Game controller Statistics Service (economics) Execution unit Virtual machine 1 (number) Insertion loss Replication (computing) Distance Internetworking Different (Kate Ryan album) Computer network Physical system Pairwise comparison Key (cryptography) Moment (mathematics) High availability Database transaction Überlastkontrolle Incidence algebra Storage area network Particle system Arithmetic mean Database normalization Computer animation Buffer solution Geometry
Computer architecture Service (economics) Computer animation Inheritance (object-oriented programming) Personal digital assistant Multiplication sign Phase transition Data center Virtual machine Incidence algebra Normal operator
Functional (mathematics) Distribution (mathematics) Computer file Block (periodic table) Data storage device Sound effect Basis <Mathematik> Incidence algebra Client (computing) Computer architecture Propagator Computer animation Personal digital assistant Different (Kate Ryan album) Object (grammar) Quicksort
Game controller Service (economics) Multiplication sign HD DVD Constructor (object-oriented programming) Data storage device Client (computing) RAID Number Cache (computing) Computer architecture Arithmetic mean Computer animation Bit rate Order (biology) Computer network Endliche Modelltheorie
Service (economics) Execution unit Data storage device Flow separation Substitute good Database normalization Computer architecture Computer animation Personal digital assistant Operator (mathematics) Theorem Right angle Endliche Modelltheorie Figurate number Analytic continuation Spacetime
Slide rule Pairwise comparison Database normalization Computer animation Different (Kate Ryan album) Source code Distance
Point (geometry) Slide rule Addition Functional (mathematics) State of matter Multiplication sign Source code Data storage device Client (computing) Storage area network Replication (computing) Distance Number Database normalization Befehlsprozessor Computer animation Personal digital assistant Different (Kate Ryan album) Order (biology) Data center Video game Collision Symmetric matrix
Default (computer science) Divisor Multiplication sign Data storage device Set (mathematics) Normal operator Befehlsprozessor Computer animation Predicate (grammar) Personal digital assistant Data center Website Symmetric matrix Form (programming)
Virtual machine Replication (computing) Field (computer science) Number Product (business) Single-precision floating-point format Computer hardware Cuboid Endliche Modelltheorie Associative property Physical system Exception handling Default (computer science) Moment (mathematics) Data storage device Horizon Volume (thermodynamics) Instance (computer science) Measurement Type theory Befehlsprozessor Computer animation Personal digital assistant Logic Data center
Slide rule Multiplication sign Moment (mathematics) Virtual machine Incidence algebra Flow separation Number Type theory Computer animation Meeting/Interview Different (Kate Ryan album) Computer network Software testing Resultant Writing Social class
Human migration Category of being Process (computing) Computer animation Data storage device Virtual machine Cartesian coordinate system Replication (computing) Asynchronous Transfer Mode Physical system Spacetime Directed graph
Web page Group action Multiplication sign Source code Virtual machine 1 (number) Similarity (geometry) Rule of inference Perspective (visual) Product (business) Bit rate Computer network Cuboid Energy level Metropolitan area network Physical system Addition Channel capacity Moment (mathematics) Total S.A. Volume (thermodynamics) Flow separation Human migration Subject indexing Computer animation Logic Personal digital assistant Hard disk drive Video game Right angle
Ocean current Server (computing) Functional (mathematics) Java applet System administrator Set (mathematics) Branch (computer science) Product (business) Number Revision control Programmer (hardware) Bit rate Different (Kate Ryan album) Synchronization Office suite Physical system Moment (mathematics) Projective plane High availability Database Total S.A. Bit Incidence algebra Orbit Human migration Process (computing) Computer animation Software Personal digital assistant Asynchronous Transfer Mode Spacetime
Standard deviation Weight View (database) Feedback Moment (mathematics) Virtual machine Database Virtualization Line (geometry) Client (computing) Instance (computer science) Cartesian coordinate system Flow separation Computer programming Web 2.0 Process (computing) Computer animation Bit rate Symmetric matrix
Revision control Area Implementation Computer animation Divisor Multiplication sign Feedback Projective plane Cartesian coordinate system Plug-in (computing) Orbit
Area Process (computing) Computer animation Meeting/Interview Operator (mathematics) Streaming media Router (computing) System call Physical system
Java applet System administrator Multiplication sign Source code Survival analysis Real-time operating system Mereology Replication (computing) Order of magnitude Bit rate Meeting/Interview Computer configuration Square number Cuboid Endliche Modelltheorie Social class Physical system Area Scripting language Data storage device Sound effect Bit Instance (computer science) Flow separation Type theory Exterior algebra Point (geometry) Slide rule Open source Computer file Variety (linguistics) Virtual machine Similarity (geometry) Drop (liquid) Distance Event horizon Metadata Scalability Product (business) Number Hausdorff space Computer architecture Natural number Internetworking Standard deviation Forcing (mathematics) Projective plane Total S.A. Volume (thermodynamics) Cartesian coordinate system Limit (category theory) System call Human migration Word Database normalization Integrated development environment Logic Personal digital assistant Communications protocol
Computer animation
and good evening ladies and gentlemen welcome to my audience is a little bit smaller due to the time but hopefully um I could please you with from the is some interesting ideas and i if you have questions you may
ask from also I will stay in English because of the use of the video recording and the audience so I would translate any questions went to endure so what's the subject of the talks if theory and it's about cost but costs of course not the only thing to be was the next slide here is so this is my
and I have in architectural level for us that's the problem space of course the central subject of course is not all to look at so there's a scaling properties which is very important for any big storage systems and of course the 2nd this the right liability this the most broadly then of course the full of course is sometimes often would've thought 1 in in the hierarchy to the 1st 3 items are about problem space and that the solutions but of course OK in so let's start with 1 architectural which is proposed for of the internet and high at the moment which I call a big cluster architecture let's start from the top here and should I use this half for the other 1 for pointing in this 1
OK now let's look what what is a rep o storage of several millions of contracts and several millions of whole directory I think our last so my last called in was about more than 9 billions of all directories OK now these that each home directories already a separate space separate from each other as so your input data each but contract is already isolated or should be isolated from each other that means your data set is already partition this is an important property in and now if you want to have a a cluster architecture storage here you have 3 variables what is the was an error then you have some front ends here try our service in your address your http request and then you have the storage networking events some start start from somewhere if you are using a big cluster architecture than in the audience almost any fraud and codes sort any request for many users and the same is also true for the storage boxes that means the data is striped in some way over all of these storage boxes that's the basic idea of because the architect and if you look at its 1st year real-time exist between your fondant storage as because if something is not in the cache is here the local guesses of you call your for and and it has to be fetched from the store source and this is a real-time access at the next observation is you have and you have an machines you can imagine if several petabytes of data the if you have millions of customers and several thousands of machines so this n is bigger than what falls into the art and what it means to have over n squared connections that is the number of connections going if this grammar of course the traffic itself service giving up on linearly in but the number of TCP connections which are open at the same time schools and square that's the important point to observe here OK is this and understandable because potentially at any of these for the machines can request data from any of the beggars and to it and in case so and draw the this is not the best architecture if you have a dataset which is already partitioned there are some use cases that let let me emphasize
this 0 which short which are well before we went from this architecture for example looking at the search engine and user may enter any search keywords and then many of the fondant any of the 4 and it might be addressed so is it makes a difference whether you not all your data set but also a says pulse are already partitioned in some so for example if you use for students in architecture for 4 years storage boxes here that are used in this research is the optimal solutions but not for all web hosting here just to make it clear OK now let's look at it
and something is going on here of course that was let's the next slide
is the Shouting architecture known probably known to some of you and the idea is you have storage + 4 and and in 1 single unit which may be the same story about them also maybe in the stores source more stores sort and several
clients but in small islands even if you have some some small rented a storage network here between them is that local which following so you have no big old and square that work between and now the problem is openings in here is viewed as the most important problem and the idea of what I'm doing here is you have a small network which is all the all of em and its use for batch migration so you logical volumes of friends failed during operation during the data is being updated that's an important point the data is active during that's true data can be migrated to different our from opening signal for example for harder life cycles to get rid of your old source and migrates to newer books and so on the idea is that the this migration network is much smaller it's only all of n and its spectral track so you don't need to dimension your network in such a way that you have plenty of resources for a new node peaks the main problem the 2nd goal is some of the other architectures similar architectures is that's typical loads from the internet for example DoS attacks q it's similar scenarios are overloading at the wrong
moment upgrading packets stops and um a key is no real-time particle and it has not been designed for storage networks has been designed for nuclear and so as you probably know for from the history and this means statistic send buffer is secure by concept so if you opened they're all on the means of congestion control of flow control in unit expected loss but the basic principle of the Internet and this means you storage network from the 3 the 3 previous slide must be dimension in such a way that no incident had said you go don't get you and news has to be over dimensions and for this replication network um you simply can't
use bekannt so some some big you can even use traffic shaping the you OK now let's look at a comparison of the ideas behind it if How to get reliability high availability into the system the classical solutions some of you probably already know is the RBG gene was already was the but your similar systems OK 3 far 4 people here OK thank you so you know what I'm talking about if you want to know what meiosis it's in the same almost the same as the RBG but I think for us the main difference is that meiosis using transaction lots so all the updates are 1st recorded in a sequential and then transferred and back brought to the secondary side so you have pairs of machines typical so we have a primary sigh you normally are operating and the secondary side it's passive copying which just narrowing the active what and of course on you or 1 of them is actually service and the press the other ones the back up and if this distance the physical distance between this 2 resides is more than there's a thought your 50 km the neural up typically toggle geo redundancy in and cheer redundancy means that's the whole dataset from a failed for example if you have an elf
Drago similar what terrorist attack or whatever then you can use your data center and you have a complete back update is an array all your data society so we can switch over in case of emergency now let's look at some of the cases where only a single notes are favored failing this can occur when during normal operations um for at any time for example if you're for 1 thousand service and reliability of 99 . 9 9 per cent for 4 months total then it means that you will have a bolts 2000 hours of downtime in total of neutral to pool per year about around this 10 could could you would it is small than many people are imagine now let's look at the not the case that's true nodes are fading at the same time the so if they're afraid
at the same time here in this architecture it's no problem if they are on different parents because you see this notice this red 1 is fading US trust switching over here have a fatal to the other 1 and if this 1 phase your free legal developing novel data so it's not a big incident from the whole of the whole data center but only a small incident of your local machine and all your local descriptor are connected to this machine and of
of course no longer accessible if the same happens with the big cluster here whatever it is you can add other only into these are just examples if you have to replica here what are the basis of the problem half typically is cryptic us in in case of such a they're all objects as the corresponding to 1 file to 1 block
whatever all using only on using you which were afraid and you have to work because you have a problem because not all of your data is present anymore and the main difference is here it's metals which notice fitting so the probability for hitting it said the this same pair is much slower but hitting any of them for producing incident here is much higher by an order of around that OK so this is the 1st problem the because the architecture and the 2nd 1 if you have a storage into the storage layer you will likely have also an incident at your clients or worse but look off 1 of them but almost all of them because sort it depends on the data distribution and their functions and other effects but I've written here over all over and clients are typically affected all maybe a figured this inverse case so the spread of your incident is much more the latter kind of our propagation
here throughout this over and strong network so it is clear to you that that architectural impact here is a poor because it's not enough to you and in order to competed with this architecture you need at least 3 replica if not for OK agreement disagreement somewhere I think it's a it's clear from high will need not exact mathematical Forrester conceived by just looking at the problem OK no the question is what does that mean
for your costs OK if you have a big cluster here that you needed history because it means for 100 per cent of disks times 3 and by the very construction you have storage node and you have client nodes and they have the same or old and similar numbers to are using 3 to about 3 times the number of service you would need for this model simpler frowning model is you have just local rate 6 nothing else and the Data Protection stunned by the raid preferably how the rate control this DB you get this is very important for performance and you have how a caching
their and better Rebecca up or some smuggled kept compensatory in case of power failure of and there are others as injury with smaller cars so you're not wasting any age unit here and power consumption is also very small by this model because you have no dedicated storage service at all in this 1 columns could use that much power but not much space OK uh but of course you need the bag up but you needed in all cases because you have no full redundancy few whole cluster of such as fading you have also problem with this architectural and the probability for our customers not as lowest many people are thinking that's a problem here so bag up it's no no substitute for back up having a because for having direct more articles than in any case OK is this theorem now the next 1 is we are looking at your redundancy and many people are claiming that you're redundancies extremely costly it produces much goes
because everything is double is true when you and is especially true for the big cluster architecture some people up believing that you would just need to distribu what your be clustered into 2 days and and then you have to you redundancy this is not true because toward data redundancy is if you have to derive reading of quick gnostic means it takes several months to restore your data the 2 operations OK this is the scenario we are talking about important for instru insurance for example so um if you have a bigger company in and you want to survive this you have to be able to operate continue operation several for several months in your 2nd dataset and for this there's scenario you need at least 6 right because you have to double all of it because otherwise during you'll see your fault 5 months or even 1 year whatever it would take to restore the you will have the same figures scenarios as if you had 1 dataset not just losing half of of your total and in this case of course you also doubling its pure um but you can use
mouse for long distance or the for redundancy here but the new are we on all of them are talking about because sources then it is clear that even though the cost differences significant OK so if now let's combine it's
um it's and kind of unfair comparison here because uh the 1 side we are returning back to the 3 great because on 1 side note you redundancy and the other side we are using the redundancy from the last slide and comparing it this way is taught it's interestingly in there it is cheaper that's
that's a key point of this slide here so this means Jia redundancy you have um 1 colliders want stored store best case yeah and you say OK you have to double its and you also need this replication that work but compared to decide where you have all of and clients plus 3 times your for 3 because assuming that 1 as the state size and the same number of distance on uh and your storage network which is also replicated as a whole for the datasets and in addition you have forgotten this year we also replication that work volunteer redundancy forgot this so cost of however should be clear to you of us difference here and but there's a function of difference here we have used to your favorite scenarios and here you don't have to so you get even better functionality of a better insurance against life insurance that's the better and here you have more costs and the less secure system is a system of and this possible and this is I think it's a new idea I haven't seen it before but the corpus
correctly if you know a similar ideas over this idea how to even to make it even cheaper at your attendance what's the key idea here it's symmetric um Fallujah redundancy means that when your data center is operating 100 per cent CPU now the 1 it's almost a opposite OK so I'm wasting CPU resources their order and resources what just deploying source for the case of do so this means doubling the cost is something which could be tried to to reduce and the idea is
that the storage cannot be reduced the total storage needs times to look no chance around this but for CPU power the story can become more could become different if you have 3 datasets we look at as a set of tools and 81 and a tool means this is the primary site of the primary site is divided into half the A 1 is predicates here to anyone trying the main secondary role by default the aim is replicated here and now the same thing symmetric so the bees are replicated to adherence to heal and this year's form here is the primary site secondary sites are these 2 datasets significantly
understandable hopefully know what about CPU consumption storage is clear this a factor of 2 but uh during normal operations each data center has 100 per cent Q 1 for each path and in case 1 of them breaks down if they data center of 1 as both great and both of the the the qualitative and sorcery then you have to switch over a 1 2 here and there a tool to here that means
the CPU or consumption of status and increases only by 50 per cent it's not doubling and also with the same novel dataset what this means is a challenge for a system like Mars how to tackle this
this problem here how to have a flexible assignment of CPU to storage our 1st model was having a fixed association between storage and CPU power the 1st
attended the 2nd 1 is how to do it better this is the next flight CEO the basic idea of of my talk flexible mouse shouting and cluster on undermined what does it mean in this example here we have some type arises running on a single storage box the red systems and hardware logical Williams here and the the ends are you lock KTM moments the Alex seek entails or whatever OK running on the same box the number of course can be scared all up and down as as as you like and in case you the CPU power of this box here of this hypervisor instance in 1 of your data center is not enough in and I have another 1 which is for example in the secondary passive was all um there's enough more than enough CPU power your switching over your exporting this William here this must replicated volume to another 1 over the local replication that works preferrably but it's an exceptional case it's not the default the default is the uh the VM containers all the containers at the same have horizon work whenever possible so we have no storage where network traffic at all by default and only in exceptional cases if you have for example a tech tools from machine or whatever happens on you have this case of emergency breakdown of whatever the exception then you are exceptionally using eyes can see all the new field for most remote which should appear during this year's not get production ready so it's just replacement price-cutting him to be better performance according to my measurements yes I love
the the a test where I need 1 microphone please repeat your question yourself came in but it doesn't work I think the microphone Doesn't work you have to switch it off and yeah tried out what is the I want to use the phrase over that all possibilities if you have read thousands of machines you would rely on something like
hot Beatles similar you do of course if you have a low number of classes you can do it only automatically but for what and when we are performing the manual Minnesota uh because some automatic messes up tending to produce some unexpected results um the anyway have a network operating center which has to watch thousands of that connections or of several dozens of lions and so on and so on and the and at different times of types of incidents which can be automatically recover about and and so on the work anywhere in the 24 7 in place so at the moment we don't have an automatic there but in my last slide I'll come back to your question so for example for community purposes having use GPL softer all operating somewhere else so it's an interesting it's an interesting question your writing here for 1 and 1 originally do the men away at the moment we have our own custom enshrinement mentioned later so so hopefully there are no more questions to for the slides so it should be clear hopefully what is that and now what does a cluster Augusta
clear flexible mouse it's not clear and what's background migration that idea can be seen here you have this primary Siderúrgica vol. 3 year of course you haven't already got the old and richest favored the mode for example and you're just
trading off held directly come on this machine on the same where you already exporting my eyes calcium for you have enough storage space and this replication is doing forcing system another job also does full sink in back what why this is while your data is being modified by your application the property USR logical will turn this around after some hours or
if it's a very big huge devised and may take 1 day we have some up to 40 terabyte logical volumes in some cases at the moment and somebody boxes of of throughout the terabytes a total of what books or 48 spin or even some machines even at 60 spindles rate systems several races on 1 machine high-capacity ones and lower capacity for better performance and similar things so after a while you will have it up up similar same notion as the the R. B. gene you know this up-to-date should get ready already for the same as the master and well index that was not depicted here but you can imagine what will what will it do then not is switched the primary rules to here and the cancer connection not needed anymore and this is just the idea of the Decalogue migration create temporary additional replica additional food can even start if 1 replica come and there were created another replica at all but if you have to like radiates even the the new have a temporary 2nd replica and also what of course you are destroying your old replica of those some commands like mouse RBM joint resource and the source and similar commands have already appeared in diabetes so it's there's no way to because I in contact Philip rising from independent some of the the look and feel from the perspective should be very similar for both products so that's the basic idea and you are not using a big cluster you are migrating the data explicitly up on your request assistance that man you are telling the system where the the data should be put aside you can migrate it at any time to anywhere as you like so you have a big cluster at the MIT data
level we use all the resources you big cluster old well they're all residing but the action data pops data life was not a big cluster right the actual data is where possible or the local node network at all and only in those places where new that you are using that fork and that's the basic idea and well no my last floats the 2nd last 1 what's Mars it's under GPL was a 100 page manual on the top and you can download it
read it also some sysadmin instructions how to step-by-step instructions for the set of your for mouth clustered on similar to the operative you already know job idiots who will be become familiar with it the sub detail as some some some differences detail because of the acid quarters of modes of Mars the way you expected in the orbit something is done to support the you have a synchronous versions of mouse uh and um locate stable because it's in production since more than 2 years or 3 years now and uh the 1 I want your redundancy featureless public know badly public advertising
probably you noticed it we have redundancy and or and then there were advertising this is just the backbone of it as we have more than 2 thousand tourists uh and knowing the databases are even more so this number is too it's even more because the database is also updated server databases was also replicated by a mouse uh I think it should be more 2003 falsified total this long and even to low then the the campus achieve started in late start that it is even higher there wasn't a petabytes of data uh interesting for you is the number of I don't know about that posting means the have extremely many many many many very small files OK and they're all the farthest in which is really has a low a rating hours not set of as not by the office so on X for some problems as you know so we are using its surface and and to Mars has no more and filled the millions of operating hours wanted the because the big systems it's more for more than 2 years now OK I what yeah currently doing our job project is increasing density and saving costs and this means that the migration is the current work programmes working at the moment uh there's 1 and all branch this is quite stable branch which has been in production for on this as collected this number of operating hours and all for 2 weeks using the new will it's even in officialese of stage but already production for 2 years now for 2 weeks now and of course in in a few months I will labeling better and hopefully by a bold and before you know I will also live lead stable so it's collecting some experiences crossed maybe a little bit higher incidence rate at the 1st new versions new functionality but the experience is that the reliability of softer it's higher than of the rock and this has to be if you are creating an a space of the sufferers to be more reliable Java otherwise it's a small high availability I will be the case so this is just the requirement for the software well future plants is
what I am thinking of all and there are I would like to have some feedback from you what really understanding bottom-up here what's already done if you have been struggling with your produce a musical and migrated to Ross because of symmetric problems so you can imagine that if you do data-journalism view of several lines standard clients several I think we have all the weight between them or whatever it is but there's a lot of of all traffic this has nothing to do with your web hosting different applications with whatever and this means that you are running it sometimes it's so back is running and whatever is running so we have some people their and also pay
securing at your job at your education and you have a problem and in the another program and even a slight appalled at what happens if you are if jeopardy is overloaded was replaced by a mouse was the reason why I created mouse some years ago so the most project what we're currently doing is this 1 so we are creating a virtual LVM like virtual pool where you can buy rates so so global instance them by migrating and growth machine us at the moment of our own cluster minus receive 3 which is not ready for publication I would like to but have it published it's created by different teams the need uh but it's uh I think it's not possible because it relies on the internal infrastructure of databases instances even in our until infrastructure and this is not for for for public use for publication doesn't hurt that's the
problem here so what I'm thinking about is having a literal blogging I already have a better version of it but not yet fully tested so if you have a uh there's only for a plugin for the orbit the most 1 it's not so it's very similar so totally different from from me but somebody has to test it in my time is limited tho I would
like to have some feedback from the community responsible all the you newborn this trust your question before what about automatic dopants not only sonorous but also but also load-balancing and divide I don't now there's a question of this is to be done separate implementations hopefully under GPL and published at the top would and a professor at or going into existing projects like the throttle OpenStack over Cuba letters for example why not if it is easily possible so all I would like to call itself feedback from you what would you prefer the well not all of you probably have this application area of thousands of so that that's that that if you Jeff of this listen to just make make you check marks somewhere with what
would you prefer the Philippine user preferences what is what who you can also talk to me all flights have been the mayor so this is just a question I'm I'm asking and I probably I should keep going more experiences that I have a lot of experience is the old system but not the new system are going on between battle so be big because I'm him very old system which is operation for decades snow and has some history of course some what whatever's happening there so this means that it's a complete different of our watching in area we're a traditional system is very
stable if the processes side to processes and so on and fraud is a no easy story there are 2 to get something you could call routers stream as there are probably few things to
cheer and signal no big problem you can switch so we are starting at pesticide you're putting the call you call the pesticide than just hand over but even if you have through 2000 of machines and you have a rate of whatever it's or less than 1 let's say 1 problem but even 1 per cent and you have to have to look after it because if you have several sources of machine a fully automatic wouldn't in has some as the risks and that's the problem here so I here reliability as really to be really extremely high there uh at this area so this is my last slide any questions and this is the case for you you to models all of this and the question was what I said usage from jeopardy too much because of that for problems with yes over and formed the parts of the world and this is the only things that you can see on all we want it and the I know it's really fondue redundancy uh I can confirm that Gelbard year of survival of discourse over cables this constructive that case guarantee is simply not constructed for long-distance replication and we have some kind of long distance you have high latency and have no constant latencies a case of traffic jams and so that similar to similar events there is simply no constructed for this case and that was the reason why there is a need for for what's said simply so and this is also the reason why I'm trying to be as and to something we call but it with the 2 D Avatar as much as possible because ministers options are used it has some reputation in the community nobody has a reputation power relatively new product open source product so as to get some repetition of course so it's best for from me I think it was something similar that also in contact with in this in this case some arising here in this the right way to go on its sister I regarding system even recommending the RB gene in the case where is that of speed and if you of course overcame stored use the new security but if you're planning a high long distances if you have in more than 1 water in between or 1 1 what switch between and if you want total throughput of all of your switch ports to go along its wall and you will go into beta connections more than or 1 over of magnitude lower than the total that you should also consider mouse looking into is having some books approved concept looking into it's getting some experience and then slowly migrating as they have thought if you have some operational problems never doctrine utterances you know and it's also clear effect but in case you have an alternative and they our maturation against those of war stories the 1st the point was that uh we often I 2 that means you have to be able to rotate at any time and this is possible because it's a drop in replacement in some sense and not completely because there's a separate space for the spots master rectory in but the the metadata is on separate volumes if you have it's separate that the separate and very logical Williams full metadata it's no problem to use mosses problem replacement for the of and migrating sources of instances for problem I wrote some migration script and it was 2 nice 1 for the European Cluster because for the all the words US just rodents in new migrated more than 90 per cent 1 nite and there was some some remains of what they had problems which was 1 of my class later so we have to automated if you have use of force sheets of course and I have some more if you like if you not no more questions about the clusters this is interesting for your uh very old experience for me you know 1 product before and 1 called my that such uh started several years ago and so it's partly written in Java and Bali written in PHP and a variety of patented system and started on the green belt this big cluster architecture but it's a low number of sheets to where it's 2 stores tourists and about I think 6 guys sources it have and they are starting to study the benefits so it's a bit cluster architecture by by the late by nature but of course it's a really big and as a small and after it grow to apologize I think you don't remember exactly about 20 thousand customers at the behavioral solidarity to that's the problem is always the same who stopped in that start and you see this is to be said about some about this oversight and suddenly piece there's a peak that it's going up and after a while it's going dull so it's had some some some hanging some some some some some some hey there and experiences of men so superbly are you you all the problems of the you experience what's the problem is TCP said buffalo and it's not the real time is the no refined protocol so what did they do all the sysadmins they're all started to replace and of as this also efforts to all the nodes in the woodlot always interest was spent us obviously because it scale and the scalability for scalability
the limit is an offense was about to without and 2 thousand 20 thousand customers and uh the course if true was working until about was thought useful the 5 customers then the same problems that it's a big 1st before what did they do all that looked around they was good cheat OK and as you probably know from from talks about last deficit spending and if it's like could from yes this is true and Purvis road what do you think about 50 150 thousand customers on the boxes and then there's a problem as before so it was a difficult start a project starting now nowadays we have half a million 10 times more is in the meantime so what I do and is at a good both into the project I talked about education architect not the systematic but application architecture and I explained to him the very 1st slide about because this over n squared that work I was expecting to you and he made a was all we yes is is old in square talk by concept but what do we need we need sharding architectural what did he do all equal vote of the whole architecture to starting in this case filing of small clusters because there's already some customer festering and what do you think it responded to date scale even if you would need to scale up by another order of magnitude standards for customer we will be lucky if it would happen but he it's no problem the scale up because trialing always by varied by its definition so if training does not scale than the environment does not scale solving skills like the Internet extracting doesn't have been ignorant of book doesn't work anymore it's the same problem in innocents so this is what I want to tell you only because 1st half the merits they have even their application areas that use cases for its depending on the type of work but if you don't need them don't use in the support any more questions I think your for your attention the mn and