Bestand wählen
Merken

Simple ETL in python 3.5+ with Bonobo

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and the welcome everyone thank you for attending and thank you for the introduction of a I hope I which made expect patients and so yes and hand presence of portable which is in its year project in by from 3 . 5 and an expressions and that is about 6 months or it's of I want but it's of 4 much more time but that this this version is a brand new rewrites from the beginning of the year so we create game will not only the have a French name for the French accent and I wrote him a lot of different companies in different contexts that I've been along web development and software engineering for around us the US and I assume a lot of features so so commodity yes in different contexts and I didn't find what they really wanted so it's the main reason why Bonabeau existed in my I put so much energy Rodrigo so
I'm working correctly as an advice on this sort of accelerators of being the very best so we are a team of for multiple no linear away with something taken couplets and Exeter riddles of the where we basically give the business and technical advice to to from this so
back to the beginning and
my plan for this talk is about to take maybe something like 10 minutes to going to work they exist in values language not only by some because most of the ETSI use or well not by some of the what exists of so by why it decided that it was not meeting my money's and then I will try to use the most of the time of the of the talk to show you that in real cases in the real use age was not really really well but example use and so so you can understand a bit more so that view into the if you think you have to know tool to start using it and there's really not much a of and then really a conclusion 1 I was some pointers of where you can go from that some it hits the
as many of you probably already know it means acceptance from Rome and occult according to Wikipedia's judge was already popular in 1970 so they're thinking of something new so and it's basically a well where you have are more than 1 the Postal talking about 1 that out so if you have some master slave departure of some stocks system connected to some of and ecommerce website for example are you ready for everyday use some kind of feature to connect on that to a folders long no of that's of the most simplest scheme I could come up with about 2 years is that you have a stack of that here it's Borland because you have of not the of transformation you want to a prior and each line of that up and then exploit here can transform fooling some into something yes when he finish transforming it going to transform and wavelet transform is taking care of the resident of extracting Fu like and I start tending bar and that's it so you you can as its complexity independent you can and and that each transformation aligned in parallel to base but in the end the reward it's usually looks like that of there is that is there is maze that science that is looking maybe not Mason anything on but the generic generic and that's it is exactly the same so just it's not so not doesn't as in the hours it is here the there is a lot of tools saying I've seen in the markets mostly driven as on that looks like that it's usually energy 1st and for or you can out of it you know you stand up and say you while poor can use color but mostly it's a graphical interface that's a configuration of him from and they dialogs is very handy but I knew it when you program you feel are very limited very fast but there is a true of other never used but it looks exactly the same just a bit different this I spend I will open up until the integration of local together and I use this 1 a lot but also simple set so we have In this what we have muscling way 1st a magical but also and mostly Java-based In the pagan world is of shared libraries and not about exhaustive here and a but I think it's no marked and maintained on beta is much more freedom to face there isn't much alone but more some people in this country constantly about of 2 and I think there is a p Python furniture much but along that none of this according to my analysis work well on doing the same thing as the day that letters which is simply long connecting independent boxes together using the that of the and so expensive to create bootable I think that they are a I started to create and library which way which was taken to that 7 of but it was just that he returns of the best thing to do according to me was to start again and drove complexity that by them to support explain more little the other is also related to assume that you must know I guess jubbly desk and esters you may have of some of them they band as it's amazing about some it yet is not freedom in focus in for for example necessary grew to transform the data set into another that assets and I'm using the maybe every day but so uh when I when I want to do more engineering and that like taking 1 item at a time until something means that a State B State C State D it's not really the topic at all of the result of the scales of the tools to transform that I know recreate there is may know I have to teach yields up your which all cloud-based flows these tools to use moral dimensions of obviously this won't run on your laptop and there is a huge as tourists of which black spot by just use a few here but either you need a big infrastructure to starts doing things what is that this sentence perception is you're using a cloud-based thing and you have the same parameter lots of 0 do you walk without the cloud and would not you only in the looking yourself into 1 vandal as a city and the description of the location of not to there is no be detained is from our own that's the at I want and then you bit a story about so I came to discover each yes I where I was funding of co-funding a company so when we started we had a few different partners we're all doing a marketplace about retain 2 it was atrocities and mostly for woman and and what we did to to all different partners to integrates them stocks and get there are so many girls and be chosen etc. on all marketplace which was receive achievement so the 1st of no went very well we just put it in get the bundle and the afterlife in no way removed the new things around so I was looking so we're really happy but just we
we gotta should use sound when you get this idea how to integrate user prognosis to like compete as the code rights to the 2nd and 3rd partner because it's about the same thing but just to be different of course this is and not a good and engineering practice but of you when you used to suppress things and to instantiating with different parameters and now you have on the right you just lost because you you can't really do that so you don't know out not just and of course
is attained come when you can need to fix about
again you just go crazy and maybe it's not a
it's on new features because you didn't support could also now of different model of course and does that everything so that really what I needed was something chief I could use some of my laptops volume service to and using code as configuration and preferrably this 1 is not for any good reason except the fact that so profile but in Python done anything as this must is something that you got configuration to the beach here just like and I will bring it yet and yet this model but it's a formal to attitudes of in Python using gone on inventory some there was some kind of great may come to vision-based thing but 1st it's codes so you can write classes you can suppress things and other things like you just putting wearable maybe the engineering but I go very fast and that why it's very different our all tours existing and by that I will know that they know I maybe there's to say on those very happy if you than me about that and so I can use to me that's this should be the monkey but it was not a monkeys and apes and Frenchman rich apparently don't make the difference image and that was 2 different worlds
and so it's it's
on yeah I'm going
once again the the I would like to show you 1st 0 projects then I posed to show you what the different concepts of that I used without bidding before and then I really go back to the demo and exemplars to apply the conservation plan to different amounts so the basics
is beeping sound and you have a generic tool using cookie cutter that is just 1 in it's something and and you can run something with a bit of all right so address which could terminal
not like this obviously again so I already under in it but because I and I'm sure you don't believe me I am always you believe me but I was just that you can bother going at full for example and it would just create a for directory with
man the Wi-Fi and probably effective mandatory widely such men to
be right of as a few difference but mostly under things so this
person of soaring food because they have all the fact that we use and we used after that only and just so that yeah I can but I will end the Maine that you i for example what does a different transformation that's gonna with the degenerate all along nothing really fancy just generate numbers and take some leader of numbers and I I can also run of directory because the main fight is considered the main and so I get is the winning don't will do exactly the same I think that's that's really interesting but that's really the basics and
so that was that's so now and I
want to show you what was actually run so I don't know I I want show you the 1 2 4 2 2 0 0 2 4 2 on but so I wrote this post simpler ones here which is basically diffusion of 3 different function and 1 using your world and Python and 2017 1 just a plane that have to do to a string inputs and what just printing the thing and 1 who once I define about I can create a graph in sense of heritage in graph along which is a different thing we can do but there is also a game method to a although chains forking for some point in the graph which virtually in in an example of some and yeah I just a friend a graph and because there is a graph in some Shiite can use but a bull run on this fine and it will just add some proteins for the display on ground so here I should that the 1st few maybe it's the big
animal I have a further purified which is not at all what they
wanted to show some bucket any to example wand and I replace the coding yes
it's a visit decoding Maine the PY so it's as if you were shown and I
can but of on main the U I and we see the
outputs of the road which prints the thing so you will Python 17 which state and I show I see some statistics is very fast and so it's already great but you would see the saddest move away and widening of longer transformation OK and so
on that was this 1 yes a want a bit more of completes the
i i a just made Europe that takes to 5 and its edges expected that under you a Python society and which is a company again you buy them a lot of the conference on your plate on its like 2 3 dances standard says it was the the world maybe few is sometimes you don't have the information about the number of atoms the attendees sometimes retardates so if we have the data time but not reformatted this in some way around so I think
it is the time to the care with extracts of models on 2 paragraphs about each conference on Sunday it's using it so on to the next stage
data of this squares of buttons and then transform that using the Fury gate to finding the location the number of is if it's here to take up and create a dictionary from that's and then and I had a major elite develop a function all the to close the chance the formatting of funding to the good but none of the it's and so I created rather similar and I'm using a built-in which is pretty printer with submitted on printer to print and an
effect and that's yes I need to run training so I receive
of the name of a new dates countries that is only formidable for example it's legitimate for your Python 2017 and that doesn't so there is a few channels I can make to this distance emission to make it a bit more useful but just before I wanted IXA explain what's happening under the hood at end many screen is better
so what's happening the we created reference sense and in graph is really a list of edges and nodes nothing fancy it's not it can be represented graphically that is but it's just to list in fact and to prove that they just removed of code that yes is in use but not use food and yes the graphs on diffusion is that on on 1st color you have a shortcut to put at chain that at the 1st thing you pass to the concept of the graph but then you can at 10 anytime you want and you can specify a different inputs because you don't want every 1st not to be at the beginning of the graph but maybe for an existing shame the then 1 wants a and 1 so you defend the graph you if there really choosing the but what it's run method of all you can run it using this year like like we did before and what happens is that the bases in roughly defined before it there is an executive strategy that at a few things here it that so about context context for each nominal Freddy of it creates the for cues that offered SIFT use a Python it in that the beaches but some understand and agree on that I used to before input and output between the different amounts and in fact that the context so here is only used to keep the transformation of context lessons take less on because if we need to use to keep for example statistics or maybe a sensation of of something we need during the time of the we don't want to modify the objective you provided to to the graph so it looks like that's due to do about it and the right context creates a context which now just what I said before on the strategies relating on on the pulley executed toll of Python and control and that future I think a and we just created which is just something that we run every time you get something in the in the input to but we don't northern pushed of to outputs to the next and and then he does nothing and when it's finished it shouldn't that's implementation details you don't definitely don't need to know that to use burnable but it's a assaults to that it's not that complex what happens under the what you can use as transformations and burnable l is values thing so that you can use functions that we did before so must must if you have 1 for each line of input 1 line of outputs and you can use generators if you have for each line of inputs 0 1 or more of them but the plants are for example is very useful for us to implement joins the campus and products so even to make something that is used or not so you can use interior tells of 3 could about better tended to say OK I guess I can have transformation that have no input the it's weights could expect here that it it is no input and use a bunch of and of course you can use everything that is governed by some so I'm just trying to it if it's got evidence yet it's probably a decade and and so that's good to the do the handmade way to do that I can do the honors come Oscar contender on to do all that there is a and time on your way to do that I have but I wouldn't configured configurable class that's arose to use a few descriptor to specify what kind of options and dependency would Avenue transformation of course for simple secession you want you that if you yes if you need to configure the transformation is probably easier to to use that so are here so we defined an option code table names that we will use to Korea database as a different but you can override it the shifts on it we define a assemblies we database which the 1st database of for it's a symbolic name that we want to something we want to see data and is set up so no whatever you want to instantiate this query that the base class you can yeah or not alright the values and that can be very efficient but that's a detail on much more interesting services and services like that that that adversaries we present here is basic see basically saying OK might have assertion we relied on a database but I don't want you to tell you what implementation I would use and I just say it's probably cost that there of difference a lot and then bring me something called other bits of D for them and try to use it like I felt it would work I and of entire you can provide in the G I would get so this is a function of the simple dictionary that provide implementations allowing for example to provide a different dictionary for tests and so you will be able to provide in my database of test implementation or mock implementation so instead of this 1 to desert transformation without testing the external dependencies and posters prayer in it's too it's too and there is ban as burnable so that a lot for now and it's been of a some of theories rose to reach 5 spread fast use that's just after all nothing fancy here there is a futures to walk the lexicon say Carrollton to briefings we use pretty-printer before and there is a few extensions and play games
proteins and extensions and other presentation on all this predictors really good quality because we can see the thing because of the case that on the concentrating with social real-time the input and outputs of features emission and apparently not able to do all of in the tree and the up tree in in the concert but to the graph lecture GUI to disrupt would do that if some of you know what to do that would be really interesting because it's it it with the same feature but it would be nice and of course there I'm using Python looking to to candidates there's attribute
up again I yeah I should have attempted to show you that after of and
everything that horizon bigger libraries all the dependencies and of bundled eventually is extensions now there is for example the Socratic in extension and training and starting to use in which arose to walk with equal that is there is a book I extension of that adds RNC command to the but bullshit I and that's just do the exact same tries to the existing thing as little run but within a container on this there is a different repository curved winnable kid that's will come in different folks at the same time of of the project's probably more something for me or for anything that you want to contribute to that so it's useful to and
yes we have 10 more examples so I
will show you a lot of things so 1st what the so I start again from the and demo I I did before and will try to show you how to use the service instead of doing the opening a file of all 2 right there is a to C is the union all 2 way to just so I want
you to do you have with this 1
OK so we were reading the Europe identity DC transformative
so and what I want is instead of opening decided by using something that I will be able to switch from repent faces stand to with 3 etc. and there is a very good labor really we'll relaying on 1 model which is confined 5 system to that does exactly on we so along this so we just depending on and it would be instance so I can go on to something that is that requires of of the fs for example and we imports regressors you know sort of stuff but OK and so they could be prorated as opponents of invaded the it'll just of and I can use it's like this using a physical but instead of discipline defined and so I did and 1st so this may be already is defined but on page 21 but I defined by different
parts of but in fact all these changes is not useful so we just say OK faces but no open which by default we use the current directory as to which of the inference system object so and got a
and 1st slip uation to do exact same
thing needed did before but we're not hurting anymore directly opening file next step for
us right into a C has been so
that's a good occasion for me to show you Oh we photograph and that make something just announced so I would use the matching methods some too bad at then evolved that's is right and to a find name which we use also the fastest and so this which is what the sensors and service so we write to you a pay but see is the and to explain that I don't want it to be a new chain that just and taken and teams in position at the at the beginning of the transformation and to say OK the inputs of this chain they do not before the 1st node of the shame is on 2 quotes and tablets of 0 to cross the in fact the to produce their own transformations of transform function was returning on a 1 argument containing a data dictionary and it just transform this 1st like humans which is a dictionary into Q all arguments and by DeForest it and Euratom that's by default or writers all thinking gee what arguments as inputs so it's also weights here and it was middle refrigerant to so that should launch
and it's you know that they would really light the a skill tree
but there is a city situated that the 1 at the same level of pretty predicate didn't it didn't take the output of pretty print out because
the 2 middle didn't have any output it took to output of art do would across and decide which which is here can pretty picture and to see is you at the center but because I don't know if I removed defined before the the presentation I will remove its and run it again solution and
plane I am afraid of and
that's it should contain order that arbitral matador
SUSY we've maybe due to a number of attendees like for last year or maybe not if we don't know from the your present status OK Ireland
now we don't and so the next best had was to i to j is on which would be really easy because the syntax is exactly the same Our if I wanted to change of the deformity and there is evidence options that are not the same for SUSY hinges on obviously but of the 4 just want to rectify it's the usage of the different countries make people
OK on so yeah it's exactly the same that's your Python that Jason the so
I don't have any you operate in the design phase here on the IRA and
esteem and now I should of 5
containing the same thing but from to distance what is so that's very busy
and so I tried to find
below example to show I looked steps yesterday I think in a forum Nuprin that yeah we need as open that in fact so I I don't speak very good italian so I understand absolutely nothing about so what it was about but say understand isn't so I can put extracting and just it's and I think it could teach accounts and things which example 3 and is not really a peaceful area future carrots the yes can don't look can make commit messages on so I have a really needs few the only
we just it's a listening also that was what we had before but here we require the sums of because http on before you
can do so this is the purified and we see that so I just defined at http these requests are common I could have used anything as does that mean that people are we rely on anything that's what's like requests
on so I used a speech to the tuition be gets on your and I have some whatever makes your because each a batch of 100 the results will you if there is an phase OK next year is that's because the web service is not very good any substitute slash not by session of the disease on because it returns which and iterate until there is no next year they're nice to up to core organ just right just and fair about that so it it could be a bit
no longer because I need to
abuse accounts do its power parallel because they need to of the result of the 1st request to know the next year before I can do anything they I should have run its before I started to see that on but yeah 1 1 where they wondered the the extraction from it should be of gender of 0 2 crosses maybe instance Anderson retirees may means the is some to it's it's a big difference there to used as and narrator them to justice and encodes the worst thing because I would just anchored each line independently now to avoid to having to to have to before anything so really what it needs is only 1 another time and it will of lights review should be a few bytes every time and in effect so and maybe I I guess it's about the elections in Rimini there is but we don't that we
decided consulates let's we is segregated or pages from this was just recipe I not there's just no API but there is something I don't mercenaries users may be needed that some that created the and and a set of no and there is the on different districts so I guess that's related to where you have to go vote when there is an addiction but that's wages focus that's many bits for
something that we order understand and which is engaged and I wish all to expect under kill your Python attendees in in a book of course of you can say it's derivative also ways but this is just 1 example so arrived at you be done a book I guess everybody knows about Jupiter and work here on and the yeah so here I have an attendees of Jerusalem which should not exist on and they have
this so I read that each herself and clear approach just to be sure most the people output in fact so I'm using city and here the 1 which is basically something to contract because if you don't know about it and it's use very library and there were few reptiles but slow really doesn't contain much console so so I say OK there is a was coming patient your patent websites and I need to implement a Brazil so this which is using but I was sitting in that create goes up you could just creates syncytium positivity and you open the file system as in focus so and a bit short on time so we just execute the graph from from a ticket the graph buildings that I have a graph great and then there exist used when a trend and so there should be if efforts of opening with was
commanded prescribed down every time due to infinite scholars in the dormitory
gets as long as it's get elements
and when it can get elements anymore you try
to break the bond stop button
once just to see if it's not some just could've that doesn't work and also and if
we we didn't get any new data
that you could exodus delicious accounts well I
think there is something like 350 up to reach this announced on the on the website of course a lot of of
making curvaton to right because there's people
like me which quarter things but I want to anything which does it's just an example for him a home and yes a polarity
here it's it was bouncing probably recast is the last 1 maybe 20 bounced and yet would approve
sign here this chance to tremendous have to say that this transformation is finished user 1 is instance so we should average is only for attendees yeah and and
yeah we know if it's a speaker we know it's like letting you put 1 here that is not line here there's a line as just Our so there's very few
minutes left so risky to steer an example and so on go to the
and so yes the Buddha was a very
young group library in 6 months and is not adults affinity I'm trying not to work as other candidates and that superhuman so all its not enough of course on but see I'm I'm really by this because it's something amusing it at every time for everything in fact and I had really like to get to 1 that 0 is the end of the year know all the and wonders you will mostly means for me a stable API you can relay on that is fully documented and free tested this i it's already tested and be documented but I need much more and patented 5 is a the person get lot I started this year so I don't want by them to anymore so I'm trying to push the most I can to only use by field 5 and there is some really handy simplex to walk with that of the I don't know it's going to distill stole within the dictionary to expand a dictionary instead of a dating thing in place using deductive Dayton dictionaries you some and we see that as the we still have to go into rock of course but so maybe we overcome this limitation of running on on the 1 call using different strategies for now is this training strategy by different so we other do maybe not for every show I O-bound but if you see the event you can be a problem Our and but yet priority process for strategies be a desk distributors strategy if you like this we can and try to limited it wanted the duties on various bringing bringing us as and so 1 doesn't have 0 the of course 100 % open sources at sense so I want a very light library but of course it should do the basic seems nice is you sit around most faithful must sum tools should be included but of the sky containing dependencies and complex things should easily be implemented by the user to extensions but it's more scary to God is 1 and 2 standard is to the priority of it's not and once again that the get on a statistic now than entities and if you want to the blockchain read that you've probably in the the in the unit and good around and it's visited the manufacturing for that it's like and I have a production chain of why use or a little packets of that up and 1 at a time and adding something taking something modifying something ask you on that but the Internet is completely crazy like I can use his small console about me knowing about actually ontology of the of the taxonomy of file of monkeys and apes and primates so I I really like the last 1 that suspect anatomy at as dictating it as the the the non primates taking feature and yeah this 1 saved my not really but since it was an infant and react to it to become the person for amounts of course there is a lot more to do you can read more on the websites of the Buddha without forcing that all you can read more into documentation that you will find a link in the web sites there is a section and then discuss come from it's open 0 and there is a deterrent you will also find a little on the on the website and yet 1 more thing because of Finnish and I we try to organize a strange whether you want to go on not it's not a problem but you should reconsider to go to whatever sprint it's really amazing at you to to just got on a project where guys last year I by test which was really great way too long so government of course got 1 of all but if you don't come to all this princess really really really great thing and just before
we take a few minutes for questions if I just before that it would be really great if you could give me of really fast you guys feedback as as you think it's said I I really need rules about on this year I have it formal and yeah that really really really great for me thank you very much and if we still have a bit of time the less try to answer questions if you have something thank you very much so the couple that this thing and I think there are many question at least I have many questions but I would give you the what you can do that is following to to to ask question in the back of the room because it's good for the have the organizer hello and thank you very much for your great talk and working on this project and there's 1 question that the complex and when I work with is element something can go wrong especially with the areas of the weapons of how would a in this framework deal with the the OK so and which is about on so far now what's most happening in doing firm work is that's on each on each line excluding the world 1 of them is going to the function of the node 1 time so there's that was the case there is a world that's Skantze uh on that Ankara and UN workroom variable it's not really easy to Central the world so there is an oracle about their roles that we just of the graph and you do it and raise the exception so I can't just lend a graph so you'd everybody should fix that but instead there's also recover were River Errol's which are the walls of happens on the on 1 line of that all she of that are so of there is a difference in or on that you could override that we just use the consort to show the unrecoverable errors and just keep to the next line of that but if you want to handle it's differently you could override is handily or thing and just so do whatever you want probably there would be things like sent really is something that is but for now it's really not a priority and it should be just a few lines to implement it no questions OK then maybe I question you know that Kafka streams it some reminds me just that the Kafka streams of men for being distributed on the cluster running big data on the cover no Q but you also have accused distributing task to threats of things so Morrison looks like the same architecture yeah and so I am not a little familiar with if care but yeah we are talking of of a few instances of Q Q of Python cues within 1 process I guess that's kafka queues on but maybe I'm wrong so because I don't know I guess it's some kind of message boasts that we'll be able to pass messages from 1 cell to another of because it's kind of an architecture thoughtful being that the 1st year what they really want to solve is the poor and that's a identities from that that right now let's and study to me and and got something so it's in the in persist probably to be similar because the idea of 1st interests of messages so at 1 point you need a few but of the it's it's thoughtful different so scared of architecture 1st of course more if a tree can do the same with best distributed for example of that with the same kind of cues but not the same exact type of objects that would be able to pass messages from 1 civil to another but then you have to think of a funny programs like all the you optimize the topology of the graph of to make group the nearby transformation on only 1 server and maybe because of the same so do you balance the number of transmission of each cell it's not not easy and uh 0 I'm pretty sure that a few of that of this scale there is that you can definitely afford to install and that infrastructure and user user Kafka all had little balls of by you buy stock for example all things like this became the questions question the 1 OK so that everyone is hungry the soul and big things yeah so
Energiedichte
Arithmetischer Ausdruck
Subtraktion
Gruppe <Mathematik>
Termersetzungssystem
Spieltheorie
Versionsverwaltung
Web-Designer
Projektive Ebene
Kontextbezogenes System
Software Engineering
Hacker
Bit
Sichtenkonzept
Formale Sprache
Automatische Handlungsplanung
Maßerweiterung
Quick-Sort
Computeranimation
Last
Rückkopplung
Reelle Zahl
Zeiger <Informatik>
Integraloperator
Demo <Programm>
Resultante
Web Site
Bit
Quader
Desintegration <Mathematik>
Hausdorff-Dimension
Transformation <Mathematik>
Komplex <Algebra>
Marketinginformationssystem
Computeranimation
Deskriptive Statistik
Last
Task
Code
Notebook-Computer
Fokalpunkt
Programmbibliothek
Integraloperator
Optimierung
Parallele Schnittstelle
Konfigurationsraum
Gerade
Analysis
Streuungsdiagramm
Zentrische Streckung
Parametersystem
Transformation <Mathematik>
Betafunktion
Applet
Web Site
Nummerung
Physikalisches System
Datenfluss
Fokalpunkt
Teilmenge
Energiedichte
Menge
Benutzerführung
Kantenfärbung
URL
Mini-Disc
p-Block
Benutzerführung
Faserbündel
Streuungsdiagramm
Aggregatzustand
Intel
Parametersystem
Software
Rechter Winkel
Drei
Code
Computeranimation
Integral
Retrievalsprache
Server
Subtraktion
Desintegration <Mathematik>
Vererbungshierarchie
Klasse <Mathematik>
Virtuelle Maschine
Euler-Winkel
Code
Computeranimation
Dienst <Informatik>
Informationsmodellierung
Code
Notebook-Computer
Codierung
Notebook-Computer
Spezifisches Volumen
Mini-Disc
Konfigurationsraum
Benutzerführung
Bildgebendes Verfahren
Implementierung
Bit
Demo <Programm>
Adressraum
Cookie <Internet>
Radikal <Mathematik>
Automatische Handlungsplanung
Projektive Ebene
Energieerhaltung
Computeranimation
Subtraktion
Schreib-Lese-Kopf
Total <Mathematik>
Dienst <Informatik>
Template
Computeranimation
Metropolitan area network
Zoom
Mathematisierung
Zahlenbereich
Transformation <Mathematik>
Dateiformat
Verzeichnisdienst
Template
Computeranimation
Ebene
Lineares Funktional
Schreiben <Datenverarbeitung>
Punkt
Graph
Datensichtgerät
Anwendungsspezifischer Prozessor
Ein-Ausgabe
Gerade
Computeranimation
Eins
Graph
Last
Verkettung <Informatik>
Spieltheorie
Integraloperator
Zeichenkette
Ortsoperator
Statistik
Schreib-Lese-Kopf
Hochdruck
Template
Gerade
Computeranimation
Graph
Last
Gruppenkeim
Codierung
Integraloperator
Aggregatzustand
Funktion <Mathematik>
Schreiben <Datenverarbeitung>
Bit
Graph
Zahlenbereich
Spieltheorie
Ordinalzahl
Benutzeroberfläche
Dateiformat
Computeranimation
Graph
Last
Information
Demo <Programm>
Lineares Funktional
Hochdruck
Zahlenbereich
Gerade
Computeranimation
Graph
Informationsmodellierung
Verknüpfungsglied
Quadratzahl
Gruppenkeim
Dateiformat
URL
Integraloperator
Innerer Punkt
Retrievalsprache
Bit
Konfiguration <Informatik>
Computeranimation
Softwaretest
Stützpunkt <Mathematik>
Gerade
Kette <Mathematik>
Funktion <Mathematik>
Verschiebungsoperator
Softwaretest
Schnelltaste
Lineares Funktional
Statistik
Assembler
Plug in
Datenhaltung
Abfrage
Ein-Ausgabe
Biprodukt
Kontextbezogenes System
Knotenmenge
Konfiguration <Informatik>
Generator <Informatik>
Dienst <Informatik>
Funktion <Mathematik>
Strategisches Spiel
Programmbibliothek
Graphentheorie
Versionsverwaltung
Tabelle <Informatik>
Subtraktion
Wellenpaket
Gewicht <Mathematik>
Klasse <Mathematik>
Implementierung
Dienst <Informatik>
Transformation <Mathematik>
Physikalische Theorie
Code
Datenhaltung
Graph
Knotenmenge
Spieltheorie
Vererbungshierarchie
Generator <Informatik>
Abstand
Maßerweiterung
Konfigurationsraum
Implementierung
Demo <Programm>
Touchscreen
Soundverarbeitung
Fehlermeldung
Graph
Transformation <Mathematik>
Default
Konfigurationsraum
Mathematisierung
Mailing-Liste
Nabel <Mathematik>
Data Dictionary
Objekt <Kategorie>
Auswahlaxiom
Thread
Parametersystem
Gamecontroller
Kantenfärbung
Term
Graph
Spielkonsole
Anwendungsspezifischer Prozessor
Stichprobe
Kombinatorische Gruppentheorie
Ein-Ausgabe
Information
Menge
Computeranimation
Netzwerktopologie
Benutzerführung
Maßerweiterung
URL
Software Development Kit
Wellenpaket
Dokumentenserver
Stichprobe
Programmbibliothek
Projektive Ebene
Maßerweiterung
Horizontale
Maßerweiterung
Computeranimation
Dateiverwaltung
Demo <Programm>
Transformation <Mathematik>
Dienst <Informatik>
Elektronische Publikation
Visuelles System
Gerade
Computeranimation
Graph
Dienst <Informatik>
Gruppenkeim
Rechter Winkel
Nichtunterscheidbarkeit
Vorlesung/Konferenz
Integraloperator
Demo <Programm>
Inferenzmaschine
Konfigurationsraum
Mathematisierung
Dienst <Informatik>
Physikalisches System
Gerade
Quick-Sort
Computeranimation
Homepage
Objekt <Kategorie>
Informationsmodellierung
Arbeit <Physik>
Endogene Variable
Mereologie
Verbandstheorie
Integraloperator
Verzeichnisdienst
Default
Instantiierung
Gewicht <Mathematik>
Ortsoperator
Transformation <Mathematik>
Dienst <Informatik>
Computeranimation
Graph
Knotenmenge
Vorlesung/Konferenz
Kälteerzeugung
Integraloperator
Default
Demo <Programm>
Dateiverwaltung
Parametersystem
Lineares Funktional
Matching
Ein-Ausgabe
Elektronische Publikation
Data Dictionary
Dienst <Informatik>
Verkettung <Informatik>
Gruppenkeim
Tablet PC
Term
Innerer Punkt
Inklusion <Mathematik>
Ebene
Netzwerktopologie
Total <Mathematik>
Hochdruck
Dienst <Informatik>
Kombinatorische Gruppentheorie
Computeranimation
Übergang
Funktion <Mathematik>
Graph
Graph
Total <Mathematik>
Supersymmetrie
Zahlenbereich
Vorlesung/Konferenz
Integraloperator
Kombinatorische Gruppentheorie
Gerade
Computeranimation
Konfiguration <Informatik>
Ortsoperator
Dateiverwaltung
Fehlermeldung
Elektronische Publikation
Remote Access
Dienst <Informatik>
Benutzeroberfläche
Computeranimation
Graph
Flächeninhalt
Webforum
Total <Mathematik>
Statistische Analyse
Integraloperator
Stab
Phasenumwandlung
Message-Passing
Demo <Programm>
Resultante
Bit
Gewichtete Summe
Selbst organisierendes System
Konfigurationsraum
Sprachsynthese
Dienst <Informatik>
Gerade
Computeranimation
Web Services
Speicherabzug
Substitution
Stapelverarbeitung
Phasenumwandlung
Inklusion <Mathematik>
Resultante
Soundverarbeitung
Subtraktion
Geschlecht <Mathematik>
Total <Mathematik>
Parallele Schnittstelle
Gerade
Computeranimation
Leistung <Physik>
Instantiierung
Abstimmung <Frequenz>
Bit
Menge
Vorlesung/Konferenz
Dienst <Informatik>
Fokalpunkt
Computeranimation
Homepage
Caching
Chipkarte
Bit
Web Site
Ortsoperator
Graph
Gebäude <Mathematik>
Parser
Indexberechnung
Gebäude <Mathematik>
Dienst <Informatik>
Fokalpunkt
Computeranimation
Digitale Photographie
Selbst organisierendes System
Twitter <Softwareplattform>
ATM
Programmbibliothek
Dateiverwaltung
Spielkonsole
Funktion <Mathematik>
Offene Menge
Euler-Winkel
Parser
Datenmanagement
Computer
Element <Mathematik>
Benutzeroberfläche
Analysis
Computeranimation
Maßstab
Gruppentheorie
Code
Softwarewartung
Total <Mathematik>
Speicherabzug
Computersicherheit
Statistische Analyse
Software Development Kit
Softwareentwickler
Prozess <Informatik>
Spieltheorie
Public-Key-Infrastruktur
Digitale Photographie
Sinusfunktion
Software
Polarisation
ATM
TVD-Verfahren
Programmbibliothek
Eindeutigkeit
Datenfluss
Schnittstelle
Server
Web Site
Instant Messaging
Identitätsverwaltung
Dienst <Informatik>
TUNIS <Programm>
Graph
Physikalisches System
Open Source
CLI
Systemprogrammierung
Front-End <Software>
Kommunalität
Minimalgrad
Skript <Programm>
Ereignishorizont
Modul
Streuungsdiagramm
Datenmodell
Indexberechnung
Gerade
Thread
Strukturgleichungsmodell
Modelltheorie
Hydrostatik
Kernel <Informatik>
Server
Parser
Computer
Dienst <Informatik>
Transformation <Mathematik>
Lineares Gleichungssystem
Computeranimation
TUNIS <Programm>
Systemprogrammierung
Spezialrechner
Graph
Open Source
Freeware
Verzeichnisdienst
Vorzeichen <Mathematik>
Mittelwert
Code
Zoom
MIDI <Musikelektronik>
Flächeninhalt
Gerade
Haar-Integral
Hardware
Streuungsdiagramm
Internetworking
Softwareentwickler
Architektur <Informatik>
Elektronische Publikation
Abenteuerspiel
Datenmodell
Fehlerkorrekturmodell
Applet
Spieltheorie
Gemeinsamer Speicher
Gerade
Keller <Informatik>
Digitale Photographie
Sturmsche Kette
Sinusfunktion
Software
Funktion <Mathematik>
COM
Login
Demoszene <Programmierung>
Notebook-Computer
Instantiierung
Web Site
Subtraktion
Prozess <Physik>
Wellenpaket
Gruppenkeim
Maßerweiterung
Komplex <Algebra>
Computeranimation
Internetworking
Open Source
Standardabweichung
Zoom
Programmbibliothek
Inverser Limes
Vorlesung/Konferenz
Affiner Raum
Spielkonsole
Maßerweiterung
Formale Grammatik
Softwaretest
Statistik
Prozess <Informatik>
Ontologie <Wissensverarbeitung>
Plug in
Open Source
Systemaufruf
Binder <Informatik>
Biprodukt
Elektronische Publikation
Ereignishorizont
Data Dictionary
Numerische Taxonomie
Simplexverfahren
Datenfeld
Verkettung <Informatik>
Grundsätze ordnungsmäßiger Datenverarbeitung
Strategisches Spiel
Garbentheorie
Projektive Ebene
Programmbibliothek
Schwebung
Standardabweichung
Zentralisator
Bit
Punkt
Prozess <Physik>
Selbst organisierendes System
Gruppenkeim
Zellularer Automat
Zahlenbereich
Element <Mathematik>
Transformation <Mathematik>
Framework <Informatik>
Überlagerung <Mathematik>
Netzwerktopologie
Task
Streaming <Kommunikationstechnik>
Variable
Knotenmenge
Datentyp
Optimierung
Gerade
Beobachtungsstudie
Lineares Funktional
Zentrische Streckung
Graph
Güte der Anpassung
Datentransfer
Ausnahmebehandlung
Objekt <Kategorie>
Flächeninhalt
Server
Projektive Ebene
Computerarchitektur
Message-Passing
Instantiierung
Orakel <Informatik>
Fehlermeldung

Metadaten

Formale Metadaten

Titel Simple ETL in python 3.5+ with Bonobo
Serientitel EuroPython 2017
Autor Dorgueil, Romain
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben
DOI 10.5446/33794
Herausgeber EuroPython
Erscheinungsjahr 2017
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Simple ETL in python 3.5+ with Bonobo [EuroPython 2017 - Talk - 2017-07-12 - Arengo] [Rimini, Italy] Simple is better than complex, right? That’s true for data pipelines too. For more than 5 years, I hacked together extract-transform-load (ETL) processes in various different positions (ETL is just a fancy term for «bunch of things that take data somewhere and put it elsewhere, eventually transformed»). I did it as a founder, as a consultant, as a technical co-founder, for some side projects, and now in a big corp (to be continued…). In each case, I felt frustrated with the tools available, and in some serious cases, I had to hack things myself to get the job done. https://www.bonobo-project.org/ Bonobo is the repackaging of my past experiences for python 3.5+, and grasping the basics should not take more than the length of the presentation. Topics outline (subject to small changes) : • INTRO : State of the art / different tools for different needs. • Where does it come from. • Writing a data processor. • Running and monitoring data jobs. • OUTRO : The road ahead. • Q&A Bonobo is the glue you need to tie together regular functions in a transformation graph (think unix pipes). Execution strategies are abstracted so you can focus on the real operations. As a result, you can engineer simple and testable systems, using the same good computer development practices as you use in -insert your favorite field here-. Spoiler : there is no «big data» in this talk

Ähnliche Filme

Loading...
Feedback