Merken

Twisted Names

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and I'm still high and mighty team organized you can Europe was in 2009 and 2010 region will not team members literature who's going to tell us all about DNS interested unskilled fat fees thank you thank you all for coming and
yet so money name's Richard and I'll start by telling a little bit about the spend too long that on the other think programmers from the
UK and john so as I'm
involved in Python UK conferences an occasional speaker and 1 2 point job to organize the conference dinner table plans are and that's why I maintain the table plans of course I'm also an enthusiastic really enthusiastic the twisted which is framework which hopefully will heard of 1 the oldest python frameworks and I have become over the last year a contributor and a cortical contributor and with a critical mass of core contributor and I guess that the thought to maintain your twisted names which is the component of twisted which um and then we talking to you today about and I'm currently working in the Bristol in the UK for a company called clustering HQ uh where we're working on some software to and manage the deployment and the the status of and docket contain it's a really interesting open source project that we're working on called flock you should look better you're any those technologies and working on that with some of the twisted family which is the super excited me OK well I haven't along this talk and I about did this in in the UK last year I over over underestimated how how long can type and so on because I have a lot of stuff this can be a much shorter version of anyone's all the talk in can and a less about myself less about the history of twisted and more about the technology is interested names and also the project that I've been working on recently to implement the DNS Indiana what interested and and a start with an overview DNS well over a short overview or explain why in a minute and I'll give you a tolerantly components interested names and I'll hopefully give you some interesting examples so should quite quite interesting examples and also give you a status report on this project and I hope to have some time at and to answer any questions that there will be some questions so I had planned to give us an overview of the domain name system but um I didn't have time and again need to anyway because you probably already been to a talk on Wednesday by Lin route to explain really clearly about the domain name system and its and its structure its operation and terminology and I think some of the software that you may be familiar with the serving and sending a DNS requests so she did about a much better job than I can't explaining it in so what I'm saying he's just going what shuttlecock digit to great I didn't make it to the talk I wish I had been able to that so it's it's watched last nite and I'm glad that it is means I can't get to the interesting bits of of the this knowledge talk about some get this skip this is this is this get and someone I might Sokolov my talk briefly about software maybe maybe familiar with mixed just as a contrast to the twisted Name System which I'll explain in a minute and so probably your roll we're all familiar with phi which is the original DNS server I think that's true and sites the original DNS server and it's an authoritative DNS server and the recursive DNS event forwarding DNS server and all sorts of other governs the mixed in with it and not part of the problem it tries to do too much is feature-packed complicatedness it's full of vulnerabilities this idea of 1 binary 2 to to satisfy all the different DNS requirements DNS server requirements in state which has been learned and implemented better in another piece of software coal-powered interested like you using bind that I recommend you go and look at power DNS so much more modern much better designed and much more secure DNS and it's it's actually more powerful than binding away because it has a much cleaner way of interfacing with a database factor for example and it also displays the the the duty of an authoritative server from the duty of recursive DNS which is important to avoid and cache poisoning other services you may have come across all unbound and honesty I mention those because they're written by they're written by an organization that I'd been involved with this project and it's only about an element labs and again they are much more modern and much more secure DNS servers dedicated to unbounded dedicated to doing to answering recursive requests and is dedicated to answering authoritative requests OK so let's now gets to the end of the subject of this talk twisted names so trusted names is is Canada's oldest twisted itself so celebrating its 30th birthday this year it's probably started life as you may or may not be able to see from this this check and this is the 1st the change that twisted names 1st 1st landed it was probably introduced as demo of a new unit the what was then you UDP transports and facility twisted I am at the little bit of digging
and found the commits from the beginning of twisted names like and that some of the newer commits that I've been working on and Julian's worked on that I see so you can say is that were originally written by a guy called machines that back in 2001 and that was in the good old days when twisted had a kind of wild west development process everyone was just committing randomly to try and and I guess they they didn't they haven't yet implemented what is now called the ultimate quality and development system which is there the talk it's own right it's the way it's the way it's the way we develop and twist is where developing branches and ensuring that every change that gets merged to trump has been code review and this fully tested code is fully covered and there's an audit trail showing between the ticket in the code that lands interest you should go and read about the ultimate quality development system if you haven't already so twisted names was it's it's kind of actively maintained to start with then over the years it's kind of strict sex being neglected and and then I started getting involved in about 2 years ago and had a background in Indiana so I thought that that's not twisted where I could help and I been busily updating the documentation having test coverage and adding some new examples to demonstrate how to use twisted names and you'll find all of those on the on the new at redox documentation website linked to that in a minute so yeah like the the rest of twisted the twisted names packages really well tested it's got so comprehensive unit tests which are run using a tool called trials which is a great test for uh it then that the unit testing twisted names if you had to read the code and the probably not the most sophisticated unit tests in the world but that's a reflection of the way that these testing techniques improve over time the 1st of these is it is it is a set of ideas it's there was 13 years old so if you if you do go and start acting on it you'll find the bits of it which are kind of hard to read hard to look at the they're hard to look out that's bursting into tears but then again you have to recognize that this this this history of the project it's actually quite interesting to see how a particular developers who have been interested from the very start the change their ideas in their approaches to things like that testing I think that's another interesting told in its own right is interesting from the point of view of a new contributor who has to deal with this old-style code in the new stock and understand how to have what's the current best doing this developing the twisted and so we got plenty of unit tests and we've got to have a reasonable coverage of the code some of the some of the molecules that I don't know whether you can see it on the slide our I'm not very well covered but those are areas which I'm working on that and modules such as the authoritative DNS server and the secondary DNS server not particularly welcome and in fact that's that's the radius wrote ugly head lately in above that's been discovered in the secondary name server in the latest release of twisted and it's policy down to a lack of test coverage this but wasn't wasn't noticed earlier and it's partly down to the port the old style design that's part of the and that the package that the I and others are going to improve that over time twisted name wasn't particularly well documented but that's improving as a set of rewrite I've been working quite hard on improving the documentation the twisted names and twisted as a whole is better documented these days you can you can go like most of the projects these days and and redox on read the docks and it's nice if presented and nicely indexed and easily searchable so I recommend that you know if you're interested you go and read that the documentation twisted names on the rest of twisted because it is much easier these days to navigate documentation
so have it in time I'm running out of time and have found going and reaffirms let's crack on will have a look now at the time of the different modules interested names and then starts at the lowest level and work out there like like everywhere interested there layers of abstraction layers upon layers in the online and was not altered and give some examples of how you can use these low-level API is and then later we'll see some of the high-level abstractions so so let's start with the twisted names DNS uh module now this contains protocol protocol level API is representations of the DNS records and representations of Dennis messages and routines for and for serializing and deserializing these messages from the from the wine and also this in this module it you'll find the protocol implementations both for UDP and TCP because DNS operates over by the transport so we got a little example here which I'll try and talk you through can you see that this the transition and this is this is revealed JS somewhat shorter I can everyone see that great OK so as I am and what we're looking at here is there's a couple things I need to explain and we have got 1st of all let's start at the bottom and look at the the last line which is tasked don't react now that we've used twisted before and you may not have come across this this API this is a new way and for you to start the reactor for a short lived twisted program and and what it does is it says it supplies and men you supply it with a function that you want to run a function which must return a deferred and task react will take that function and supplying with the reactor at run your function and then wait for the deferred that it returns to 5 and the firing test react to learn turn down the reactor and take care of stopping the services in the right order and it will long any errors that have been handled amount of so the priest we then move up to the main method we can see that having supplied the reactor where instantiating a DNS Datagram Protocol so this is this in this example we're only going to be this this idea example we're only going to be dealing with UTP and we instantiate the protocol and then we pass them to react to listen UDP on Paul's 0 which means any high ephemeral hold and because this is UDP uh we don't have any connections so we have to say it may look all to be in a client list that you calling a method called listen but we are going to send a UDP datagram and we have to be listening for the response whereas in TCP the operating system would set up the connection for you and you wouldn't have to to choose the ephemeral port the response comes in on and so so we listen we listen on the UDP ports X Y Z and then we send the query using the protocol using the weak culprits in the query to send a DNS query that's 2 in this case the Google DNS servers and then when the query has been answered are we going to print the result not by way of a that adding a call back to the deferred returned by protocol the query but again I was I I sometimes think we should get started we talk with an introduction to differ but I guess everyone's heard and more people these days of familiar with the the idea because it's now part of jobs script things so yeah when when the when the answer comes back we simply we take the results and we take it so that the result is a message DNS message which I'll I'll explain a little bit more detail later but the message has a has 3 attributes hasn't ounces attributes and authority at tributes and an additional attributes and those represent the 3 um the 3 categories of records that might be returned by Dennis so in this case all we're doing is printing out the answers returned by the DNS server and in particular we're returning we're going to print out the payload and which is the uh the the the time the data in this case the quad a record it might be a record of MX record we're not interested in printing out the header information which wraps around pay so I'll say the Treaty of I'm running out of time rapidly again so we we've got an answer from the server coordinate 1 single quad record now that's quickly move on that to the next example so that was a client
the next example is a server and in this case it's quite similar but we we instantiate the datagram protocol this time with the controller which takes care of handling the the the the the the query which comes into us and when a query is received received by the Protocol on ports 10 the 10 thousand 53 and are protocol and calls out to the controller and calls its message received method and it's the message received method on the controller which is responsible for for and for constructing announces that message so this is how we write low-level servers and low-level DL interested and in this case we're just going to receive respond with the candidate a record with a with a fixed IP address so that makes sense I haven't got time i'd like to I don't go into any more detail about 100 time that's the there's the the server running and there's us that issuing a request to using data OK so now there a low level API is if we move up now to interested names don't clients this is a much higher level API much more friendly way of interacting with twisted names and in this example we going to you know introduce a couple of new concepts we're going to see use twisted names to look up concurrent concurrently the um the reverse DNS records for a whole class the network and and so you can see in our main methods that we are constructing a list of all the IP addresses in a in a in a slash 24 network using an really useful module called next which does all the uh the construction of those reverse DNS names for us I think there was a there was a lightning talk on yesterday so I recommend that module and for each of those reverse DNS names we're going to call clients don't look up point and client has a has a a series of these look at methods 1 of each type of DNS record that you can receive from a dentist does not all of them but we're working on implementing some of the missing ones but it has a at method for almost every commandeered S and type and so we construct a series of and this define all of them in flight and all of them and then added to what's called a differed list now deferred list is a really useful and API for collecting and the responses to the list of different and that it as far as it's called back with all of the defendants have themselves fired fail and so it they handle the results in this example we are living through the results checking whether the result was a success or a failure and it was a report it was successfully print the um we go again print payload and we also print a summary expert and summarizing how many of the requests were answered successfully and how many of them said that what we're not answered either because the record didn't exist or perhaps the the the query time down so that the results to that far as follows so you can see it all happened rather quickly and because everything's happening happening concurrently that's a real advantage of using twisted this sort of way I might skip it now to a better example of that and 1 which follows on from the instead talk on wednesdays so let me let me quickly summarize some of the other modules we have an interest in you have the modules for creating DNS servers have an example of that and it's really easy to use because there's a twisted DNS plugin for the twisty and uh uh command which comes twisted and so you should explore that to explore all the options that you have using that command that twisted itself from it's the twisted matrix . com that domain is actually so from from a twisted DNS service sex it's it's it's pretty it's pretty stable and said it's not it's not a it's not a fully featured in cell that is good enough for some cases and you can see that when you start a server logs to standard out and that we can query that's once it started and we also have an authoritative server and it's interesting that time to go into it that you can load DNS so based by defining it as a Python module and so here we have a Python module with extend their describing is but also the the the and and the the objects you see their oral and global which are imported at the time this module is evaluated by twisted names it's quite a clever mechanism but you should look into that too it's interesting piece of and
again as the example of how it runs in again these examples are on the documentation the twisted documentation site so I recommend you can read those and there's a bunch of the model which I have to skip through a common contains some helpers and and some some API is common to all the twisted client and service results I won't go into caches about caching responses to the queries research is a bad thing recursive DNS resolution which lintel that helpful the 2nd tree is about transferring zones and serving them authoritatively and from another authoritative server which some of you may be familiar with and the point I want to make by describing all of these is that these all of the building blocks can be put together in interesting ways and 2 identical live examples of this on the website for example you could create very easily using a low level API a module or a PostScript the test the compliance testing of DNS servers and clients because you have complete control over the years and the the flags and the and the parents that you put into messages so it's easy to construct an uh non-compliant messages to see how the DNS servers respond to those or it's easy to use to you and it's easy to see how clients respond to non-compliant responses from so that's a that's a good use of that these building blocks we use that we use twisted it work for a twisted names and where the functional testing so we have a bunch of code which does and DNS lookups and we want in in our tests supply canned responses to those DNS lookups and it's very easy using twisted to set of a set of lightweight DNS server and tearing down at the end of the test it would be released to set up a database back DNS server or a DNS server which looked up its data from a REST API for example would be easy to combine these with other part of the components and twisted like the web module or the that module uh to look up DNS records from held out the database will to learn to control and manage the DNS records in your server using a REST API some now let me see if I can quickly show you the uh finish off with a quick example a more complicated example of using twisted named so in Linz talk about is really interesting is that a tool called DNS map which can is a tool for all and full brute-forcing is owned but for guessing which named maybe in his own DNS but does that using a dictionary of words of course common domains and as as you said in your talk in that so it's is it does in the series is quite quite a slow it's quite slow to complete because this is this better thousand words in dictionary so I gotta be interesting to write to write the same tool and twisted because it can do all these lookups concurrently so this is this is how dentist the dentist map the original Dennis maps documented you pass it a domain name and you pass it become the list of words which it will then look at each of those words a subdomain of the supply apparent but as you can see it is quite slow so I started this going against sponsored by the common 48 seconds later today only reached cheese so it can take forever I want to compare that to another example was the example I wrote which and a lot of actually giving you the link but would always code on my gets home page I got a link at the end of my talk and in this example we are and we're actually sending all of our request that concurrently but I think that it is not a lot we're not sending a thousand requests the time using some another interesting part twisted called the cooperated with the cooperator um API and in the task module and what we can do using that to limit the concurrency so we can say is only have 100 in-flight DNS requests at the time so we're not going to overwhelm the DNS server query and I haven't implemented in this the random time that's the DNS map actually puts it in between its requests is not quite the same but if I show you the results in this case it's looked up old and by the thousands of domains in 2 and a half seconds such as the demonstration of the power of the twisted in the power of the and gate guys and the and the way that it can be it can efficiently process efficiently send out requests and process the responses always code on my guess of page now I think I've run out of time so I want and I want to talk
about my project I'd love to talk to you about it I'm gonna be sprinting at the end of it tomorrow at least on twisted so that if you're interested in helping out so learning about the development process if any of your Dennis experts and help me with my projects then I'd love to hear from you it's all about the against in a set of golf funding and so you might get paid for it I had made modest progress and I think that's the summary of this talk is this a summary of what is this how I want to be still to be done and trying to cover and so those are the links so that makes the documentation that's the gates linked to the examples in my talk want to investigate those if you are put this up with these on gets of as well now linked to these from Twitter or some way of making available on the conference website have I got any time questions not the questions that you may have to Woods may be delayed due to the
few
Berline
Code
Direkte numerische Simulation
Direkte numerische Simulation
p-Block
Dialekt
Computeranimation
Programmiergerät
Bit
Demo <Programm>
Punkt
Familie <Mathematik>
Versionsverwaltung
Element <Mathematik>
Computeranimation
Metropolitan area network
Einheit <Mathematik>
Prozess <Informatik>
Eigentliche Abbildung
Vorlesung/Konferenz
Kontrast <Statistik>
Gleitendes Mittel
Große Vereinheitlichung
Nichtlinearer Operator
Datenhaltung
Computersicherheit
Ereignishorizont
Teilbarkeit
Software
Dienst <Informatik>
Tableau <Logik>
Digitalisierer
Client
Server
Projektive Ebene
Aggregatzustand
Web Site
Subtraktion
Selbst organisierendes System
Mathematisierung
Automatische Handlungsplanung
Schar <Mathematik>
DNS <Internet>
Transportproblem
Framework <Informatik>
Domain-Name
Software
Datentyp
Direkte numerische Simulation
Datenstruktur
Gammafunktion
Leistung <Physik>
Videospiel
Open Source
Routing
Physikalisches System
Quick-Sort
Komponente <Software>
Softwareschwachstelle
Caching
Mereologie
Leistung <Physik>
Speicherabzug
Direkte numerische Simulation
Rekursive Funktion
Verkehrsinformation
Resultante
Bit
Komponententest
Spiegelung <Mathematik>
Punkt
Prozess <Physik>
Selbstrepräsentation
Computeranimation
Übergang
Metropolitan area network
Client
Prozess <Informatik>
Skript <Programm>
E-Mail
Gerade
Softwaretest
Lineares Funktional
Sichtenkonzept
Termersetzungssystem
Kategorie <Mathematik>
Abstraktionsebene
Wurm <Informatik>
Systemaufruf
Abfrage
Rechenschieber
Dienst <Informatik>
Funktion <Mathematik>
Menge
Rechter Winkel
Client
Server
Projektive Ebene
Information
Ordnung <Mathematik>
Message-Passing
Fehlermeldung
Subtraktion
Web Site
Gruppenoperation
Mathematisierung
Implementierung
Code
Task
Virtuelle Maschine
Weg <Topologie>
Datensatz
Task
Modul <Datentyp>
Netzbetriebssystem
Direkte numerische Simulation
Endogene Variable
Koroutine
Softwareentwickler
Schreib-Lese-Kopf
Attributierte Grammatik
Autorisierung
Einfach zusammenhängender Raum
Radius
Protokoll <Datenverarbeitungssystem>
Verzweigendes Programm
Einfache Genauigkeit
Mailing-Liste
Physikalisches System
Modul
Viereck
Flächeninhalt
Mereologie
Resultante
Matrizenrechnung
Server
Punkt
Klasse <Mathematik>
Zellularer Automat
Rechnernetz
Extrempunkt
Netzadresse
Computeranimation
Übergang
Eins
Metropolitan area network
Client
Domain-Name
Datensatz
Reverse Engineering
Direkte numerische Simulation
Datentyp
Endogene Variable
COM
Konstruktor <Informatik>
Expertensystem
Protokoll <Datenverarbeitungssystem>
Reihe
Wurm <Informatik>
Systemaufruf
Abfrage
Mailing-Liste
Plug in
Modul
Quick-Sort
Konfiguration <Informatik>
Objekt <Kategorie>
Dienst <Informatik>
Funktion <Mathematik>
Client
Datenerfassung
Gamecontroller
Server
Reelle Zahl
Ext-Funktor
Personal Area Network
Message-Passing
Verkehrsinformation
Resultante
Prozess <Physik>
Punkt
Datenparallelität
Computeranimation
Homepage
Übergang
Netzwerktopologie
Metropolitan area network
Client
Fahne <Mathematik>
Chi-Quadrat-Verteilung
Haar-Integral
Bildauflösung
Inklusion <Mathematik>
Softwaretest
Datentyp
REST <Informatik>
Datenhaltung
Reihe
Abfrage
Zeitzone
Dienst <Informatik>
Verknüpfungsglied
Menge
Twitter <Softwareplattform>
Client
Datenerfassung
Server
Projektive Ebene
Reelle Zahl
Cloud Computing
p-Block
Message-Passing
Server
Web Site
Code
Task
Textur-Mapping
Domain-Name
Datensatz
Benutzerbeteiligung
Arithmetische Folge
Direkte numerische Simulation
Endogene Variable
Digital Rights Management
Vererbungshierarchie
Softwareentwickler
Gammafunktion
Leistung <Physik>
Expertensystem
Zwei
Mailing-Liste
Binder <Informatik>
Modul
Data Dictionary
Komponente <Software>
Caching
Mereologie
Gamecontroller
Wort <Informatik>
Direkte numerische Simulation
Rekursive Funktion
Numerisches Modell
Code
Vorlesung/Konferenz
Computeranimation

Metadaten

Formale Metadaten

Titel Twisted Names
Untertitel DNS Building Blocks for Python Programmers
Serientitel EuroPython 2014
Teil 24
Anzahl der Teile 120
Autor Wall, Richard
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/20024
Herausgeber EuroPython
Erscheinungsjahr 2014
Sprache Englisch
Produktionsort Berlin

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Richard Wall - Twisted Names: DNS Building Blocks for Python Programmers In this talk I will report on my efforts to update the DNS components of Twisted and discuss some of the things I've learned along the way. I'll demonstrate the EDNS0, DNSSEC and DANE client support which I have been working on and show how these new Twisted Names components can be glued together to build novel DNS servers and clients. Twisted is an event-driven networking engine written in Python and licensed under the open source MIT license. It is a platform for developing internet applications.
Schlagwörter EuroPython Conference
EP 2014
EuroPython 2014

Ähnliche Filme

Loading...