Merken

Writing Domain Specific Languages with Python

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
and what Unit women low only so this talk will be about writing domain-specific languages so I think the 1st question we need to answer is what is a domain-specific
language come and and so our domain is like it's like a business domain of problem
space on the sediment languages could be a kind of language that is more suited to solving problems in that space than a general-purpose programming language so piconets a general-purpose programming language advice to the language well let's meet a few examples so it's equal domain-specific language it's much more efficient for writing a query against a relational database that by constructing some objects in Python that represent the the things you want to query in the tables was the old query from and so on CSS domain-specific language regular expressions we need rigorous pressures were domain-specific language cultic parcels in the standard library the domain-specific language for but considering things intended in this case for of people who don't Python to change direction string formatting so got to 2 domain-specific languages free as the media formats some of our methods on a string is a domain-specific language that will interpolate the things in curly brackets and they times have a special and undone the format method that means that you can use the stress time syntax within the collective reStructuredText that's a domain-specific language for writing technical documentation so both of these problems so that those domain-specific languages are trying to solve and so 1 I think is that the readability of of the of code using those languages is going to be all systems using these languages that can be better than a nest of Python expressions and and syntax and so it's also kind of as a as a corollary of it's gonna reduce the repetition in in the codes and the B as you reduce repetition you reduce the amount of syntax the of the errors you can you likely to make your parents so the right ability of the categories but 1 of the other reasons we want do this is to sanitize or manipulate the data as it's slide into memory or that the the the structure has is loaded from a DSL into into program memory and then as we saw this was a use case where non-technical people um are going to be all people who want to many points and game to be manipulating these things so as witnessed as I think that we have of more constraints on what exactly would want because we want we want to use poisonous and general-purpose implementation language and we just to want to use the DSLs to simplify and improve point and so we need we need use Python for implantation we need to trade off like where we want to inject extra expressiveness into an applicant code and sometimes we used to produce strings as the within a small structure of the sort that later and we're going to want to work with the kind of patterns that of familiar to to apply the developers so you don't want to surprise people by simply changing the semantics of and the square brackets or something but if you all working in a in an environment where you're going to be FIL switching between those 2 context so I when we talk
about to building a DSL you're generally going to be converting some piece of text which might be so within your program code that could be from another file into a structure in memory let's say that it's not like this this is a made-up language so that you can imagine is boss of a service that Gretchen logs and those things when a regular expression expression matches and in this case I'm I can kind of thinking of this as a language that would be intended for non-Python people to right so this is like the user-facing interface of of a consists of and so we're simplifying the and and removing bugs from the configuration of this by putting into a language where we can see more easily express what we want to to say and then not because these people with Python said these the of turning this this plain text into structure is is posing so or May talk about domestic life is is is parsing and say in this talk could include like a guide parsing I think what we draw out what we want to get to with the mainstream languages we consider constructive more coherent narrative that will serve of help us to understand how to design these things and and work with them so the 1st approach to getting a parser I guess is there is one built into Python so it any code that you write in in
Python Python will be used the past and will have structure in its own way or when executable construct structure so we can manipulate the we can use all of my friends dynamic tricks to construct structure in a more elegant way but that doesn't necessarily executed Python semantics so the 1st thing going to talk about just some meta-programming approaches to event metaclasses so you know what a cost does in Python than it is you define it and then the the object itself is a factory for instances of that event type so this is a class cold dark and when you call Dr. returned to duck instances and you can call a method on instance that's quite semantics with medical you can completely change format class definition dollars and what would the object that is the assigns to the name . by the be the class definition for that object would do so is 1 for example this is the 2nd of something I wrote a few years ago and I guess this kind of similar to scrapie but what we need do here is to find a number of X path expressions of the will match against a of the Haitian a document or something that's been free from euro from and if you look at the example where it's actually called is not even returning a strict review instance is going to return a dictionary we need to do that quite very small number of people OK so this is kind of an approach to uh combining like treating say functional your you crazy 1 piece of functionality that you're doing in such a way that you can declaratively say is specified parts of implementation and it's even like this part we can inject custom functionality like cleaning a value by stripping of um of a cone so how would we that I will 1st we need the facts of the facts where you got so the subclass of facts but they basically all the take a a and export expressions random and so you need to have a beginning and an XML document and you can then just evaluate the export return a result the metaclass itself even says there 2 phases of which this so this is going to inject code into the come into the class definitions so I guess the on the news media and on the new method is called so that the at the point where you a class definition and so to the point where the indentation goes back and they name gets assigned at this point they underlined the new method will get called on the metaclass and some and in this case it's going to collect all the facts of the big in instances of the fact that we defined in the class space into a dictionary for free later
and then the and the call overloading the the you know the called parentheses operator if you want all the class object itself the following limitations return an instance of c and b by by changing completely would returning addiction scanty evaluate all of those ex parte expressions against whatever we pull them from the euro and then run any cleaners and so and then is a you create base class so this is you could assign on every instance of the strength of the and is somewhat nicer to create a base class and then extend from and that's what you need so beyond his another approach for DSLs in Python um in in python metaprogramming and this is a using of context managers to construct structure there is a library that does this the code of top my head but I I don't like this very much this is a bit slow verbose and forced his name server and all also uh and implementation problem here that you have to be careful about which is perhaps this is then not for this is rule requiring global states that is the you know all these functions that been defined here a somehow storing the states at work which they are in the I in constructing a review of the result of lattice property inter-stream immediately I said that's sort brings concerns about the thread safety and race conditions into into the into the mix another approach is operator overloading so this is a giant of RAM is so this was the the the DSL where he doing double on the schools and double underscores and you'll constructing an expression which is thank you for and the the but 1 against Oprah here is the the use of the point to mean all because in Python called overload the logical operator you can I believe the bitwise OR operator which is quite I and lots of things to but there are problems with this so this is a this is supported real progress and we're using this some some horrible stuff and had effect so this is using the ampersand operator to mean not bitwise and biological and and it's overlaid it's the comparison operators to greater or equal to to construct an object that sort of matches that thing that this is a representation of the expression that would be I guess it's kind of deferring the of the act of doing that comparison that's amazing like in this thing he could spell an
infix operator like that in Python so you um it turns out like that is actually a the left shift operator followed by right shift operator and he takes place and it looks like an infix operator and you'd implemented something like that but they look that because they do it please the place this really breaks down is that the so so-called overlaid logical and and or operators we can oblige lay the bitwise and and or operators that those all are different although they have different operator precedence so that means that the where is the and and are the right precedence to makes the simple obvious code work the the ampersand bind more tightly if you will and say the UN actually seen cases where where over 90 people have written the top 1 and then the bottom 1 has remembered his broken up 1 and meant there was there was on that slide so practice like that and obviously that doesn't work so that's not very readable so to what we doing here we're aiming for readability but we've created constructed situation we need to put inside the brackets together brackets right so that you're not really achieving the readability resent but here's another example is like a different example of related Unicode based the term but this is where so good at it but so that the topic of this hot springs work so you got a table object and accessing the name of the column on the table were also unstable table and then the intention is that you can filter down the table by an expression in square brackets and you filter at 1st you you compare the overlaid comparison operator that will return a an array of the of the rows that you want and you can select using that that lists but that means that table operators when you compare them to you have to evaluate will and return a list stays that is the same length as the table In number of race and that means that we had people writing tests where they're trying to compare it a return value function to none of them and obviously say if you did a search is not that would work but that has passed passes even if you have a table said overloading operators is a and I was overloading so that comparison operators in a way that changes the semantics will cause problems if you want to over the past operators then make the return of and that sort of makes sense in the way that python normally works or otherwise construct a situation we come right that text and have indirectly pass the the case of then um I guess moving on to a a way of writing DSLs where where you are using Python's own possible but not so in line in the code so you just as we've seen overloading operates things that bought by using as the module to construct the Python structure and then completely change the way that actually excuse so the this is maybe slightly nicer than the the way that we saw writing previously on it means that the operator precedence is correct and it's not very surprising to a Python developers to but in in how this executes but in light is this is a this-worldly tricks where this like a certain elegance in the box it's not extensible so you can't ever take this beyond what I think into in its passing c Python as a favorite content parser and you can abuse or those features up to a point and then and because there is like user paid implement that so St . Paul's will give you the Python's parse tree for a piece of code and then there is an AST don't know visitor which will allow you to sort of walk through the the and do something with it so In this case we just sort of returning a they were returning something like that like an NIST but all of the different type of thing where the semantics of Rome so that's equal to semantic and this is even more insidious so this is some of this is a very of the last thing but actually it's been completely hidden within Python code so you this is they think this is any different to this this is this is explicitly like with this is a string and so it doesn't look like Python that's going to be executed as Python it in a certain sense of it's possible like it's an excuses Python by overhead this looks like it's going be executed Python and for certain reasons the what if the cell is doing is constructing a a sort of it was found Saturday analyzing what could you do for various reasons but well that's horrible so exceptions that work in that and so and the behavior is very different to the way the Python will would execute that code then battery surprising to to users 1 this is another 1 that has a similar kind of trick um maybe a little more successfully I guess because you pass in something that looks like a generator and what actually does this gets the compiles to
structure and then back is turned into a sequel query that represents what the bytecode would do In in evaluating the generated by ridiculous so I I don't I don't really have any particular feelings about that if it works it works in but there is an interesting thing to note is that decompilation for turning a white codes into at a structure and all and then finally into a sequel expressions pre-selected passing a DSL because is converting a stream of so characters into a structure and then converting structure into and whatever else you want evaluation structure In summary over piconets programming tricks I guess there's some clever things in there but then there's lots of surprising things that's all pixels I guess that I can point you as as you start working with these things and obviously that will be apparent immediately you might serve define this thing and it has this great and then use of fair in a really complicated expression like patents and there's a very good quotes the ice on a blog post from Mozilla last weekend which was that we could extract good instructions on a continuing source of greatness where's bad abstractions are a continuing source of pain so during the data abstractions right and the seems like if you get if you actually pick a painful part you can continue to suffer pain for a long time all these approaches online meta classes that the nice because and they don't really suffer from the operator relating things so you can and that's what we're going suffer from those pitfalls in the same way and it's just a way of fashioning a piece of some of the functionality analysis of of a piping pass statement and that's sort of the intended use of metaclasses the 1 of the intended use of that so then revealed like a different category of of how we can count gets a structure that we get a positive answer of uh so how we get a structure of text that we get a positive there are plenty of parsers that we have access to so here are some of Jason and all widely used confocal so I mean it's it is intended specifically for this but you could use it you could be used as they can create something completely different after config parser but there was examples this so this is the elastic such DSL very familiar with this is at present working with this said to him yeah it's this is sound current loads of curly brackets and and is not always clear whether curly brackets you guys so my favorite is
like you get to the end and this is like an empty dictionary is like what a network and and so you need that extreme that you need like exactly the nesting that you have and it's not a very elegant thing and can I guess you could take the view that for elastic search this is this more like a one-level protocol ADSL it may not be intended for human use writing as a programmer some of you may expect a rapid completely up in a nice Pythonic API um but all of the elastic 1st documentation is written with these sword you have no choice but to engage this in your API provided we documents all elastic search using their own components and they don't and this is sensible playbook so this is the real mishmash of yam all in ginger and that the US and loops in there and is like this magical lookup thing I've right so I don't know the answer all that well but this is the reason why I'm not going to ask so this is just at the point where you break free of this and write something new gets so I'm on the subject transform of surrealist subject gamble for DSLs uh this I guess is kind of tangential but there some who can spot an error in this woman all right kernel yeah so the column in the middle of that line up as a sort of non-technical user I'm I'm writing just some metadata for everything I write the title as I've been told to look for in a column and suddenly but not a list of strings of a list of some strings and some dictionaries so that's a potential pitfall you can spot the bug in this way the I think we should use is of under than that's a valid to strings this will be a dictionary containing a dictionary of that that where the values of strings interfaces following so the I tell you Ontario is key to with the string all and all that evaluates to true the so I guess that's the you likely trends that maybe not in some parts of light I kind of think gamelans tragedy way too much it's like a superset of Jason and it has a of features and their and their we understand so that well is not perspectives and document that well and I he is more of a so sordid pseudo readable serialization format of the kind of more complicated stuff what 3 well as I I think it works may be kind of in a really simple cases as a positive so you In summary of the off the shelf poses we can get our hands on the wrong right I guess is not very suited to your domain it doesn't really fit that's the that's the overriding thing is that that's how your shoehorning a structure into just because it's off the shelf in your your able to use that doesn't mean that is really going to figure main well um and as you feature set grows you also have struggling more to shoehorn were back into the the expressiveness of those ways languages so that brings us all to the last category but cover and the I guess that the biggest category from which is parsing error cell so this gives you complete freedom to to break free from any of the existing abstractions and start fresh and press come up with a language that is completely seated to what you want to so vibrant a few DSLs my time and this is kind of the approach it takes area existence like a a blank text editor and you start thinking about concepts from other languages or from other units things that might be familiar to your target audience um as as kind of the syntactic elements and then you that you know a new problem domain well use of that start cos some of the examples from your your domain into a text From the iterates a bread and uh and eventually come up with a few examples of of the kind of language that you want to create and then having done that we should use could be very proud of like your languages literally nice and then you can straight away take this example breaking into test cases and so you can saying in that a kind of of covered some of this stuff when you are designing a DSL you want to make it familiar to people you want to make it so that that work in the the same way as Python what promised the public DSL is the scoping is just completely wrong and so you ought to try and make it work as people expect the principle of least astonishment and then how you pass this language I guess this is a lot of things so you can't you if you composite you it's all the work so you want to understand a bit about then you can designed for the possible you can write this this is 1 thing in particular there which is that is very useful to you embed DSL sometimes in Python string literals and so you ought to have like some sort of concept of what is going to be convenient to that in a string literal and what isn't the only see the likes of raw strings them and and so on but the assessment to pet tension um account that a bit later I guess so I think the 1st approach the I usually take that the 1st principal work for a full the simplest approach that work for parsing itself is that line was this suggests a library From this is good so we have a template structure among organizations and the was a lot of code and particularly test code written like this the rate of return is the force that was a lot of syntax and there was a lot of quotes from and what also happens is these 2 lines start to drift apart and so before when you got like a schema defined elsewhere and what European into it is no longer clear it's no military have fields much up and so I wrote this simple line like this I inspired more or less by a lot of lettuce um go can syntax but we have tighten tables and his these time tended to have them and then that will always his becomes a very sort of literals all of a testing ways say this is my input and then going to run a function get help and you can
visually compare what the input and output are expected to be that's really great we're writing a test with the structures and but then to draw another point from what I was saying before so saying that if you're working in a string literals it I think we consider that as a consideration of this is the tolerant of indentation and that kind of thing so and also you can put comments inside the uh literally on the end of the line so before the lines but not in the middle of the table actually um so so it's flexible in situations is likely to land you code how you however you want so here the pass something 1 way is the way that it so this is sort of basically I'm considering writing a parser from scratch the way that would do whether starting it is right a finite state machine effectively say you start in state and then consider each line in the context of the state the and each line will cause a transition to another state or maybe outputs value and that that like this that language that has head is in the body so the all states there of which 2 sort of terminal states that sort of it may be uneven encoded states that is the kick out parts or return exception and so in the expected state you get ahead of the stand expect had a slight but you get a blank line you moved to expect all the states and then any line is his body and this is like something that this may be something the implementation of this is in loop um I I don't want to mention that I think the thing to note is that uh your switching on 1st we're switching state so you're not the evaluating the line and then considering in the in the context of the state and I think it's better to have a way around where you take a line and then switch immediately into different blocks of code depending on the current state in which you can do it as a class that this is the slightly more organized way of doing things thing I guess which is that you current status just encoded as a method and the year a by switching out the methods in this case the bound method you'll currently using an instance of you can transition state power say funny a she's like technically complex pass rate constant regular grammars and I'm I in no way mean that I guess when means because you can sort starts and adding a stack you can maintain state and you can continue building up this sort of from the parser the 1st enough time and continue to parse the increasingly powerful stuff and but your considering 1 line at time but that doesn't mean that the structure is 1 minus sign it means the structure can I move by span multiple lines and can switch states across these lines and so I guess so we need to dig into battle with further 1 can span multiple lines but what does it mean to parse 1 lies in the context of a bit more parsing theory so quickly detouring parsing theory there is a book on this and it is very mathematical and I don't really suggest you read it because the role of libraries that were just so that they call that mathematics down into much more simple stuff and meet some of those later I'm about to summarize some of the basic passing theory is in it is usually splits into 2 phases so there is the lexical analysis tokenization phase where you are given I just like the fact that it takes to start breaking into the words for or symbols that make up the language and then given that sequence of symbols you can start assembling into a tree lexical analysis is usually done with regular expressions mn just what regular expression match next and indicates what token type and value has and then the syntactic analysis has dropped various algorithms that are the really mathematical but so this is an example of the tokenized module built into Python and the provides particles of light and this brother simplified because actually it's like 0 points of the data about each taken on an expression like X to the power of Y plus 1 and it will output the tokens we see that some of a left bracket less free left princess name X so you know that has the type of as well that is the sequence of the the words as well but white space is not 1 of those words of them and then since we consuming the output of tokenized behind the scenes the world convert that into a structure so you got like a nested did not somewhere the opposite class and to a name is a class and so on so coming back to line was causes this is this is all there is to to so distinguishing line was passes from anything else that you just taking a line as a token but then all the same practices say if you've ever written line was policy you're not actually at that far from writing a possible is affecting language and and then forget this brings me onto a brief interleaved where I wrote a line was positive reading right so in October 2014 I want high we create the week-long Games training competition with an adventure game code legend of goblets which is kind of adventure stage place narrative and the some of the characters is sort of moving around in the stages was defined as the state was like states right states um and that's scripted descriptor with a language like this so literally sat down at the start of the week on Sunday was like and then I have to write an adventure game but of arresting scripts that I want to have a very soon I realized that description needed to be executable so I added all the features but it's a by choice cannot afterwards but um the sort of building structure even before a written code like this needs to evaluate and move people around the station at this point in a breakout puzzle and so on if we have time later on actually that but I so that we nearly it's so full passes we if we consider line is taken and we've we've written something that can construct a whole IST out of line ways things but
there are tools that make this even easier so we need to understand a little bit about how you instruct 1 of these parser-generator to pass along the tree DSL this is the kind of stuff that you'll see when you're describing languages so this is a sequence of productions that define the grammar for a DSL and in this case is like a simple calculator expression language seems to be like the de-facto standard for demonstrating passes from other it's kind of not that effective not to business not useful thing to write in the real world often um but you notice that Yugoslav records instructions where there are an expression will match the the an expression positive so that means that you can nest arbitrarily many a plus sign for example 1 in term plus sign and so on and and that will create a structure that is to because on on the left hand side uh um and the way that this grammar has been written is such that the precedents of the plus and minus operators is correct in as as in normal mathematical syntax and so what is that like left recursion mean there is something we need to know in order to construct a parser is about a section of the survey if you have an expression that has most popular operators of the same precedence then but whether they brackets left to the right so let's association of operator and say there is brackets the left hand side expressions and Python is left associative for what its operations OK so all but 1 of his operators and so there grew up I guess so we need to know that
we that actually so that makes a difference on how our uh expressions are evaluated and the other thing that matters is operator precedence before we already met in a sense but how that we need to know that the expression we right if so if some class and asterisk had the same precedence for example it would construct the wrong structure so we need to know that when we write that the simple undirected code than does that as as against policy effectively gets bracketed in the way that you would expect that I can mention that scientists give on time so this is a proposal could be aligned and full Python Lexia Aleksandr act of a well known see parser-generator so area this is a Python implementation of various things and this is the tokenizer faces criminal selects um so that is this is a DSL of each of these is a regular expression that will define token types and so the capital of the letters in the the names of the things all the token type we got to have a list of the tokens are also included in the module and then there's magic that times the elect the number is slightly more complicated because the docstring is the regular expression and then you could provide a mapping between what we actually pass which is a string and the value that we want to return as like a tokens value and then PLY and construct concert like either in in the same module or different modules based pour tokens in the lecture and then you ought to finding the grammar productions another DSL as you can see in the P. expression did not for defining what the grammar is in this in pretty much the same kind of language as the like beneath the grammars often described in so talk through this kind of it more this is not constructing in this case this is sort of constructing a parse tree and collapsing it down to result as it part it so the operator is actually and pulled out and executed as being called and that's like I guess that's just the rest of the positive so you again it will pick up everything in the module namespace you call the active you actually get a positive and then use that like this so positive possibly give the lecture again and then you can evaluate an expression of because the expressions being evaluated as the syntax tree collapses just simply parsing the results of parsing is the result so we got another library passing takes a different approach so whereas inference of a couple of really fast parsing algorithms which are they all are 1 and that's a lot of preprocessing that is recursive descent which is more powerful but also potentially much less so recursive descent allows the positive backtrack in it follows mentioned in fact can backtrack even if it didn't those match so it's trying to match tokens are and you can you can have a look for the best possible match those tokens and uh and try it would the routes through code which can be kind of expenses my 1st pi passing cars around incredibly slowly I just use the wrong or by right didn't mean all and and natural and so so through bypassing the role these classes the velocity is constant will have come similar similar API these ones are the ones that sort of match tokens so the quoted string that means that you can match together Python quotes all you know that from a quote up to in this case is promising quite up to the next quote excluding a single quotes the state a backslash um and unquote results that will mean that when we actually get the constant it's just been converted into the string is represented by an intersection quite powerful and little bit of functionality that and so so know right regular-expression matches and just like a literal common and you can combine notes so you the this is not DSL they have been operator going there so when the pipe operator means match 1st so is there is a classical match beacons short-circuited by the actual the actual cover right now is to use the the pipe to match any of those things this is kind of also defining something very similar to the grammar rules in them back to the grammar but defined
in Python syntax of productions here of the combination of those things assigned to another value and that the the thing that they get assigned to it is also an instance of armies parsing objects and this stuff there around so this value thing the Semantic Web because is it needs to be recursive which means gonna define and then get it is value later and so that's where the left shift equals comes in as the ugly and this is the the the problem with operator overloading I guess that you you run into these concentration each of those parses allows you to in the same that we saw in pale pairwise you can match the the token as its is returned from the match In this case the command the man be constant product which was like either a string or reflect on it because when handily describes days in python syntax accused Aristide literal about how to evaluate its actual in Python value list is very similar thoughts and you because of the way they are defined a list which is with interleaved comments from as in Python you need to exclude both the leading a square bracket and the comments and the training square bracket which is what that's lifestyles and then use just by calling upon strain on any 1 of these things so they will provide same because they will provide a set path action and they can policy and some input matches so it is another 1 that there are the there are many of these but they will take this so moralists income of approach which is matching tokens and providing some a grammar that defines language and parsley has chosen a different approach for its DSL for finding that the grammar of policy insane string parsing triple quoted strength um so with these of this is something as it was right was able to rewrite that stuff that we sort of beginning middle of weird operators into something that would pass with exactly the semantics I wanted and proper error handling and matching things like is not novel which is not something that's valid but never right is not very to capacity right is not that you can write on copyright and there are some of the sequel things like like him and so on say that much cleaner and to avoid social problems that we saw with the operator overloading and missing brackets and so on the OSCE's PL white defined to parse a metric definition language that's kind based on pockets and that was quite useless times said so I guess moving on from distant like some of the practices writing and work with their cells this stuff you generally want to have a nice to have a new way with these things which is 1st of all the ability to having constructed unless the converted back into some source that might represent as turns out to be very handy as 324 writing tests as well um and also you need to work quite hard to ensure that the errors come out clearly with the line number because really horrible to have a DSL the working with and this is there somewhere and then you like in a thousand lights it doesn't help me at all um and uh and things like syntax violent highlighting and limiting the that this for example is how you syntax highlighting them is just a tokenizer so aside a set of regular expressions that match certain types of of token and then the colors that you assign to it I don't really going to be syntax with again so we're summing up there are times advantages to doing this I found
it very useful for improving the quality of codes for making it more literate what you mean and for reducing bugs but only if it's done very well you can also use for example the where expression of the soul and take something that looks very much like sequel but completely validates its there's no possibility of injecting code in the um and so that can use um so that used to protect yourself from all kinds of vulnerabilities on the other hand if you encounter a project that is using tons of DSLs that you've never met before and you don't know what they mean and all pretty well documented then that's a that's a potential pitfall for for new developments makes it difficult to work with your project and also by doing this if you so instead of Python you presenting a DSL that implements them something that is you will be working a lot with the your data please large amounts of code and then you can start missing the IT support that you have for Python which may just be a gender is brilliant but it may just be syntax highlighting all water adopt things so you're encourage you to do inference at withouth
you there are 2 things I guess mostly irrelevant ones there you now demanding that I find sprout magic methods for overloading of the logical and more than that of known by encouraging people not to try and overload operators gender 2 and 2 I don't leave us hanging how do you fix that those young errors because a backslash above lemma of node you use a different way of expressing a 7 and uh use quite cities quoted strings then there automatically strings but there are other pitfalls and again switching to another full express yourself and you will see memories that are there but if you added quotes around both of them yes called problems and beyond problem that would steps so the gamma is a superset Jason so you can use produce strings and curly brackets and square brackets the lists and of the above the damn well in fact when you use the DocuComp balls corporation you almost at the end of the it was important just you feel of along the question was how do you feel the Python compared to other languages like Rubio hospital in order to you easier I don't know about Haskell but seen some horrible stuff well so it turns out that lots of the DSL moment cells in Ruby are written by monkey patching the object the object class which the rule of law objects for the you know the object hierarchy and adding methods that and then they just like available whenever you want about uh monkey patching both object seems like a terrible idea but it's highly encouraged to really effective syntax and we like just open another class and stuff stuff into it and that Somerset with which we model did you use for your ideas on the act so which just of the ones you again so that was just my own parser written line 1 with the thinking so circuitry because runs to quite localized the important thing for that is that uh so the indentation is important and it needs to maintain like the state of indentation so each token so it is not just a line line is um as a tokenizer which turns a line into the amount of indentation has and the 1 on 1 of the regular expression patterns that matches the token type and then the value of that taken that feeds into a finite state machine so this is upon question we can we see you again soon see the demonstration of the DSL was considered a wonderful thing is you don't and what the
somewhere in the middle we can we have a list like Newton's please act what I think this is the
introduction school in the middle the other is missing neutral notice that the same music and when I can right say I'd like source would back this also dialog he's telling the story of the well because the work very suggestions the things to do kind of think of the and no 1 can be so severe the emissions of the new millenium the people in the form of the this is all you do is you know a couple couple of a catalyst by this right is looking at simulation in which the Paris the and you know how all the we can play through the whole thing here that it would together takes probably about half an hour and a half so going to that's a thing exists scenery along but I think it's interesting to
consider how many lines in itself instead so there's so
it's like this and that would give too much of the fossil away the last 11 hundred and in many ways and sometimes the 3 thousand this some the more the DSL uh so if plan the government has a positive passes the entire script for the game at the beginning and I think you probably will say if it was a one-liner runs into a problem of the problem which we get if he seconds for last than question if you want to show more interest than the code because of this the the system of allowing tonight and the other findings for actions that can happen in the game so in this exact same define a script that like top level and it displays of each of these findings when you performers actions it will play a set of scripts which could kind of both this debate with other characters and so I can think contraria all kinds of operations can trigger the same giving you something for example that you might need to so a problem OK and I think you done in
Punkt
Freeware
Formale Sprache
Kartesische Koordinaten
Raum-Zeit
Computeranimation
Richtung
Metropolitan area network
Poisson-Klammer
Arithmetischer Ausdruck
Einheit <Mathematik>
Code
Mustersprache
Vorlesung/Konferenz
Domänenspezifische Programmiersprache
Kategorie <Mathematik>
Abfrage
Kontextbezogenes System
Dateiformat
Druckverlauf
Festspeicher
Ein-Ausgabe
Dateiformat
Programmierumgebung
Normalspannung
Zeichenkette
Tabelle <Informatik>
Fehlermeldung
Nebenbedingung
Maschinencode
Mathematisierung
Fächer <Mathematik>
Implementierung
Code
Spieltheorie
Programmbibliothek
Vererbungshierarchie
Softwareentwickler
Datenstruktur
Optimierung
Implementierung
Gammafunktion
Relationale Datenbank
Physikalisches System
Domänenspezifische Programmiersprache
Quick-Sort
Regulärer Ausdruck
Objekt <Kategorie>
Zeichenkette
Bildschirmmaske
Quadratzahl
Differenzkern
Resultante
Punkt
Formale Sprache
Abstraktionsebene
Extrempunkt
Login
Raum-Zeit
Computeranimation
Formale Semantik
Metropolitan area network
Arithmetischer Ausdruck
Code
Randomisierung
Vorlesung/Konferenz
Elektronischer Programmführer
Phasenumwandlung
Domänenspezifische Programmiersprache
Lineares Funktional
Digitalfilter
Knoten <Statik>
Ereignishorizont
Dienst <Informatik>
Benutzerschnittstellenverwaltungssystem
Festspeicher
Datenerfassung
Dateiformat
Instantiierung
Fächer <Mathematik>
Klasse <Mathematik>
Mathematisierung
Implementierung
Zahlenbereich
E-Mail
Code
Programmcode
Datentyp
Vererbungshierarchie
Datenstruktur
Konfigurationsraum
Hilfesystem
Meta-Tag
Drucksondierung
Videospiel
Diskretes System
Varianz
Parser
Elektronische Publikation
Domänenspezifische Programmiersprache
Programmfehler
Portscanner
Regulärer Ausdruck
Objekt <Kategorie>
Injektivität
Mereologie
Hypermedia
Faktor <Algebra>
Resultante
Bit
Punkt
Selbstrepräsentation
Parser
Computeranimation
Formale Semantik
Metropolitan area network
Poisson-Klammer
Arithmetischer Ausdruck
Datenmanagement
Minimum
Mixed Reality
Vorlesung/Konferenz
Große Vereinheitlichung
Gerade
Verschiebungsoperator
Differenzenrechnung
Softwaretest
Schnelltaste
Nichtlinearer Operator
Lineares Funktional
Dicke
Datentyp
Syntaxbaum
Gruppe <Mathematik>
Kategorie <Mathematik>
Systemaufruf
Ausnahmebehandlung
Ähnlichkeitsgeometrie
Digitalfilter
Kontextbezogenes System
Rechenschieber
Verbandstheorie
Rechter Winkel
Konditionszahl
Datenerfassung
Server
Reelle Zahl
Ext-Funktor
Tabelle <Informatik>
Aggregatzustand
Instantiierung
Overloading <Informatik>
Subtraktion
Abstrakter Syntaxbaum
Quader
Klasse <Mathematik>
Zahlenbereich
Implementierung
Zellularer Automat
Ikosaeder
Nichtlinearer Operator
Term
Code
Datensatz
Lesezeichen <Internet>
Datentyp
Vererbungshierarchie
Programmbibliothek
Inverser Limes
Thread
Inhalt <Mathematik>
Softwareentwickler
Datenstruktur
Gammafunktion
Schreib-Lese-Kopf
Soundverarbeitung
Matching <Graphentheorie>
Relativitätstheorie
Mailing-Liste
Schlussregel
Paarvergleich
National Institute of Standards and Technology
Domänenspezifische Programmiersprache
Parser
Ausgleichsrechnung
Modul
Quick-Sort
Objekt <Kategorie>
Bellmansches Optimalitätsprinzip
Bildschirmmaske
Quadratzahl
Metaprogrammierung
Iteration
Parser
Oval
Computeranimation
Streaming <Kommunikationstechnik>
Metadaten
Poisson-Klammer
Softwaretest
Code
Tangente <Mathematik>
Gerade
Auswahlaxiom
Domänenspezifische Programmiersprache
Softwaretest
Befehl <Informatik>
Sichtenkonzept
Kategorie <Mathematik>
Singularität <Mathematik>
Stichprobe
Strömungsrichtung
Bitrate
Rohdaten
Forcing
Rechter Winkel
Fehlermeldung
Zeichenkette
Tabelle <Informatik>
Selbst organisierendes System
Klasse <Mathematik>
Ikosaeder
Überlagerung <Mathematik>
Loop
Hauptideal
Lesezeichen <Internet>
Iteration
Reelle Zahl
Perspektive
Programmbibliothek
Elastische Deformation
Datenstruktur
Konfigurationsraum
Ideal <Mathematik>
Pixel
Protokoll <Datenverarbeitungssystem>
Parser
Domänenspezifische Programmiersprache
Menge
Data Dictionary
Programmfehler
Portscanner
Zeichenkette
Bildschirmmaske
Personal Area Network
Programmiergerät
Bit
Punkt
Web log
Formale Sprache
Fortsetzung <Mathematik>
NP-hartes Problem
Element <Mathematik>
Kernel <Informatik>
Metropolitan area network
Arithmetischer Ausdruck
Einheit <Mathematik>
Existenzsatz
Byte-Code
Funktionalanalysis
Nichtlinearer Operator
Softwareentwickler
Abstraktionsebene
Template
Abfrage
Quellcode
Ein-Ausgabe
Dateiformat
Arithmetischer Ausdruck
Arithmetisches Mittel
Teilmenge
Texteditor
Datenfeld
Twitter <Softwareplattform>
Dateiformat
Message-Passing
Zellularer Automat
Transformation <Mathematik>
Code
RFID
Zusammenhängender Graph
Hilfesystem
Leistungsbewertung
Physikalischer Effekt
Mailing-Liste
Ausgleichsrechnung
Quick-Sort
Flächeninhalt
Last
Mereologie
Serielle Schnittstelle
Freeware
Formale Grammatik
PASS <Programm>
Parser
Oval
Aggregatzustand
Extrempunkt
Raum-Zeit
Computeranimation
Netzwerktopologie
Deskriptive Statistik
Poisson-Klammer
Dämpfung
Algorithmus
Vorzeichen <Mathematik>
Faktor <Algebra>
Radikal <Mathematik>
Skript <Programm>
Vorlesung/Konferenz
Gerade
Auswahlaxiom
Phasenumwandlung
Domänenspezifische Programmiersprache
Softwaretest
Managementinformationssystem
Gebäude <Mathematik>
Singularität <Mathematik>
Ausnahmebehandlung
Kontextbezogenes System
Rechnen
Biprodukt
Token-Ring
Rechter Winkel
Bildschirmsymbol
Ordnung <Mathematik>
Zeichenkette
Tabelle <Informatik>
Lesen <Datenverarbeitung>
Instantiierung
Folge <Mathematik>
Subtraktion
Wellenpaket
Zustandsmaschine
Klasse <Mathematik>
Regulärer Ausdruck
Ikosaeder
Demoszene <Programmierung>
Loop
Spieltheorie
Datentyp
Programmbibliothek
Generator <Informatik>
Datenstruktur
Analysis
Symboltabelle
Parser
Modul
Zeichenkette
Portscanner
Wort <Informatik>
Rekursive Funktion
Compiler
Bit
Punkt
Formale Sprache
Familie <Mathematik>
Gradient
Sondierung
Komplex <Algebra>
Analysis
Metropolitan area network
Arithmetischer Ausdruck
Regulärer Graph
Große Vereinheitlichung
Informationssystem
Funktion <Mathematik>
Nichtlinearer Operator
Physikalischer Effekt
p-Block
Ein-Ausgabe
Arithmetisches Mittel
Funktion <Mathematik>
Garbentheorie
Message-Passing
Aggregatzustand
Standardabweichung
Ortsoperator
Gruppenoperation
Implementierung
Nichtlinearer Operator
Term
Abenteuerspiel
Code
Physikalische Theorie
Unendlichkeit
Multiplikation
Datensatz
Arbeitsplatzcomputer
Schreib-Lese-Kopf
Leistung <Physik>
Assoziativgesetz
Physikalischer Effekt
Matching <Graphentheorie>
Mathematik
Token-Ring
Quick-Sort
Gerade
Keller <Informatik>
Regulärer Ausdruck
Mereologie
Spiegelung <Mathematik>
Formale Grammatik
Schreiben <Datenverarbeitung>
Parser
Extrempunkt
Computeranimation
Formale Semantik
Poisson-Klammer
Last
Interleaving
Algorithmus
Standardabweichung
Abstrakter Syntaxbaum
Vorlesung/Konferenz
Gerade
Domänenspezifische Programmiersprache
Metropolitan area network
Verschiebungsoperator
Softwaretest
Namensraum
Datentyp
Ausnahmebehandlung
Knoten <Statik>
Biprodukt
Gruppenoperation
Konstante
Backtracking
Konzentrizität
Menge
Rechter Winkel
Zeichenkette
Fehlermeldung
Instantiierung
Geschwindigkeit
Subtraktion
Wellenpaket
Abstrakter Syntaxbaum
Fächer <Mathematik>
Klasse <Mathematik>
IRIS-T
Regulärer Ausdruck
Überlagerung <Mathematik>
Mailing-Liste
Datentyp
Programmbibliothek
Datenstruktur
Automorphismus
Varianz
Schlussregel
Parser
Domänenspezifische Programmiersprache
Modul
Zeichenkette
Kantenfärbung
Eigentliche Abbildung
Sigma-Algebra
Resultante
Chipkarte
Bit
Texteditor
Inferenz <Künstliche Intelligenz>
Formale Sprache
Fortsetzung <Mathematik>
Eins
Metropolitan area network
Arithmetischer Ausdruck
Lineares Funktional
Nichtlinearer Operator
Konstruktor <Informatik>
Syntaxbaum
Klassische Physik
Programmierumgebung
Quellcode
Ein-Ausgabe
Variable
Diskrete-Elemente-Methode
Verschlingung
Semantic Web
Aggregatzustand
Gruppenoperation
Schaltnetz
Implementierung
Zahlenbereich
Zellularer Automat
Nichtlinearer Operator
Code
Gravitationsgesetz
Hilfesystem
Gammafunktion
Matching <Graphentheorie>
Linienelement
Rekursiver Abstieg
Kanalkapazität
Mailing-Liste
Routing
Token-Ring
Paarvergleich
Quick-Sort
Regulärer Ausdruck
Mapping <Computergraphik>
Objekt <Kategorie>
Quadratzahl
Uniforme Struktur
Flächeninhalt
Mereologie
Ruhmasse
Informationssystem
Vektorpotenzial
Maschinencode
Overloading <Informatik>
Momentenproblem
Zustandsmaschine
Inferenz <Künstliche Intelligenz>
Wasserdampftafel
Formale Sprache
Klasse <Mathematik>
Zellularer Automat
Hierarchische Struktur
Fortsetzung <Mathematik>
Gesetz <Physik>
Code
Computeranimation
Eins
Arithmetischer Ausdruck
Poisson-Klammer
Knotenmenge
Code
Datentyp
Mustersprache
Lemma <Logik>
Softwareentwickler
Gerade
Softwareentwickler
Schlussregel
Token-Ring
Parser
Domänenspezifische Programmiersprache
Programmfehler
Regulärer Ausdruck
Teilmenge
Objekt <Kategorie>
Quadratzahl
Geschlecht <Mathematik>
Softwareschwachstelle
Festspeicher
Projektive Ebene
Gammafunktion
Ordnung <Mathematik>
Aggregatzustand
Zeichenkette
Fehlermeldung
Binärdaten
Mailing-Liste
Quellcode
Extrempunkt
Computeranimation
Metropolitan area network
Bildschirmmaske
Diskrete-Elemente-Methode
Rechter Winkel
Thetafunktion
Gruppe <Mathematik>
Simulation
Gammafunktion
Binärdaten
Nichtlinearer Operator
Dualitätstheorie
Datensichtgerät
Gruppenoperation
Zwei
Automatische Handlungsplanung
Physikalisches System
Extrempunkt
Domänenspezifische Programmiersprache
Computeranimation
Übergang
Metropolitan area network
Diskrete-Elemente-Methode
Menge
Spieltheorie
Bus <Informatik>
Skript <Programm>
Bildschirmsymbol
Gerade

Metadaten

Formale Metadaten

Titel Writing Domain Specific Languages with Python
Serientitel EuroPython 2015
Teil 171
Anzahl der Teile 173
Autor Pope, Daniel
Lizenz CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben
DOI 10.5446/20100
Herausgeber EuroPython
Erscheinungsjahr 2015
Sprache Englisch
Produktionsort Bilbao, Euskadi, Spain

Technische Metadaten

Dauer 58:35

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Daniel Pope - Writing Domain Specific Languages with Python Python is an expressive general purpose programming language. Its syntax provides many ways to represent structure and minimise code repetition and boilerplate. But Python not always expressible enough. Perhaps when you've built a complicated enough system with hard-to-express inter-relationships, the code required to construct or operate on it can become complicated, repetitive and unreadable. Or perhaps you have users unfamiliar with Python who need to understand or edit a system. In cases like these, stepping beyond the syntax and semantics of basic Python can be an advantage. Daniel will describe various ways you can implement your own Domain Specific Languages, languages perhaps completely unlike Python that can succinctly describe more complicated Python systems. This talk will cover: * What and why of DSLs * Metaprogramming tricks * Writing simple parsers * The libraries PLY and PyParsing * Building tooling around your new DSLs
Schlagwörter EuroPython Conference
EP 2015
EuroPython 2015

Ähnliche Filme

Loading...