Declarative Thinking and Programming
Formal Metadata
Title 
Declarative Thinking and Programming

Title of Series  
Author 

License 
CC Attribution  NonCommercial  ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and noncommercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license. 
Identifiers 

Publisher 

Release Date 
2017

Language 
English

Content Metadata
Subject Area  
Abstract 
Declarative Thinking and Programming [EuroPython 2017  Talk  20170713  PyCharm Room] [Rimini, Italy] Declarative Programming is a programming paradigm that focuses on describing what should be computed in a problem domain without describing how it should be done. The talk starts by explaining differences between a declarative and imperative approach with the help of examples from everyday life. Having established a clear notion of declarative programming as well as pointed out some advantages, we transfer these concepts to programming in general. For example, the usage of control flow statements like loops overdetermine the order of computation which impedes scalable execution as well as it often violates the single level of abstraction principle. Following the theoretical part of the talk, some practical examples are given how declarative programming can be applied easily within Python. This comprises the advantages and disadvantages of using a configuration file, e.g. config.yaml, versus a Python configuration module, e.g. setup.py. Furthermore, the benefits of avoiding statements of control flow with the help of list and dictionary comprehensions as well as sets are demonstrated. The talk is concluded by a short, highlevel excursion to a logistic programming language, namely PyDatalog in order to build the bridge between logistic and declarative programming. This is accomplished by showing how a mathematical crossword can be easily solved with the help of declarative and logistic programming

00:00
Mobile Web
Word
Focus (optics)
Digital signal processing
Multiplication sign
Software
Projective plane
Moment <Mathematik>
Declarative programming
Computer programming
Communications protocol
Field (computer science)
00:56
Boss Corporation
Computer font
Email
Multiplication sign
Workstation <Musikinstrument>
Planning
Bit
Mathematics
Mathematics
Natural number
Different (Kate Ryan album)
Right angle
Figurate number
Address space
Task (computing)
03:09
Building
State of matter
Block (periodic table)
Forcing (mathematics)
Weight
Bit
Instance (computer science)
Mereology
Complete metric space
Digital photography
Different (Kate Ryan album)
Personal digital assistant
Query language
Cuboid
Energy level
Right angle
Energy level
Quicksort
Abstraction
Task (computing)
Abstraction
Task (computing)
05:47
State observer
Group action
Equals sign
Decimal
Set (mathematics)
Parameter (computer programming)
Mereology
Data dictionary
Computer programming
Order of magnitude
Computational complexity theory
Computer configuration
Set (mathematics)
Square number
Functional programming
Imperative programming
Abstraction
Scripting language
Programming language
Computer font
Mapping
Computer file
Electronic mailing list
Parallel port
Range (statistics)
Bit
Instance (computer science)
Markup language
Complete metric space
Declarative programming
Flow separation
Degree (graph theory)
Arithmetic mean
Computer configuration
Configuration space
Functional programming
Right angle
Data type
Writing
Resultant
Arc (geometry)
Momentum
Programming language
Computer file
Transformation (genetics)
Markup language
Patch (Unix)
Online help
Electronic mailing list
Modulare Programmierung
Code
Declarative programming
Number
Power (physics)
Wave packet
Pie chart
Goodness of fit
Structured programming
Flow separation
Software design pattern
Energy level
Task (computing)
Default (computer science)
Default (computer science)
Matching (graph theory)
Physical law
Projective plane
Equivalence relation
Computational complexity theory
Singleprecision floatingpoint format
Mathematics
Integrated development environment
Personal digital assistant
Query language
Function (mathematics)
Complex system
Abstraction
Library (computing)
15:22
State of matter
Multiplication sign
Source code
Set (mathematics)
Prime number
Subset
Query language
Programmanalyse
Declarative programming
Different (Kate Ryan album)
Personal digital assistant
Square number
Information
Series (mathematics)
Recursion
Information security
Computer font
Mapping
Digitizing
Bit
Instance (computer science)
Declarative programming
Information extraction
Prime ideal
Computer science
Website
Pattern language
Information security
Resultant
Data integrity
Numerical digit
Disintegration
Online help
Mathematical analysis
Modulare Programmierung
Rule of inference
Number
Root
Operator (mathematics)
Logic programming
Energy level
Integer
Divisor
Lie group
Metropolitan area network
Condition number
Capability Maturity Model
Rule of inference
Standard deviation
Equals sign
Physical law
Computer network
Database
Line (geometry)
Mathematics
Subset
Number
Software
Query language
Predicate (grammar)
Personal digital assistant
Universe (mathematics)
Vertical direction
Square number
Abstraction
Library (computing)
21:11
Point (geometry)
Run time (program lifecycle phase)
Numerical digit
Multiplication sign
Electronic mailing list
Mereology
Rule of inference
Graph coloring
Hand fan
Advanced Boolean Expression Language
Number
Mathematics
Semiconductor memory
Personal digital assistant
Energy level
Error message
Abstraction
Partition (number theory)
Condition number
22:43
Web crawler
Beta function
State of matter
Code
Multiplication sign
Source code
Execution unit
Combinational logic
Prime number
Mereology
Computer programming
Declarative programming
Hypermedia
Electronic meeting system
Matrix (mathematics)
Cuboid
Arrow of time
Abstraction
Algebra
Error message
Scalable Coherent Interface
Constraint (mathematics)
Theory of relativity
Digitizing
Stress (mechanics)
Instance (computer science)
Declarative programming
Flow separation
Metric tensor
Message passing
Arithmetic mean
Prime ideal
Right angle
Dataflow
Link (knot theory)
Letterpress printing
Online help
Rule of inference
Declarative programming
Number
Latent heat
Pi
Flow separation
Logic programming
Operating system
Energy level
System programming
Condition number
Window
Module (mathematics)
Domain name
Dialect
Multiplication
Expert system
Code
Line (geometry)
Linear algebra
Cartesian coordinate system
Limit (category theory)
Computer programming
Frame problem
Mathematics
Error message
Software
Personal digital assistant
Moment <Mathematik>
Abstraction
28:06
Operator (mathematics)
Multiplication sign
Expert system
Control flow
Mereology
Neuroinformatik
Subset
00:05
hello everyone and thanks and a few words about me um synonymous million him and
00:13
the Yemen Data Scientist at in makes innovates is an IT protocols in Germany and uh yeah its focus is on digital transformation so we have projects in the field of Big Data Engineering and data science but also for web mobile and so on in my spare time I like to contribute to the PPI data stack and I also integrate of pi scaffolding and really found it cool that this proposed a fold was actually mentioned a few times in some of the torques so um short hand up has uh if you have has used place careful just to have the only if you have OK but intellectual talk to so and outline its
00:59
of 1st give a small motivation and I'm about decorative thinking about the concept behind this then and show some examples in Python how you can actually apply this way of thinking and in the end uh I conclude with a little math natural so 1 likes riddles hopefully
01:18
so let's assume you just moved to another city and after all the moving business you want to to uh make little housewarming party and friends so let's say you have 2 friends to best friends uh Alison vault and you know that Alice she's really good with attack and so on so you just send an email with the new actress saying OK please come over a wanna make a little housewarming party and be there at that time at that date and you know she will be a right you will use Google Maps or whatever to figure out how to get to your to you new address and to be on time but that they will will pop it's a bit it's a different kind of stories so he's uh as a valley able to to write email so you send him not only from an email with your and with the new address and place and date you also make the whole trip planning for him at say you say OK it would be cool the best if you take that trained from your city to to to to my new city and then leave the main station take this tram and the that boss and finally you will arrive on time so this example we directly see so we interact it's completely differently with Alice and Bob so that Alice we more or less stated what we 1 of her pretty clear it would be 1 over and the ball we not only told him that he should be at the housewarming party we also define how he should take actually accomplish the task but it's not always that easy to tell the how from the what and the how clearly is more in in protiv way and what is more declarative way so to see this some
03:11
let's take a look again what our actual task force in this case in this example so we were we our task was to do to plan like housewarming dinner so we have different tasks like to clean up the mess fightin easy recipe you to cook for your friends put the last remaining boxes in the basement by maybe more beer and 1 of these only 1 part of this was to in white and your friends so this is the kind of abstraction layer we are dealing with if you want to to invite you to to make it a housewarming party and we see that if we have given this abstraction layer so with Alice the states actually and that abstraction layer and the Bob it's a complete different kind of story went down to actually planning a trip for him so clearly be left and this um this kind of layer also uh making a and of planning a housewarming party
04:12
and this is an important aspect actually here when talking about declarative thinking that 1 has to think about the level of extraction and where our tasks extra side and we know this from many their money other examples like if you wanna take a picture for instance and then um you're kind of abstraction layer is that you you know how to to to handle a camera and you don't care about all that you never cleans them of how actually the the photo is created and to be a bit more techie the um and you more like techrelated conference right so maybe you remember like 5 or 6 years ago there was this big about had met no 1 was sort like hyped about this year it's so cool it's just too easy building blocks MapReduce and you can do basically everything with it but then later people realize that weights so most of the tasks ixion leave more like query tasks so more like SQL a secret related tasks so having realized as people realize OK so but to do something like like like queries to do this in in in MapReduce it's a lot of work and it's actually not stating the problem anymore on that on that level so people came up things like Hive and Pig and and you name it so this is also an example of that the layer of abstraction is really important for this the so again to some
05:50
this little bit op um so I hope I can wait the the idea behind this imperative and declarative thinking in a way that um imperative is small like specializing on the Howell and uh and you can see this if you always specify things like when we talk to Bob we said OK you should end at the train and stuff like this which is completely unrelated to the actual task of giving it a housewarming party and detailed instructions and declarative as the more the what and this is also related if we think about programming and um also power design patterns that was 1 talk about Python design patterns which are fun really cool and and 1 of them is separation of concerns and the single level of abstraction and those also he also highly connected to declarative thinking and programming meaning that you wanna keep you when a module ISOcode you wanna make sure that you all know that 1 part of the court is actually dealing only with 1 concern and when you calling when you when you write a function that you stay with incited function on the same level of abstraction of going up and down all the layers but this of course is as it depends on the level of abstraction you're dealing with right now so 1 of them 1 node about abstraction just the so called uh lawfully abstractions by people Spolsky so this is actually more like observation so Americans tend to to call everthing the law but this is more like an observation so he said that all Montreuil abstractions to some degree our leaky and and so from my experience is absolutely true so um I mean many of you have to use SQL and it may have also what you want SQL and you might have been realized it is to start the slope very you can sometimes do a lot of equivalent transformation of that query and you end up having the same result but the the actual queries some magnitude faster than before and this is what's clearly leaky abstraction because actually SQL should kind of and covered the complexity below and just so that you can just state what you want and what you want have but sometimes is then better to know what's going on the need to to optimize something and this is where even a language as old and as such groups as SQL so good for this 1 specific task is um is leaky so this is something 1 should be aware of when talking about and yeah when speaking about uh abstractions so am having now at the idea of all of the creative thinking I wanna give some small and you have some small examples in Python some uh quite easy example so let's say our task is to get the list of script numbers from 1 to 10 and iterative and maybe something is someone's coming from CO Chabot maybe do is you would create an empty list and he would just say just move over at can't uh and the square numbers of 1 to 10 and the this clearly is imperative because we and we're I mean our task is just saying we want some some kind of lists and by here we are always specifying beginning in all the of how we wanna do it and uh um more decorative approach which would be just to use this comprehension as usually you all know and love of a list comprehension because they're so easy to use their much more readable and actually staying more on the level of our cast so we're basically transforming the task of getting a list of squared numbers from 1 to 10 directly to come to the syntax and this has many advantages so what 1 could think that this could also be a paralyzed I mean it's not paralyze the right now in the Python but it mean it could be possibly so someone could say OK we change the code in in the back so under the this abstraction layer and would in parallel so this is an was an advantage that could be done besides being more readable and an example would be let's say you wanna write some kind of these patches so given some document you wanna call it different functions on some value and an impetus some way of doing this would be just like you check if arc this option a then you call function any uh with the value and so on and if it's not the case you call default and also here we're giving an order and be going down to them at a much lower level actually so what would be better is to use a dictionary here so some way more complex data type and that data type a teacher is actually more like a mathematical mapping so you met 1 thing to another and I mean this is what we wanted to if we wanna ride dispatcher like this so basically with the help of a dictionary can just say OK we met um 1 argument to certain functions and then we can just use the gat uh method of the dictionary to do this so this would be way more declarative and even if we had a lot of options this would even be faster because in the back uh and it's implemented this some kind of a hashmap I've got them so maybe a little bit more complicated um example let's say he wrote a paper your Peres called and you wanna see if someone stole from your paper so you want to check your pay by against a paper and and the question is how would you do this so of course they're really naive approach would be you take each sentence from your paper p and take all the sentences from paper and a and see if their equivalent so this would be in bed with a midget grid like this with the complexity of all n squared um but to the power of 2 so and quiet yeah quite a huge complexity and if you think a little bit more about a problem see that actually a good abstraction layer could be set theory so given that Python has also set data types you could then think of a better way to which we formulate your problem if you just wanna compare the sentences and just put all the sentences of work 80 and Burke into dictionaries and then finding and the the intersection is just the simplest this and this is also nice example where the light if you think a little bit more in advance about what kind of problem you're dealing with and if you find the right abstraction layer for it then um it's actually much easier to write codes and um yeah it's actually uh way more readable and in this case also way faster because also this will use some hashing matching so another thing you
13:25
ever experience quite often in the in the Python ecosystem our configuration files mean it was also mention a thing in the in the 1st lightning talk and another talks that in Python a lot of libraries like things and also set up I a set of tools they use and thing the use pied modules for configuration and this is quite bad I think because a Python module it's kind of hard to check right what happens in the everything can happen it's it's a tool in complete weighted to something as simple as a configuration and um for instance in ITU so it was also mentioned in in in in the PPI chart tutorial of pi jump talk that an IDE like like pie chart could not take what kind of requirements you put into set up i if you put it get directly it's much easier to pass something like a requirements TEXT TEXT so them thing to take away his here is that it's way easier to um to useful configuration and way you wait declarative um approach is to use something like yum or some some mark up language or Tolman for instance uh um that is less powerful so you have to kind of think more about the actual problem how you wanna set up the structure of the configuration but you earn a lot but it is easy to check its and uh and yeah it's it's way better and this was also 1 of the reasons why I chose um I use for my projects PPR which in pi skinfold because there you can instead of using a set up I you can see use as set up . e f xi 2 Dec should configuration in a more in a more uhm declarative way the OK so coming from some like really
15:22
simple and pattern example of some let's talk about a little math brittle so the you might
15:32
know the German newspaper the sites all the time and they have this little map adults called uh localized that's which is some kind of mature work between logic and law the lie the global a some kind of man made that attracts you and and kills you so this is kind of results you start and then you can't you can't stop until you finally I guess all of them have also here will give up and and yet so this is a riddle and and fly crossroads but these numbers and we see here that the that for instance here to make sure that the digits somewhat of a of a source on c and that c is a prime number and so on and 1 of my friends sent me um this and are showed me this and said OK how would you solve this with the help of a PC all the help of Python and this got me thinking and um and then I've remembered like yeah back in the days at universities someone mentioned the portal and declarative programming and uh yeah logical programming so if OK maybe I looked this up again and see and what I can do and see if I can solve this staying understanding level off um of abstractions that are basically just wanna state what's in the riddle and in the end come up with a solution so how many of you know portal maybe this occured quite a quite a lot so a lot is actually just a subset of Pollock uh and um it's standards its use your so it is used to be able to reason more about DataLock it's a declarative and logical programming language it's used to query and deductive databases so databases where you just specify facts and certain rules and then you can uh um ask this kind of knowledge database um for certain facts if it can be deduced from whatever you've given before it actually has quite many use cases so um Microsoft history of research in this regarding security a network security that's in some kind of fiable rules for this then it's also used for data integration Information extraction networking program analysis and of course yeah you can do it for you also use it for for silly riddles and um there's a nice um python library actually called pi datalog where you can just use it and run it uh a Python module and I just wanna like so little bit how 1 would go about this and how it is possible to to stay like really at the level off and this the idea of the uh conditions in the in the real so we saw that 1 of those conditions lost that's the that is some numbers r squared numbers so how would you how would you specify this with the help of PPI data lock so you would just say that the that squared x is uh so X squared if you can take the square root of it and if it's still an integer and um you read the leftmost um it's smaller than equal sign justice and if so the operators overloaded and it's best to just read that name if and you can see it's just like you're just stating it so and this is squared is not some kind of predicate for X and um it's some kind of of set of some you you buy this you defining some set but this is all you need to know and um the same goes for divisible if X is divisible by Y and we can also take only for the remainder it's visible if you do this and and then it's equal to 0 so maybe something more interesting how would you check if a number is prime and we know that too was a prime number so the plus operator over let it be so the therefore for just check if or states that 2 is a prime number as a fact and for other numbers and 3 and 4 the other numbers the use of rule and stating more less that all numbers larger than 3 which are a OCR numbers some not so this is the to the so the curly 1 and not divisible by 2 and if just not any other factor or between 3 and and the square root of and this be defined by this by the 2nd last line um with some kind of recursion so in the last line we're basically saying so if y plus 2 were is smaller than um some some other provinces and square root and then we check all the numbers and in just a few lines we we can stay this lazy for all numbers and since most of our um conditions are defined and numbers and we actually looking for the digits we of course need to have some kind of mapping between digits and actual numbers and this is really easy so the digits of the number of 2 digits is just 10 times the 1st digits plus on and then just a 2nd and uh then we can just recursively defined this for um for all um digits for different number of digits here again so I'm sorry here
21:14
at that point we would be no able um this some more rules and so on to actually just write down 1 and condition after another off the of the riddle and here is the
21:30
part where the leaky abstraction uh bites us so and since what happens below is that uh mn what the uh and what pied elect dust it's uh and generates a whole list of solution can be that's and starts eliminating them and so on and uh and if you start with the wrong kind of uh and condition 1st and then it might happen that a list of candidates gets tool to a large at certain points and uh and then you can run out of uh in into out of memory error all our um into the Dorchester runtime is just a 2 long so this is done some kind of leaky because it's lead should play no role uh um if you wanna stay on this level so 1 thing to to deal with this is to define certain parts like this uh partitions this color petitions and um using this petitions in now in case of this riddle helps to keep the number of solutions slow at all time and then you just combine them so just to give an an example how this looks like for the upper left corner
22:45
so for the blue corner and this is to so large enough to be red so and the 1st line a little uh um larger so this policy and we wanted it to the prime numbers to 2 prime numbers for but sorry the 2 digits former prime numbers so in that pie dough paid out in PPI data lot we can just easily state that 8 2 ways of digit ranging from 1 to 10 because it's the start of a number so it can't be 0 and 210 or sorry 1 2 9 uh and not 0 to 9 and at the same goes for a 3 and then we'd be just can use um the prime check the prime rule that we have defined and and by this we can fall uh um yeah all conditions no or it'll be can just write them down and actually it's just like this you see here like each line in the box by half is uh more or less 1 of the conditions in and the riddle and it's really staying at this abstraction level is using node moves or always specification and it's just a declarative way off and specifying a real How would be Crary this now it's really it's easy it's just saying um just like print and the riddle and the riddle as a combination of all those 4 boxes which I have left out here but you can look this up those later show a link and you just say OK now and give me the uh the number of digits forward that fulfill all the constraints I have given you and uh then it will just come up with a solution and not only that the solutions so if there were more than 1 solution would print all solutions so we know now in not only 1 solution of also short it's a unique solution which also the really nice thing and uh yeah if you follow the the you well below then you can also get the whole source code and then I think this so for me this was a really nice experience to to work with a logical programming language and maybe some of you get interested in this and 1 were try the spider log out or it maybe portal dialect directly and uh yeah wanna conclude that there are
25:10
many other applications where actually declarative thinking and programming becomes much more important some of the torques also makes o o s was the most mentioned and this is 1 operation system where you have a decorative way of stating how you want your system to be it's not like aptget install and moving from 1 state another you have a decorative way and this time has gained quite some traction over the last year also and it's was at a frame metric tensor flow I mean they include just the the the layer submodule where you can know easily defined like in a curious style of way and your your network because if you wanna build a network if you on that limited layer units in interested in really all the details how you do the matrix multiplication so on but it has now provides like 2 layers like this layer submodule where can justify layers and of course also the lowlevel parts with um where you can do like almost everything that's possible with linear algebra and others young and as for instance also Luigi which is kind of a competitor to to air flow and it is also using a more and more declarative approach so to um to make a small summary of the talk so and what are the advantages of a decorative way of programming and of thinking because actually it's it's more like thinking so in in in most cases you can actually improved the readability of your of your coach your and your more like stating what you want of course and a keep it then to some other part of the code to do this it also reduces the number of arrows most of the time because I am if you for instance if you write some Korean ask well if you would do this really with the help of um some MapReduce or if you would tried to to mimic this uh the own code you would surely end up having a lot of error and strange corner cases in many cases is also increased performance because I am and mean as as you say you stay the same level of uh this at 1 level of abstraction and what happens the lower is then ends that can be programmed by some domain experts like uh yeah uh the I mean in case of SQL beta relational algebra and so on uh and um yet and you following the separation of concerns so that the big takeaway message here is and so it's kind of my definition for media how I think that the great to program instead that declarative programming for me means finding the right abstraction layer describing the problem and then using this clay layer so thanks for your
28:07
attention and you will come to your
28:10
questions if we have enough time Future of about 1 2 minutes noted that much so I would take 1 question so when is questions part of that saves us time OK that's what this yeah height you say that that that allow the the subset of problem there what is missing and he will yeah feature or and it's kind of those desert and all of this breaking operator where you can kind stop the computation and this is missing and think this is 1 of the most important thing which is missing in it in lock but I'm also another in an expert in this but don't thanks thanks alright that was is and so given and few and remember what you