Case study: Making use of functional programming techniques in .NET
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 170 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/50805 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
NDC Oslo 2014109 / 170
2
3
5
6
8
11
12
16
21
22
23
27
31
35
37
42
43
45
47
48
49
50
52
54
56
57
58
61
65
66
67
71
74
77
80
81
83
84
85
87
88
89
90
91
94
95
96
97
98
100
102
107
108
112
114
115
116
118
120
121
122
123
126
127
128
130
133
135
137
138
139
140
141
142
143
144
145
147
148
149
150
153
155
156
157
158
159
160
161
162
163
166
169
170
00:00
AbstractionString (computer science)Function (mathematics)Element (mathematics)Software testingSimulationControl flowError messageException handlingMetric systemChemical equationMach's principleDirected graphRange (statistics)Bit rateVideo game consoleCache (computing)Line (geometry)Local GroupSymbol tableGraph (mathematics)Range (statistics)Operator (mathematics)Multiplication signRight angleExtension (kinesiology)Parameter (computer programming)Graph (mathematics)Functional programmingElectronic mailing listMathematicsObject-oriented programmingCodeNumberMappingCodecRepetitionLine (geometry)System callVideo game consoleSoftware testingProduct (business)Metric systemError messageInterface (computing)Single-precision floating-point formatFactory (trading post)Ultraviolet photoelectron spectroscopyProgramming languageComputer programmingView (database)Insertion lossExpressionPhysical systemStandard deviationShift operatorTerm (mathematics)Group actionPoint (geometry)Cylinder (geometry)SequelDrop (liquid)Address spaceLevel (video gaming)Type theoryQuicksortVariable (mathematics)Statement (computer science)Doubling the cubeCartesian coordinate systemMereologyString (computer science)Density of statesSymbol tablePhysical lawElectronic signatureState of matterAlgebraic closureTriangleGame controllerResultantLetterpress printingWordSound effectoutputCASE <Informatik>Video gameGraph coloringWebsiteConnectivity (graph theory)Key (cryptography)Link (knot theory)Computer configurationException handlingArtistic renderingDemosceneFormal grammarNormal (geometry)Social classCore dumpVideoconferencingFunction (mathematics)Online helpSelectivity (electronic)Imperative programmingBranch (computer science)Revision controlData structureCellular automatonPattern languageProjective planePay televisionData storage deviceBroadcasting (networking)Problemorientierte ProgrammierspracheHypermediaBitSampling (statistics)Instance (computer science)NamespaceMultiplicationLibrary (computing)Grass (card game)Graph theoryKeyboard shortcutReal-time operating systemMixed realityScaling (geometry)Moment (mathematics)Transformation (genetics)Front and back endsDataflowGreatest elementForm (programming)StatisticsSubsetGraph (mathematics)Row (database)Configuration spaceTotal S.A.Different (Kate Ryan album)Client (computing)Dependent and independent variablesObservational studyFLOPSFunctional programmingDirected graphConstructor (object-oriented programming)2 (number)Position operatorWorkstation <Musikinstrument>OscillationShape (magazine)Computer clusterForcing (mathematics)Process (computing)DecimalComputer architectureDigital photographyPresentation of a groupWell-formed formulaForestProgrammer (hardware)Service (economics)Similarity (geometry)AdditionMacro (computer science)Frame problemLambda calculusNP-hardPhase transitionCuboidPlanningTrailEmailFault-tolerant systemSequenceTheoryStability theoryDialectNetwork topologyMechanism designLaceSpacetimePartial derivativeData recoveryLengthWeightField (computer science)BlogReading (process)Web 2.0ChainWindowServer (computing)Set (mathematics)GodOffice suiteBit rateMobile appBinary multiplierInteractive kioskSummierbarkeitInterrupt <Informatik>ParsingFile formatCache (computing)Disk read-and-write headData dictionaryCountingHierarchyIntegerReal numberStructural loadAdaptive behaviorSoftware frameworkGreen's functionCombinational logicComputer fileVolumenvisualisierungObject-oriented programmingRandom number generationThread (computing)Fluid staticsSimulationArrow of timeMessage passingRecursionElement (mathematics)Computer-assisted translationAsynchronous Transfer ModeAbstractionBuildingFacebookWritingSemiconductor memoryCasting (performing arts)Software developerFrequencyGoodness of fitRandomizationTupleMeasurementFlow separationComputer animation
Transcript: English(auto-generated)
00:03
Okay. Hi. My name is Hans Christian Holm and I'll be speaking about functional programming techniques in .NET. This is a case study, which means that I've gone through the code and a real project and I've taken out some interesting pieces and made some simplifications
00:25
and adaptions so I can show them here. And the subject is a site called yr.no or just yr or in Norwegian, ir. That's the weather site. Has anyone here not been to this site?
00:42
Oh, really? Oh, really. Okay. So you know the site. That's a popular weather site. It looks like this. I'll just make a short description, go through what the weather site is. It's a cooperation project between the Norwegian Meteorological Institute, which
01:07
is called MET, and the Norwegian Broadcasting Corporation called NRK. And the project started in 2006 and we launched it in 2007. That was just before C sharp 3 came with link
01:23
and all the functional goodies. So we were able to use those techniques quite early in the project. The site is quite popular. It's at the most 5 million users per week.
01:40
That's okay. Consider that. 5 million people living in Norway. Serving about 20 million documents per day. That's HTML and XML. And peaked as the fourth biggest weather site in the world. There are among the top ten biggest weather sites. This is the only one that's not Anglo American. So it's okay. I represent MRK, which is responsible
02:07
for the presentation. So the architecture of the site is something like this. You have the MET supplying all the data through XML REST services. And MRK takes all that data
02:24
and does the presentation and makes it available for searching and all that. So you give out HTML to end users and also simplified versions of the XML to third-party users, which is quite popular. That's more than double the traffic of HTML. So if you see small apps
02:45
and kiosks and all that, that's the XML from this one usually. That's the simplified version. Functional programming. I assume that if you are a C sharp programmer, you have been doing at least a little bit of functional programming.
03:07
Maybe not without noticing. If you've been using link, you have been using functional programming. And maybe if you use the select statements for mapping things or filtering
03:21
out things with the worst function on an array and adding a small lambda, then that's functional programming. That's a tiny bit of functional programming. What I'm going to show is quite more advanced than that. That's functional programming opens up a wide
03:40
range of possibilities and you can build these very high abstractions on top of that. So I'm going to explain first what functional programming is all about. At the core, there's of course a function. That's the function. It's called a method in object-oriented programming.
04:01
We call it a function. That's an excellent piece of code that I've written myself. And I'm going to tell you why this is so great. First of all, there are no hidden dependencies here. Everything you do is dependent on the input. It's not fetching data from somewhere else in mysterious ways. Anything that comes out of it is a result of
04:27
what comes into it. And it works the other way around as well. It doesn't make any changes to anything outside. That's called a side effect. It doesn't have any side effects. So you can really rely on this. And it also means that it's thread-safe.
04:47
You can run this from many threads and it's not going to break. Also, it's isolated because it doesn't have any side effects. So it can be
05:03
made static. You still have to have a class to put it in. That's just the formality. You can use this as an isolated function. And you can take the function and do interesting things with it. So it's composable in completely new ways that you can imagine
05:21
doing with the traditional imperative programming. That's the non-functional programming. Also, it's small, which isn't necessary to do functional programming. But functional programming also often causes small functions. So you have very well-defined functions.
05:47
Easy to manipulate and compose. And this means that this is a pure function without any side effects that could adulterate this. We'll be using this in aggregating some data.
06:03
This is just a core function. It can be used for aggregating, say, a list of numbers so you can make a sum of them. If you try to do that in the traditional object-oriented way, I tried it, and it looked something like this. It's not...
06:24
That's the actual operation down there. It's the plus. So you have some interfaces, and this is the operation, and you have to use it somewhere, and you have to make this aggregator. It works. I tested it. It prints out 23.
06:42
And you could make some more things, factories and all that, and lock it, and testing. So I don't know. This is probably wrong in according to some standards. There are some problems with this. It's very verbose. Which you can see.
07:04
And it's not thread-safe. This thing here, it modifies this value. Which is up there. And this, yeah. And coupling settings go across each other. And composable, reusable, maybe you could make
07:26
composable aggregating. If you take the interface, you can reuse it. Well, let's go back to this one. Much more nice.
07:43
The definition of functional programming isn't very exact, but there's one thing you have to be able to, and that's be able to take a function and use it as a value. So functions are first-class members of the language, which means that it can be assigned to variables and assigned to an F. So the F is just like the add, and it can be invoked
08:05
and called. And it works the same way. This works. So let's get some numbers to work on. Here are some numbers. And you're making other functions. And it's even smaller. That's the multiplication. Just call it G. It could
08:24
be called Mo. Called G. F and G. So we can aggregate. This makes the sum. And this is the product of those numbers. That's the output. The aggregate, this thing here, that's a static method which takes two arguments.
08:48
The first one are the numbers to aggregate. And the next one is a function which looks like this. F and G. And it takes them and aggregates them. It takes the first two.
09:00
Plus the next one. Which is quite nice. It looks nice. But if you were to do many of these, many types of aggregating functions, we will be repeating ourselves down here. So we can do something about that. We can take that numbers aggregate function
09:25
and assign it to a variable. This is quite interesting that this works. Because this is an example of partial application. You have the function with two arguments. And now it's come a function with one argument. Because we took the numbers.
09:42
And we called it with that. So now it remembers the numbers. And you can input another function. That's a higher order function. A function that takes a function as input. It can also return, but it returns an int. So now we can take our operations and they are values as normal. And we can make an array.
10:05
And we can do this. What's this all about? If you're not used to functional programming, this may be very confusing. We take the operations and we do a select. So for each of those operations, we use the aggregate as a mapping function for this.
10:31
So we have 50 and 60. And that's the sum of the products. This is going to be very confusing. It's very simple. Just a few lines of code and you start
10:43
losing if you're not used to this. This can be easy if you look at it graphically. It's not difficult at all. We have on the left, we have a set two functions. And we have this function here. We feed those through those.
11:05
For each member here, we have a mapping to something there. So the plus applied to this gives 15. This multiplication applied gives 60. That's functional programming. Or that's just the beginning of functional programming.
11:25
So you should get your head around this and recognize these kinds of expressions to become a functional programmer. It's called select in link. That's a bit unusual.
11:44
Because in other languages, it's usually called map. But Microsoft, they decided to use names from SQL statements instead of functional programming. So it's called map. And the aggregate, that's the use of many other languages. So to our first example, which is
12:05
a slightly unorthodox way of rendering HTML. It can be done with functional programming in a functional language called VB.net. So first to a language called C Omega.
12:22
That was an experimental language by Microsoft Research. It's a kind of predecessor to C sharp 3. And it introduced all those concepts we know as link. It has some more extras, but mostly it was link. And it had this thing here called XML literals.
12:43
I don't know exactly what it's doing because it's returning an object with a type. At least it's XML literals. That didn't make it to C sharp because the team thought it was too specific. And that's probably true. I don't think people want XML in C sharp
13:03
now. But it's been used in other languages, such as Gala. They have these kinds of expressions to build XML structures. And Facebook just last year, they released Facebook React to the JavaScript library with an extension to JavaScript called JSX, which also included
13:24
this kind of syntax. And there are languages using this in as a part of the language, as a DSL, internal DSL. But it did make it into VB. So it looks like this. And this is also a pure function. It's quite easy. I used lower case for the keywords.
13:49
That looks so much better. That's a function taking an integer and returning an excellent. It's a pure function and it's quite obvious what it does.
14:00
So you have these inline values. You can do this in C sharp as well. And it also supports comprehensions. So you can put a link statement inside this. And the input is now an array of integers. So here's something to build an unordered list.
14:26
In HTML. It looks the same. There's a call to the view item up there. So if you use it just like this, input and you get this as output.
14:44
Let's do some refactoring of this. What's happening behind the scenes here is that our last link statement gets compared to something resembling this, which shows us that
15:01
you're not actually calling the view item inside the statement. You're giving it the address, a reference to this function, and it's evaluated lazily, which means that the select operation doesn't apply at once. It does later. But when you
15:22
try to build this unordered list, it gets called and then it's applied. This also shows that you could replace the call to view item with something given as an argument.
15:42
That's a function which has an integer and an excellent as a return value. So now you have removed the binding between the view list and the view item. So now you can enter here any type of function that could return this list item. How do you do that?
16:08
First, this is the original. You give it the address of view item and that's the output. But if you want something else as the view item, you can supply this function right here.
16:24
That's the lambda syntax in VB with this XML literal. And that's the output. So now you've separated those concerns without using any special frameworks. You haven't made any interfaces to do that or anything like that. Just work right out of the box.
16:43
So what can you do more? You can... Well, this is only for presenting data. You can put in some numbers and get out some XML that's also HTML. Let's do some more. We have a function and we can apply those in many different ways.
17:01
So let's build something that resembles a component. Here's a small function just to get some data. It has a side effect. It changes something outside. It changes our console doing the right line. And we can combine what we've done into a new function, returning an excellent.
17:25
So we get data and we call the view list with those data. We can refactor this the same way so we can have a repository function getting data and we can supply that. Give us an argument when we call this function.
17:42
This is enough for now. And we have another function called this component. And we can do interesting things with that as well. Let's try to cache the function call. This is a common exercise called memoizing, which is
18:01
common in some languages. In Python, it's quite trivial to do this. In C sharp, it's not. First of all, we set up a dictionary. We want to store the result of the function according to some key which defines the call.
18:22
And we can make this function. This is a higher-order function. We have a function in as input and we also have a function as return value. And the return value has the same signature as the input.
18:41
So it takes a function, returns the function, and the key as well. What does this thing do? Well, it looks in the memo. Do we have a value for this key? If we don't have it, we call the F function, which was our original function.
19:03
And we wrap this thing up in a function, returning that. So that's return value. So what happens if we use it? We call memoize, take our original function, and we get a memoized list component. It looks the same.
19:25
Can we call the same way? It works the same way. It's getting data. If we call it again, then we're not getting the data. So the original function wasn't called. But the caller didn't notice.
19:44
So it was completely transparent. That's the decorator pattern. If you were to do this in ordinary C sharp, you could use, like, post sharp, which you have to install, and it does choose your IL code and something else.
20:05
You can do it this way as well. This is not a pure function anymore, because it gives different output each time. But that's the purpose of the memoized function.
20:24
It doesn't call backwards every time. So we can take this even one step further. And we can do some generalization. We're moving to our next language here in C sharp. Give me three examples. One in each different language.
20:44
The first one is VB in C sharp, and there's going to be some F sharp as well. Here's the same thing. Does the same couple of changes. Turn it into a generic function so we don't have to return X element anymore. You can just return anything. And also, it's an extension method.
21:07
You see this over there. Because functions are values, so they can have extensions as well. Which can be quite interesting. And let's make another one. This one. Times the execution of the function.
21:24
A stopwatch. Calls the function. Writes out the time taken. And wraps the whole thing in something that looks just like our original function. And some logging. Quite similar. Just writes out any message you supply.
21:44
And calls the function. Returns the value. Now you can put all this together. Yes. First you have this. Needs more data. This one doesn't write anything out. It just sleeps for one second. Now we can do this.
22:05
What's this doing now? We're taking the get data function. We need to do some casting to have this work. And then we can take the function and wrap it using these functions.
22:21
So the first one wraps it in something writing out this. And then we use our memories. That's the key for the dictionary. And we log it again. And then we print out how long did this take.
22:41
So let's call it. Yep. Getting data. That's this one. Getting data from source. Yes, because this one didn't find it in the dictionary. So it went one step back. Called this one. Getting data from the source. That took one second. And call it once more. Now it's just getting data.
23:06
And we ended up here. And that returned the data. And time taken was approximately zero. So now we have something which you can call a DSL, a domain specific language for calling functions in a controlled way.
23:27
So these are not statements or anything like that. They are functions that build up a kind of structure which you can execute later.
23:40
Yeah. Yeah. So now we have a structure for caching about almost anything. We can it's not restricted to output thing in HTML. You can use it for any type of call. And you can do some timing and all that
24:00
anywhere. You haven't used any other libraries. Just the internal structures in C sharp to do this. We've got some side effects here. This thing is writing to our console, which is not maybe what we want to do. So let's do something about that.
24:20
You can refactor all those side effects. So instead of just calling console write line, let's make somebody else take care of that. Just give them some place to log. And you log the message. Whenever this function is called. There's something called action, which is just like funk without a return value.
24:45
It's a void. The input is string and it doesn't do anything more. Doesn't return anything. So you can use it like this. Now we like to log not to the console, but to our own log, which can be just a list of strings.
25:04
It can be anything. Question? Not inside the function. That's outside the function.
25:30
You have to have a side effect somewhere if you're going to see something on the console. So printing something out, that's a side effect. So as soon as you want something out of the system, you have a side effect.
25:43
So it's very difficult to make a C sharp programmer feel side effects. Because you have to have it at some point. But languages without side effects, like Haskell, where all functions have to be pure, so you can't do things like this.
26:03
And they solve it in different ways using monads and that stuff. But we're not doing pure functional programming here. So I don't care, really. That's the answer. Yeah.
26:23
The side effect is at least outside our function. Now the side effect is adding to the list of strings. Because the list of strings have this add function with the same signature as our action taking a string. So you can just supply that function
26:40
as a first argument like this. Now it's approximately the same. But when we call it, we don't get an output because the logging has happened elsewhere. We get the output here. That's the console write line. Length two. Okay. Same twice. So we can look at our log. And our data was put in the log.
27:04
List. Looks like this. Which is the same as we had before. But now we have some more control of this. And there are many possibilities with these techniques. We have implemented quite a few of these. Most of them. Not all of them are in production. We have used them for testing.
27:24
You can go through them. They are meant for data flow control. Because the system has quite a few external dependencies. We have this backend where all the data comes from. The met. And there's a lot of IO going on back. So we need to have some control over that traffic.
27:45
So what we're interested in is all these things. First of all, we can make a timeout function. We can put our function in and say you can
28:01
keep doing this for two seconds, then I'll just kill it. Exception handling. You can call the function. If it fails, do something interesting. Retry on fail if the function fails once. And you can try it again. Which is a very dangerous thing to do if it failed because the backend was overloaded.
28:21
So don't do that. We did it once. Not going to do it again. Instead you can use all the strategies such as polling. You can have a function with its own thread. Just fetching the data from the backend. And when it's called, it returns whatever came last. That can be done transparently in the same way. But you have to keep track of the thread.
28:44
So you have to have some instances going on. And tracing and logging, we saw that. Also the caching memo thing. You can implement grace, which is kind of caching taken from the varnish.
29:01
They call it grace if it's too old for the cache, but you still have it somewhere. And the backend fails to take some old data instead. You can do performance metrics like the timing we did. You can do all kinds of metrics. You can count how many times this fails, how fast it is and average and peak and all that. And if it's too slow, we can trigger an exception that can trigger a quarantine.
29:27
You have a function saying, well, this thing is in quarantine, so I'm not going to call to the backend. You take this exception or cache or whatever. And same with throttling and load balancing, which you know probably.
29:43
This can be extended to testing and simulation. Just like we can change our calls here, we can also change what comes from the backend. You can start with simulating the data from the backend as a function. And you can simulate a delay in the form you call the function back.
30:04
You can modify the data coming from a function and you can also simulate errors. We're going to do that shortly. So you can have an infrastructure. It looks like something like this. Which is not unrealistic. We have a number of sources.
30:22
They are very different in the behavior. You have these babies infrequent changes to give us a new value every day. What would you do? You can just call it, checks for new values every hour or so and give that back.
30:40
Don't have to call it all the time. Something with frequent changes, lots of traffic. You can cache it the normal way. You have some of these as well. Maybe you want to do a lot of things to make sure that this behaves or looks like it behaves
31:01
okay. So you can cache it and use metrics. And you can use it for testing, like I said. You can simulate data, distort data, throw errors and delays. So now you can test all this infrastructure. It's very difficult to test all your caching mechanisms and all that without simulating the data.
31:22
How can you... The only way to do this otherwise is to have a copy of your backend. And that's usually too expensive for many. So we start to simulate the data. And we can also simulate a slow backend and an error-prone backend like this. So I'm going to show you how that can be done. Using the same kind of technique.
31:48
Now, this one fails at a certain fail rate. It's called to the random generator, and that's an impurity. Because the random is not only also a function,
32:03
but it also has some internal state. They are not true random generators, these types. They are pseudo-random generators, so they keep the number in the side and they do something with it, so it looks like this random. That's why it's instantiated with new.
32:22
So in language like Haskell, doing random generators is non-trivial, but they can do it. And you can simulate a delay. That's how we do it. Just delays our call before calling back.
32:40
Here's another version of measuring. This one doesn't take the time. It counts how many fails there are and how many total calls there are. Now, there are some big problems with this one. We've been using static functions
33:02
all the time now. So it's very tempting to, if you need some state to modify inside this one, to make a static variable. You can't do this in production because this is global state, and modifying global state like this is something you never should do.
33:23
So this is just for the demonstration, just to make this work. In the real version of this, we have state objects that are given to the function, and we have some windowing going on. So we can slide in windows and see how
33:44
this thing forms over time. And a cache. That's the same as the
34:09
quite bad backend, which fails at a rate of 1.3. That's three failures out of ten calls. And you put the cache in front of that, and you put a delay on all calls.
34:22
And how does this perform? What kind of effect does this cache have? It's very difficult to answer. What's actually happening here? Well, the cache for two seconds, if you call this ten times per second,
34:46
you get this sequence of 20 passes. So it's okay, call from the cache. After that, it can fail because it's going to the backend. It can fail zero times, one time, many times. You don't know exactly.
35:01
The probability for each is 1.3. So what's the fail rate down here? Well, it's almost impossible to know just by looking at this. You can say two seconds, so it's something. You can calculate this. So let's try that. Probability of failure, 1.3,
35:24
timings for caching and delay like this. Probability for passing, that's this one. Numbers of passed calls, that's this one. Number of failed calls, that's a difficult one.
35:41
The probability for each one is 1.3. How many fails do you usually get in a row? What's the acceptance for that expected value? Well, it's this one. You have to look it up in your statistical formula book. This is one.
36:01
So the probability for failure is now calculated by the number of failures divided by the total number of calls. That expands to this one. Simplified to this one, which looks okay. So the original is zero. This is zero. This is one.
36:32
0.0210. That's an improvement by 15. Okay. That's good. We can also run this. The problem with this is that it's realtime.
36:48
Because we have this delay down here. That's a realtime function. So you have to this takes ten minutes to run. We have been doing things in nonrealtime,
37:02
especially we have also these are very simplified versions. We have versions with many arguments. We have versions with async support, which is more complex. These are the very simple ones. So you can simulate the clock
37:21
for the async versions. So you can run this in just a few seconds. But this one runs in realtime. So it takes ten minutes. And what we get at the bottom, you have 6,000 metrics. And you take the last one.
37:41
And if you go back, you see that what we returned here was a tuple with the value and a fail count, which is the fail rate. So our simulation gives
38:01
0.0225, which is quite close. So this must be correct. You can do it this way. You should be doing some of this. Because I think many people, many developers, they do this and have no idea what effects are. And but this is quite complicated. And developers, they don't know mathematics.
38:25
They can put up some testing infrastructure and run this. So you can do this instead. And this works without any other infrastructure. You can run it from a command line. You don't have to put up a web server and all that. You can use this to test your caching mechanisms.
38:45
So let's go to something completely different. A third example. Graphs. We have quite a few graphs on the website that look like this. We get emails from developers
39:08
who say, oh, great graphs you have. How did you do that? We even got a formal request from the Norwegian Navy. They wanted to have a look at our code and see, oh, how did you draw this?
39:24
It's very nice. We like to do the same. My standard response is just do it. It's not very difficult to just... There's lines on the bit map. How difficult can that be? This is a cat mode ROM spine, but it's taken care of elsewhere. Surprising how bad
39:44
many programs are at mathematics. This impresses people. There's no idea why. There are some interesting things there for a program. And that's all the similarities. You have... Well, they look much the same. You have this frame and this title and the textiles
40:05
are the same. And the line styles and the symbols are the same. Wind arrows are the same. Colors and all that. So what we want to do is we want not to draw all the details all the time.
40:22
We want something higher level. And we want not just a high level. We want the DSL. We want to express this as a language on its own. We want language where we can say, oh, I want the line. Just use the standard line. I want the symbol. Number N. And
40:45
at this position. That's our brand new one drop symbol. And I want perhaps I want some color markings. That's the red color marking. And I would like to do some grouping. I group lines, group the symbols. And I want to render this in many formats.
41:06
We're not talking about client rendering here. This is all server side rendering. But you can use JSON for rendering elsewhere, maybe. This is all about server side.
41:22
So Watson DSL. We saw in the previous example, we have this chain of functions that looks like another language. We have cache and fail and all that. That's in DSL. And we'd like DSL for this. But what's DSL, actually? There are two types.
41:44
External ones, which are separate code pieces like you need the separate parser. You have to read a file and evaluate this and execute it somewhere. You usually wouldn't do that unless you have special needs. Example of this are SQL and HTML and SVG. You could use SVG for
42:08
the graphs and render those to other formats. But it turned out it wasn't that practical. And it was a bit too low level. You can make an internal DSL, which is quite popular.
42:20
Then you're using a host language and you're trying to express your domain specific language in something using the host language. You can use anything from types and functions to macros. Some people seem to think that you have to use macros to make DSL and you don't have to do that. You probably shouldn't if you don't really have to. Some examples of DSLs,
42:49
ribbon rails, all these interfaces, we saw one recently. You have these for config, quite popular. And also for testing, I think it's called. Where you use extension methods
43:05
and use those and chain them and you build something. What's the difference between that and a normal API? You can ask. Well, it's interesting problem because I haven't found
43:21
any definition on what's the internal DSL as opposed to everything else you make. So I had to make a definition. My definition is that DSL is code that you use to build some custom structure and you can execute that somehow elsewhere. So these fluent interfaces,
43:45
they build a configuration structure and then it's run by all the code. A previous example, we have this chain built up this chain of calls and then you can execute it on the client. So what can you do in F sharp then? Because this is F sharp.
44:09
First, you need a data structure. F sharp has many nice features and many nice constructs. This is one of them called discriminated union. It's a very lightweight structure.
44:22
Defines a type which can have four shapes or forms. You can call it that. It can be a line or a symbol or anything that styles something or a group or something. And it's very lightweight to the extent that it doesn't even have names for the values
44:43
or the line. You're just given four integers and you have to know which one is which one. The symbol, you have to know that the last one is the number. It's a recursive structure. The style value takes one argument, a color, and another one is the graph object which
45:05
refers to other objects. From here, you could use green. And the same way with the grouping, you say you want the group of these objects. That's convenient. So you just put objects like
45:21
lines in the group. Unfortunately, these are not functions in F sharp. In F sharp, most things are functions, but this is not. But you can make functions quite easily in F sharp. This is going to be short tutorial in F sharp as well. These are functions just
45:42
enabling us to build those kind of objects by calling functions. And now it's easy to make more functions. You can have if you want to make a line, you can just make a function calling the line with these parameters. And to mark something, you can say that it's
46:05
well, it's going to start it as red. So it's quite powerful. We have defined the whole structure in just file lines and code. Now we have this kind of API to build that. Something missing here in the last one. We didn't give the third, no, the second
46:27
argument. That's what we talked about in the beginning. It's called partial application. If you don't give all the arguments to a function, you are given a function back,
46:41
which takes the missing arguments. And now you can call that one. So it's a shortcut in this case. It's the same as this one. So now if you see something like this, and it ends with the same, you can just remove it. Now this is the same. So now we have language for describing our graphs. That's what we need to make these
47:06
graphs and other code can take care of the rendering. So we're making an abstract graph just by calling these functions. So what have we got here, then? That's first of all, there are some partial applied functions. That funny symbol, that's
47:28
the multiplication as a function taking two arguments. There's one missing. So we're left with a function called scale X. Takes one argument and multiplies it by 20.
47:42
And the same with scale Y. We have list comprehension. We have comprehensions as well in C sharp and 3D. This is how it looks in F sharp. We need a set of lines. Like six lines. Scaled. Same.
48:00
Coordinates 0, 200. Put different types. And we have this function here creating a symbol. It takes the number and uses as a Y value. What we see here is that in F sharp, we have expressions everywhere. That's also characteristic for functional programming.
48:23
You have statements modifying your state. You have expressions giving something back. So everything is more or less an expression. So if it's not something that branches your execution path, it's expression. Giving either the first one or the last one.
48:47
So if the symbol code is 4, then the expression is marked version of the symbol. Otherwise, it's just a symbol. And since this is the last line of the function, that's also the return value of the function. We also have this syntax or maybe concise
49:06
syntax. We have these symbols. We have the data input. And we can map those to symbols. And we can make that's the overview of our structure. We have a group consisting of
49:25
a group of lines and a group of symbols. And we can make it like this. And print it. So this is what we were looking for. Hierarchical structure describing our graph.
49:43
So now we can execute that elsewhere. We can run this as text. Just for making an example of an easy one. So what have we got here? We got some function composition. Function to the left, function to the right, combines those
50:03
into something else. A new function, which takes the input or the first one, and you get the output or the last one. So render children is a function. And render child, we see that's another case of the partial application.
50:26
And we have some pattern matching, which is quite expressive way of doing a select. Same way as with the if statement, this is an expression returning values. So it matches on these. So now we have to know the order of the values in this structure.
50:47
And it returns those values. It doesn't print out anything. It returns strings. So we have expressions everywhere. Which is also typical of functional programming.
51:04
You don't modify things. You just make expressions. You combine those into something. So if you combine all this, what have we got here? We have got some pipelining. Those are also functions. Start with the data and you pipe that into a function.
51:21
You take the output of that one, and pass it into another function, which is a partial applied function here. And another one, print function, waiting for the output of that one. And it does this. So we have this infrastructure for writing graphs, and you can output them in very few lines of code.
51:45
And it all interrupts quite well with C sharp. It's not exactly the same all the time. You have these functions. They are more or less the same. The FS graph, that's a name space we
52:00
used in F sharp. So that's a static class. And there's a static function called make graph. And you can print it out. So that's the same as we saw. So we can draw some conclusions here. The main point of this is not to show you how to
52:24
print out HTML or how to do the caching in a system or how to make graphs. The point I was trying to make is that with functional programming, you can do almost anything. It's not specialized or academic programming. It's general purpose.
52:45
I would even say that object-oriented programming is more specialized. We didn't have anything resembling object-oriented programming in any of these examples. All static functions. No instances. No state being modified.
53:07
And also, because these languages become so powerful with functional programming, you can do more language-centric programming. We didn't even make any callouts to external libraries. All was done inside the language itself. These things are quite unusual to many
53:28
people, I think. So it requires a new way of thinking. If you're used to doing statements and get input and you write things and change things here, you have to change that way of thinking. You have to think of things as transformations or mappings and filtering
53:44
and all that. You take some data and you put it through a pipeline and get something out in the end. You don't change anything. There are some excellent things you get. You get composability. It's very easy to reuse what you made.
54:05
And also easy to maintain, because the functions are usually very well-defined, which also means that they can easily be tested. As you saw, we could test infrastructure structure from the command line without shutting out something else. Yes, that's about it, I think.
54:25
Any questions? Was it too difficult? Yeah, that's what we did here.
54:48
You don't see it here. This is type. It's a class. At the bottom is class, because it's not everything underneath. You have to have classes somewhere. I'm not quite sure
55:09
how this looks. I think there are different classes. I think there's a small class hierarchy. I'm not quite sure. But it has to fit inside the .NET structure, the CLR, so you can do interrupt.
56:05
Okay, can you hear me now? Yeah. The question was about function signatures, which is, ideally, that's been quite a problem, especially with the arguments. So we solved that by, you can just use closures and wrap your arguments into one function.
56:27
That's what we usually do. That's the easiest way. It's much easier in a dynamic language, where the calls are, you can have intercept the calls and you get all the parameters in one object. That's what they're doing, post, sharpen, all those things.
56:43
That's maybe easier, but it's not as lightweight. Nothing? Okay, that's it.