Writing Clean Abstractions
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 130 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/49995 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
AbstractionPerfect groupRight angleMeeting/Interview
00:16
AbstractionSoftwareOpen sourceCompilation albumKolmogorov complexityPiCrash (computing)PlastikkarteDuality (mathematics)DatabaseFormal languageWeb pageDirection (geometry)Client (computing)LeakRight angleGoodness of fitComputing platformOnline helpCharacteristic polynomialSlide ruleFluid staticsOrder (biology)Type theoryCodeTerm (mathematics)Vulnerability (computing)Multiplication signLogic gateArray data structureConnectivity (graph theory)Sound effectCASE <Informatik>Computer fileDigital electronicsRemote procedure callAssembly languageProgrammer (hardware)BuildingInstance (computer science)Complex systemTowerForcing (mathematics)Virtual machineLevel (video gaming)Connected spaceObject (grammar)SoftwareInternet service providerLibrary (computing)Figurate numberQuery languageObservational studySoftware developerProduct (business)PlanningLatent heatProcedural programmingComplex (psychology)Machine learningMathematicsJava appletComputer animation
06:25
Wechselseitige InformationGrand Unified TheoryCommercial Orbital Transportation Services10 (number)ArmDuality (mathematics)Computer programAbstractionMaxima and minimaAnnulus (mathematics)Computer clusterInformation managementDivision (mathematics)Single-precision floating-point formatInversion (music)Interface (computing)1 (number)Social classDependent and independent variablesOperator (mathematics)Default (computer science)BuildingWeb 2.0Software frameworkCharacteristic polynomialPrimitive (album)Point (geometry)ChecklistBitCodeSubsetParameter (computer programming)Procedural programmingNumberSoftwareInversion (music)Client (computing)Single-precision floating-point formatSinePressureDecision theoryGoodness of fitCondition numberFile formatSoftware developerBoolean algebraInformation overloadDifferent (Kate Ryan album)Game theoryDatabaseObject (grammar)Computer fileDataflowText editorFehlererkennungCASE <Informatik>State of matterFlagMixture modelException handlingVideo game consoleUsabilityTrailEndliche ModelltheorieMathematicsLevel (video gaming)Field (computer science)Core dumpInformationModule (mathematics)System callIndependence (probability theory)Mereology2 (number)WebsiteMultiplicationArithmetic meanGraph coloringConnected spaceClosed set
12:34
Social classInjektivitätInstance (computer science)RadiusInheritance (object-oriented programming)Dependent and independent variablesConnected spaceGame controllerConfiguration spaceDifferent (Kate Ryan album)Default (computer science)Single-precision floating-point formatClient (computing)Object-oriented programmingMultiplication signExtension (kinesiology)Point (geometry)ParsingParsingCodeComputer programmingObject (grammar)Uniform resource locatorGradientEngineering drawing
15:21
Computer virusGeneric programmingChemical equationCode refactoringUsabilityCodeComplex (psychology)Context awarenessOperator (mathematics)Social classPrimitive (album)Library (computing)Projective planeBitDefault (computer science)Goodness of fitClient (computing)Level (video gaming)System callCASE <Informatik>Perfect groupMeeting/Interview
Transcript: English(auto-generated)
00:06
So I have here with me today, Smith Takharth, and we're talking about writing clean abstractions, right? Yep. Perfect. So take it away. All right, people, let's get started. So as she introduced me, I'm Smith and I currently live in Dubai.
00:25
And I work for a biggest classifieds company in Dubai called Dubizel. And today I'm here to talk about, as you can see, writing clean abstractions. So abstraction is somewhat abstract topic, so I will try to make it more concrete for you and give you tips on how to write clean abstractions.
00:43
So at Dubizel, I take care of building highly scalable and resilient infrastructure with infrastructure as code as a platform engineer, and also focus on building tooling for developer productivity. You can find more about me on my LinkedIn or GitHub.
01:03
So, yeah, let's get started. So to start with, if you were to explain someone what is an abstraction, just so that we all of us are on the same page, I would say in layman terms, abstraction is a way to say to your clients, give me the gist of what you want to do and let
01:22
me worry about the details of how. If you are a client of abstraction, you trust the provider to handle the details correctly. So why to abstract? Because abstractions give you leverage. Leverage is a term that comes from a physical lever, which let me move a huge object, which
01:40
I couldn't possibly move without a lever to amplify my force. Similarly, abstractions hide the low level details. This is called encapsulation and allow us to develop more and more complex systems. One of the classic example is this abstraction tower. The abstraction towers shown in the example shows us how building abstractions on top
02:03
of each other have enabled us to do really complex stuff. So for instance, here you can see assembly language is an abstraction of machine learning code, which is in hexadecimal, which is an abstraction of signals controlling the complex integrated circuit, which abstracts the components built out of gates, where
02:22
gate is an abstraction of certain arrangement of transistor. And we can go on and on about it. Someone who studied transistors in college may be able to tell us what transistors are abstracting on and so on. So before we talk about good or bad abstractions, let's understand the characteristics of abstractions.
02:43
So I like to think about things as strong and weak abstractions. Strong abstraction encapsulates a lot of complexity, and this is not necessarily a good thing. There are many times when you don't want to hide a ton of complexity. Conversely, a weak abstraction encapsulates very little complexity.
03:01
And here term weak doesn't necessarily mean a bad thing. Sometimes you actually need people to get down to the nuts and bolts of what you're providing them, because that's what they need in order to build something on top of your API or your platform. Another characteristic of abstractions is that they all leak.
03:21
And we have to live with that fact. By leak, I mean you need to know sometimes the underlying detail to build on top of it. Some of the common example can be a two dimensional array or SQL or NFS. Something as simple as iterating a large two dimensional array can have a radically
03:42
different performance if you do it horizontally rather than vertically, depending on the grain of the wood. One direction may result in vastly more page faults than the other direction. And page faults are slow. The second example is SQL. So the SQL language is meant to abstract the way the procedural steps that are needed
04:01
to create the database instead of allowing you to define merely what you want to do and let database figure out how out the procedural steps to create. But in some cases, certain SQL queries are 1000 times lower than another logically equivalent queries. You're not supposed to have to care about the procedure, only the specification.
04:21
But sometimes the abstraction leaks and causes horrible performance and you have to break down the query plan analyzer and study what you did wrong and figure out how to make the query run faster. Another example of a leaky abstraction can be NFS, even though network libraries like NFS and SMB let you treat files on remote machines as if they were local.
04:42
Sometimes the connection becomes very slow and goes down and the file stops acting like it was local. And as a programmer, you have to write code to deal with this. The abstraction of remote file is same as local file and it leaks. That being said, things can come crashing down if you do abstractions wrong.
05:04
If abstractions are not done right, the side effect of abstractions can be you introducing a technical depth to support backwards compatibility or you might end up rewriting abstraction in which case you will break the backwards compatibility. You need to think carefully when we design the abstractions or you will end up spending
05:20
time to clean up the mess in your code base. So what do we do? So we should think and analyze before we go ahead and make changes in the code and write clean abstractions. So before I talk more about how to write clean abstractions, let's understand what
05:45
are bad abstractions and how they are formed. I know it's not patriotic to use Java snippet in Python conference, it's just the static typing help to deliver better example, rest assured I have Python examples in the future slides. So here, so I wanted to have this interactive, but unfortunately I think the audio is one
06:05
way only. So the question was that which side, left or right, looks more cleaner and you think it's a good abstraction. In my opinion, it's the left. Why? I don't think there is a question.
06:21
Because the second one is a complete mess. If I look at the second one, there are questions that pop up in my mind, like connect to what is, what in the world is alt? Why do some mutators return nothing and others return bool? Why does close have a boolean to tell it whether or not you want to close?
06:42
Of course you do, or you wouldn't call close. Why are there two deletes that require substantially different information? Is one better somehow? What does that thing piles do? Why does only add only some fields? Notice the core of the object and has to do something with the abstraction respectively.
07:04
For example, there is open and close, but no method for connect. So it's a complete mystery what this does and if you should use it. The second overload of alt adds a mysterious parameter that seems to indicate this overload is some kind of consolation prize method or something, meaning a possible temporal dependency.
07:24
There appears to be some ad hoc mixture of exception or error code handling. Close wants a state flag. You need to keep track of the things internal state for it. It's inappropriate intimacy, inappropriate, by the way, inappropriate intimacy is a code smell that describes the method has too much intimate knowledge of another class or methods
07:45
in our working in our data, et cetera. Then does this interface want ad hoc primitives or first-class objects? It can't seem to make up its mind what defines the customer. The file stuff makes it seem like a class is a database access class,
08:01
retrofitted awkwardly for a corner case involving files, which is completely different ball game. The rest of the operations have at least one overload that deals with customer, but ad doesn't indicating ad is somehow different than other code operations. So how are bad abstractions formed? Like we developers are not evil.
08:21
We don't want to write code that could possibly, you know, provide bad interface to our clients, right? So how are they formed? As you can see in this example, existing code has powerful influence. Its very presence argues that it's both correct and necessary.
08:42
We know that code represents a part extended, and we are very motivated to preserve the value of this effort. And unfortunately, the sad truth is that more complicated and in a comprehensible code, deeper the investment creating it, the more we feel pressure to retain it.
09:00
This is like saying some cost policy, which refers to the fallacy of honoring the sunk cost, which a decision theoretically should have just been ignored. It's as if our unconscious tells us, goodness, that's so confusing. It must have taken ages to get it right. Surely it's really important.
09:21
It would be a sin to let all that effort go to waste. When you appear in this story, in step eight above, in above, this pressure may compel you to proceed forward. Is that to just implement a new requirement by changing the existing code attempting to do so is however, mutual the code no longer represents single common abstraction,
09:42
but has instead become a condition laden procedure with which interleaves a number of vaguely associated ideas. It's hard to understand and easy to break. If you find yourself in such situation, please resist being driven by the sunk cost. When dealing with wrong abstraction, the fastest way is
10:01
back. The fastest way forward is back do following reintroduce duplication by inlining the abstracted code back into every color within each color, use the use the parameters being passed to determine the subset of inline code that this specific color executes the little bits that are needed for this particular code. And yeah,
10:21
we'll prevent from the formation of a bad abstraction. So how can I go around building a good abstraction? Well, there are certain points you can keep in mind on, you know, I would call it a checklist maybe. So the first thing you should do is identify your data flow and primitives,
10:41
then define the methods that are essential to do all required operations. Then another characteristic of a good abstraction is to have safe defaults. Every good abstraction have safe defaults. So for the ease of use, take anything from DBs to web framework like Django and flask and text editors, which are the best example.
11:00
They are simple enough to be easily adapted by newbies, but are customizable enough by advanced user. So yeah, have safe defaults and then build layers on top of the abstraction for further common operations. And then last but not least, follow the SID principle from solid, which comes in handy while designing the software.
11:23
So I'm going to emphasize a bit on a SID which is single responsibility interface segregation and dependency inversion. So by SID, I mean, you know, the SID principle says that a class should have one and only one reason to
11:43
change. One class should only serve one purpose. This does not imply that each class should have only one method, but they should all relate directly to the responsibility of the class. A client should not be forced to implement an interface that it doesn't use. This means that we should break our interfaces into smaller ones.
12:02
So they better satisfy the exact needs of our clients. Similar to single responsibility principle, the goal of interface segregation principle is to minimize the side consequences and reputation by dividing the software into multiple independent parts, depend on the abstractions, not on concursions.
12:21
By applying the dependency inversion, the models can be easily changed by other models, just changing the dependency model and high level model will not be affected by the changes to the low level model. Well, that being said, let's take a small example of a SID. So I tried to come up with a more real world in a Pythonic example
12:43
that we can all relate to. Well, I stole some snippets and also just mapped how the RedisPy is written because I found it quite beautiful and well abstracted.
13:01
So as you can see here in the init method, it has safe defaults with the ability to have fine gradient control for the configuration of Redis. You can see here that we are using a lot of inheritance where extension is required. Another thing I would like to point is,
13:21
please always differentiate between abstract and non-abstract class. I've seen this a lot. People instantiate the base class, which is all written a million times in the code base. And then, you know, that damn method, that is instantiating the base class always breaks for no reason.
13:42
So it's better to prefix the abstract classes with something like base. For instance, here we have a something called a base parser. Yeah. And because it clearly indicates that it is only meant for inheritance
14:00
and it must not be instantiated. Another thing, we can always see things are abstracted out with SID principles in mind. All these classes we follow, we see they follow single responsibility principle and they use composition or dependency injection to work with each other. Okay. So what is composition? In another, it is another object oriented programming approach.
14:24
We use it when we use, want to use some aspect of another class without promising all the features of that other class. In my opinion, dependency injection is quite similar to composition. The only difference is when class B is composed by class A,
14:41
class instance owns the creation and control or controls the lifetime of class B. Whereas in dependency injection, we inject the instance of class B from outside. You can see we pass the instance of connection pool into the Redis client. Example of a composition would be based on the availability of high Redis.
15:00
We are swapping the parsers as you can see here. And another example would be the connection class based on the URL. We are setting the connection class in both cases. We are creating the instance inside the class. Well, that covers up the SID.
15:20
And I think I went too fast, but that's all I had to present any questions. Hello. Hello. Yeah. You, you still have 15 minutes left. Let's see then. So, well, thank you very much. And we have our first question. Very good.
15:41
So Thomas is asking how to balance abstraction, which is complexity and flexibility, which, which introduces complexity. Can you repeat the question again? Yes. So how to balance abstraction, which, sorry, which eases complexity and flexibility, which introduces complexity.
16:03
So how to balance between how to balance abstraction and flexibility. So what I would do is I would clear a layer of abstraction. So I'll create a one class with just the primitives and then
16:25
for the common operations for the ease of use, I will create another layer on top. That way, if someone wants to touch the low level details, he can use the primitives. And if someone wants to perform the common operation for the ease of use, I would use the layer above the primitive class.
16:43
And also, yeah, safe defaults can also give you ease of use. So yeah, I hope that answers your question. I think it does. Very good. And then we have another question from Jill. So Jill is asking, do you know, sorry, do you know any good examples of code following this following Python projects?
17:04
Yeah. You showed JavaScript code in the middle of a Python conference. My goodness. No, I also took a, yeah, I'm sorry about that guys. I am really sorry. The Redis client of Python is really good example. Another thing which I found pretty beautiful was the requests
17:24
library. I think they are just reading the code. They would give you a really good idea. And since you are a Pythonist, you have the context of, you know, how these things work. So you will be able to relate stuff also and understand the code easily.
17:47
Perfect. Thank you. Yes. Very good. So we have another question. Can you say something more about refactoring incorrectly?
18:00
Created abstractions towards something more flexible. Refactoring incorrectly? Yes. So how incorrect refactoring create abstractions towards something more flexible? I'm sorry. I didn't get the question.
18:21
Like incorrect refactoring would create abstractions towards more flexible abstractions? I think that's what he's asking. So the literal question is, can you say something more about refactoring incorrectly, created abstractions towards something more flexible?
18:40
Yeah. Sometimes what I've noticed that maybe you are too generic and it becomes too complex either for the users or trying to use that abstraction. But you thought with genericness in mind. So I don't know, like you should have a balance between genericness and what you are trying to
19:01
achieve. Maybe just limit your scope to certain use cases that you are trying to solve. Don't try to solve all the possible things that come to your mind. Sometimes I see that, you know, we don't keep business context in mind. We just try to, you know, solve every possible scenario. And I think that that's what leads to way too flexible and way too generic
19:22
abstraction, which brings in a lot of complexity while refactoring. Does that answer your question? Cool. So we're waiting to see if he replies on this course or to see if that
19:40
answers the question because we're all confused a little bit by the question. Don't worry. Well, perfect for now. The questions are over. Thank you very, very much. And there's still people typing. So I'll give you it. This is the last call for questions.
20:01
OK, so Jill is trying to understand exactly what Pawel asked beforehand. So restructuring the question and we think would be something like how bad refactoring could lead to a worse design. So a less flexible design. But you answered that question pretty well.
20:22
So it's all right. Cool. So any more questions at all? No, it doesn't seem like it.
20:41
Well, thank you very, very much.