Memory Problems, Did Collector Forgot to Clean the Garbage?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 112 | |
| Author | ||
| Contributors | ||
| License | CC Attribution - NonCommercial - ShareAlike 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/60827 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 202262 / 112
5
10
14
17
19
23
25
29
31
32
34
44
47
51
53
54
57
61
69
70
82
83
88
93
94
97
101
105
106
00:00
Read-only memoryObject (grammar)CodeComputer hardwareUsabilityAbstractionPhysicalismCartesian coordinate systemRead-only memoryBlock (periodic table)Analytic continuationSpacetimeProcess (computing)Data dictionaryAnalogyInstance (computer science)Computer fileAuthorizationElectronic mailing listIntegrated development environmentWeb pageLeakError messageObject (grammar)Different (Kate Ryan album)Presentation of a groupResultantMedical imagingPattern languageVirtualizationData structureData storage deviceMemory managementCodeAddress spaceOperating systemVirtual memorySpeicherbereinigungSet (mathematics)Asynchronous Transfer ModeRead-only memoryException handlingNeuroinformatikDisk read-and-write headComputer architectureSoftware developerSurfaceLevel (video gaming)Variable (mathematics)MiniDiscCausalityPhysical systemCore dumpAdditionPhase transitionFocus (optics)CASE <Informatik>MereologySpeicheradresseArithmetic meanView (database)Computer clusterPhysical lawState of matterInsertion lossNatural numberFigurate numberNumberCuboidLatent heatSheaf (mathematics)Right angleSoftwareComputer animation
09:02
Read-only memoryLatent heatRational numberModule (mathematics)Object (grammar)Stack (abstract data type)Process (computing)SpacetimeCodeMultiplication signAnalogyWritingRead-only memorySpeicherbereinigungCoprocessorMemory managementCuboidMechanism designCountingSlide ruleCycle (graph theory)AlgorithmForm (programming)Variable (mathematics)Numbering schemePhysicalismSubject indexingFunctional (mathematics)Read-only memorySpeicheradresseType theoryIntegerDependent and independent variablesArithmetic meanVirtual memoryElectronic mailing listFrame problemSocial classMereologyInterpreter (computing)Extension (kinesiology)Exception handlingField (computer science)Video gameSign (mathematics)Computing platformGoodness of fitSpeciesInstance (computer science)Physical systemCellular automatonObservational studyEntropie <Informationstheorie>Operational amplifierData dictionaryWebsiteRow (database)State of matterWaveMedical imagingFood energyMenu (computing)Sheaf (mathematics)RepetitionData structureComputer configurationElectric generatorStability theorySubsetComputer animation
18:03
Object (grammar)Module (mathematics)Level (video gaming)Rational numberLink (knot theory)Computer-generated imageryRead-only memoryPattern languageComputer programmingRow (database)Link (knot theory)Object (grammar)SpacetimeElectric generatorMultiplication signModule (mathematics)Electronic mailing listAlgorithmMereologyImplementationCountingCycle (graph theory)AdditionMathematical optimizationProgrammer (hardware)Order (biology)NumberMechanism designLevel (video gaming)Thresholding (image processing)Default (computer science)SpeicherbereinigungFunctional (mathematics)BlogVideo game consoleProcess (computing)Type theoryCodeRead-only memoryInstance (computer science)Exception handlingSocial classFrequencyMedical imagingDirection (geometry)WebsiteSlide ruleRight angleCanonical ensembleSampling (statistics)Endliche ModelltheorieArmDigitizingGroup actionSet (mathematics)Uniqueness quantificationDiscounts and allowancesComputer clusterReading (process)CodecFocus (optics)Natural numberKeilförmige AnordnungGame theoryEvent horizonComputer animation
27:05
Computer virusSpeicherbereinigungElectric generatorLecture/Conference
27:32
Matching (graph theory)Multiplication signControl systemSpeicherbereinigungVirtual machineBefehlsprozessorRead-only memoryProcess (computing)Lecture/Conference
28:01
FreewareBefehlsprozessorCycle (graph theory)Structural loadSpeicherbereinigungNeuroinformatikUtility softwareOperating systemIntegrated development environmentMultiplication signDeadlockPhysical systemRead-only memoryLeakInformation overloadCASE <Informatik>Control flowConfidence intervalInstance (computer science)Group actionGodMedical imagingPoint (geometry)Roundness (object)
30:21
Roundness (object)Computer animationMeeting/Interview
Transcript: English(auto-generated)
00:06
Hi, I'm Pratibha and welcome to my talk on Python memory problems and how garbage collection happens in CPython. I have tried my best to simplify it as much as possible, so let's
00:23
get started. I hope it is helpful in understanding how, in general, the object gets allocated and deallocated. Now, memory problems are the worst nightmare of every developer whose code is serving large files or who is having a lot of computation in the production environment.
00:43
If you've ever faced the issue of memory leak in application or out of memory exception and you're using Python and you're banging head that everything is working as it's supposed to be, but you're still not able to figure out what is happening, so maybe this talk
01:01
helps you in understanding how the underlying architecture of garbage collection works. So this talk is aimed to summarise how garbage collection that is being of the memory works in CPython, and because it is overwhelming to see the memory issues, let's get started because there are many scary issues and it's difficult to fix them because they
01:26
have dependencies. So let's get started. In recent years, we have seen many improvements in Python garbage collection, but there are still some instances when we are not getting
01:41
back the memory that we are freeing it or we are not getting the memory back as we bring up the large variables. So this results in memory crunch for the applications which finally crashes. Although there are multiple ways to overcome the memory challenges,
02:00
sometimes it is difficult to find what we can improve in our code and infrastructure that can make the memory efficient. In such cases, it helps to have an understanding of what is going on behind the curtain at the low level where memory is being managed. So this presentation aims to give you a quick overview of, we'll discuss some common
02:25
memory errors that we see in our day-to-day lives and how the CPython manages garbage collection in general. So let's list down some of the memory issues we usually see in our day-to-day environment. And sometimes we have large objects, let's say we created
02:46
a very big list, very big dictionary. For any computation, maybe we are passing a large log file, but even when we are out of that function, it is still lingering in the memory and that memory is not being released. And this is hanging around for
03:05
no apparent reason. Now there can be other reasons like reference filing in your code. Assigning references don't read distinct duplicate objects, but if an object is no longer used and cannot be, you know, marked for garbage collection because it is being
03:23
referenced in other places within the application, it results in memory leak. In fact, these kinds of referencing styles are one of the main problems of memory leak in the application code. Now this one was something which I faced very recently in the last few
03:46
months, which was called unexpected memory error. And there was no solution, there was no answer to why it happened. So even if you have enough RAM, you can get this unexpected memory error. After a lot of looking around, after a lot of going through
04:02
the articles, I realized that the pattern that was installed in my system, it was 32-bit and it has used up all the virtual address space that is available to it. And because 32-bit applications are limited to maybe two or four GB of user mode address space,
04:22
that was leading to this kind of issue. But the biggest one that we usually face is out of memory. Now this one is something which we can regenerate on our own using, assigning large memory chunks to the big files or big objects, or this is something which
04:45
just from the surface because we don't know what is happening, maybe there's a memory leak. So usually when this happens, what is happening is when an attempt to allocate a block of memory fails, most systems return this out of memory error. It's a generic one. But the core cause
05:01
of the problem really has to do with the actual out of memory. Maybe your actual memory space is not full. That's because the memory manager on almost every modern operating system uses the hardest space for storing memory pages that doesn't fit it. That's just called
05:21
swapping. So in addition to that, the computer can usually allocate memory until the disk is filled up. And this results in an out of memory error. That means your swapping space is also filled, your memory is also filled. So there is a chance that your memory is not actually filled with the space, which is for swapping and everything that is filled,
05:43
and your swapping has reached. But what if everything is in place and working as expected? Can there be another reason for these errors? This was the main thought in my mind when I was preparing for this talk, that what can be the reasons? And that leads me asking the question,
06:05
how actually memory is being allocated and de-allocated inside the Python. Let's have a look at the memory allocation and de-allocation, that is garbage collection of Python. And I hope that helps us in answering our question. So the most common explanation
06:29
of memory is thinking a computer's memory as an empty book. This is the most common explanation that we get when we try to explain it to some of the people who don't know what it
06:41
is, and it is intended for short stories, which are the short applications. Now there's nothing written on pages yet, different authors come along, which are processes, and each author wants some space to write these stories in. Since they're not allowed to overwrite each other, they must be careful about which pages they write in.
07:03
Before they begin writing, they consult the manager of the book. The manager then decides where in the book they are allowed to write. This is the standard explanation of how the memory allocation works in the computer memory. So in fact, in correct term,
07:25
computer memory is, it's common to call it fixed length, continuous blocks of memory pages. So this analogy holds pretty well for both of them. And authors are like different applications or processes that need to store data in memory. The manager who decides where
07:46
the author can write in the book and plays the role of memory manager of sorts, and the person who removes those stories to make the room for a new one is garbage collector. Let's have a quick look into CPython's memory structures.
08:03
In general, there is a layer of abstraction from physical hardware to the CPython usable hardware. The operating system abstracts the physical memory and creates a virtual memory layer that applications, including Python, can access. So this whole block that you see,
08:23
it can be considered as the virtual memory layer on top of the actual physical memory. An OA-specific virtual memory manager, it's a very long name, let's call it virtual memory manager, carves out a chunk of memory for the Python process. So whenever there's a new
08:44
process, it goes to the memory manager saying that, hey, I have this process, I want to run, I want some space. So what they do, they look into its continuous memory space and say that, okay, this particular set of memories for you. The dark gray blocks in the image below are
09:05
owned by Python processors. And CPython has an object allocator that is responsible for allocating memory within the object memory area, which is the blue box here. This object allocator is where most of the memory happens. Here, if you see, there are two parts. One is
09:24
object-specific memory where your actual object lies, and the Python's non-object memory is where it takes care of the processing and all the data that is required for the processing. Not the data, but the stacks and other things that it needs to process it.
09:40
Now, this object allocator, which is blue, it gets called every time a new object needs some space to be allocated or an existing object is deleted. Now, the question is, how will somebody know that object, how will a process knows that a particular object in memory has
10:02
to be deleted? Because it is not marked anywhere, it is not, how should it know? So, that's the question we are going to answer now. So, let's have a look into the garbage collection of CPython. Now, let's revise the book analogy and assume that some of
10:24
the stories in the book are getting old. No one is reading or referencing these stories anymore. If no one is reading something or even referencing in their own work, you can get rid of them to make room for new writing. That's where garbage collections come in. There are two aspects of
10:43
garbage collection. One is reference counting. Another is generational garbage collection. Let's have a quick look on how does a Python object looks like in physical memory or, sorry, the virtual memory. Let's say you assign X variable and it's, let's, sorry, I'm babbling.
11:08
Let's assume that you have a variable X and you have assigned value 20 to it. So, how will it look like? You will have a memory space which will store the actual
11:22
integer of 20 and it will have a label which is X and it will be pointing or referencing to that memory location. Actual Python object that will be holding this 20, the integer value will look like this. It will have a type which will tell that it's an
11:44
integer. It will have, that object will have value which will hold the actual value and then it has a reference count. Now, let's see what does this reference count mean in our next slides. So, the main garbage collection algorithm used by CPython is reference counting.
12:13
The basic idea is that CPython counts how many different places that have best reference
12:21
to an object. So, if we go back to this one, the reference count here is one because we don't have only one reference to it that is X variable. If you have more variables which is pointing to this one, it will get implemented. Now, such a place could be another object or a global static variable or even a local variable.
12:47
So, when an object's reference count becomes zero, the object is marked for garbage collection or it is deallocated. If it contains reference to other objects, the reference counts are decremented. Those other objects may be deallocated and written so it can have
13:04
cascading effect. If this decrement makes the reference count zero, then of course, and so on. The reference count field can be examined using a function called get ref count. It is part of our sys module which is available in Python. Notice that the value written by this function
13:27
is always one more than as the function also have reference to object when it is called. So, when you assign X equal to object, the reference count was one. When you call the function get reference count, it is actually referencing the variable X. So, it has incremented
13:46
to two. Now, when you assign a new variable which is Y and you assign its value as X, so now you have three, sorry, so now you have actually the two again.
14:03
Now, again you want to check how much is the reference count to it. So, now it has become three. Sorry, let me explain again. When you have X created, the reference basic count is one. Then, you have incremented the reference count by using the function get reference count because
14:22
it is reference index. Now, you assign the variable Y as a value of X. So, now both of them X and Y are pointing to the same memorial and the reference count of that particular Python object that resides in that memory location incremented to three. Now, you delete a variable Y. So,
14:49
when you delete a variable Y and if it is referencing to any of the other object, then the value of the reference count of that variable will decrement. So, when we again run the get
15:08
reference counting works. Now, there is a problem with this, how should I say, reference counting scheme. It is fine when you have simple way of assigning a variable,
15:24
declaring a variable, you have very simplest script, everything will work fine for the space. But the main problem with reference counting scheme or relying completing on reference counting is that it doesn't handle reference cycles. Now, let's see what reference cycles are.
15:50
So, for an instance, consider this code. In this example, X holds a reference to itself. So, even when we remove our reference to it, the variable X, the reference count never falls to zero
16:09
because it still has reference to its own internal self. Therefore, it will never be cleaned just by simple reference counting. For this reason, some additional machinery is needed
16:23
to clean these reference cycles between objects once they become unreachable. This is the cyclic garbage collector, usually called as just garbage collector. Even though reference counting is also a form of garbage collection. But garbage collector is comprised
16:40
of both reference counting and the actual mechanism which takes care of the objects which have reference cycle. Now, there is a way of thinking that why do we have reference cycles? Maybe we can avoid having reference cycles, which is fine. You can refactor your code
17:03
and everything will work. But sometimes it happens that it is required. So, let's have a look at that. The algorithm that Python uses to detect these reference cycles is implemented in the GC module. The GC module is part of your Python 4 internals and available
17:22
just as OAS and this module is available. The garbage collector only focuses on cleaning the container objects that contain the reference to one or more objects. These can be arrays, dictionaries, lists, custom class instances, classes and extension modules, etc. One could
17:46
think that cycles are uncommon in kinds of objects. But the truth is that many internal references needed by the interpreter create cycles everywhere. Let's have a look at some of the notable examples. The exception that contains the objects contains a list of frames
18:08
in the exception itself. It's a very widely available example of reference cycle. Module level function references the module cherry which is needed to resolve
18:25
globals and which in turn contain entries for module level function itself. Instances have references to their class which itself references its module. And modules contain references to everything that is inside or maybe other modules and this can be
18:43
back to the original instances. So, like this even if we don't want reference cycle is part of our code, part of our implementation and we have to learn how to deal with it in long run. Now, let's have a look of garbage collections additional machinery which takes care of reference
19:05
cycles because by now we have understood that it is part and parcel of our package. In order to limit the time each garbage collection takes garbage collector in CPAT can
19:21
use as a popular optimization which is called generational. Just for the record this image was taken from a blog post. It is a very clear image and really loved it and there's a link and credits but this is very clear and if I created my own image it would have been this.
19:46
So, I have reused the existing image. Now, the main idea behind this concept of generational garbage collection is the assumption that most objects have a very short lifespan and can be collected after their creation. This has proven to be very close to
20:07
reality for many pattern programs as many temporary objects are created and destroyed very fast. The older an object is the less likely it is that it will become unreachable. To take advantage of this fact all container objects are segregated into three spaces which
20:25
are called three generations. Each new object starts in the first generation which is generation zero list. The previous algorithm is executed only over the object of a particular generation and if an object survives a collection of its generation it will be moved to the next generation.
20:47
So, whenever a new object is created it gets into generation zero list then garbage collection will run over it time to time. The objects which have no references will be moved to discard list and they will be discarded. The objects that have references will be moved to the generation
21:05
one list and when the garbage collection of generation one list will be run the same mechanism happens and surviving objects will be moved to generation two list. In generation two list you will have objects which are going to survive till the end of your program.
21:25
So, using this mechanism it is easier because in generation one garbage collection can be run less often than generation zero because generation zero is supposed to carry the objects which are
21:40
going to be created fast and deleted fast. Then generation one garbage collection has happened less often and garbage collection in generation two will happen less often compared to generation one. So, this way we don't have to keep on running garbage collection on the complete list. We have segregated them into generation based on their references. But this is one of the
22:06
mechanism or this is one of the optimization that helps and limiting the time spent by C pattern in garbage collection. Now let's have a look at how we can actually look
22:21
this data of generation in our python code. So, generations are collected by number of objects that contain reaches some predefined threshold which is unique for each generation and is lower for the older number of generations. These thresholds can be examined using
22:44
in gc module you have get threshold. If you use this you will know how much is the threshold after which garbage collection will run on a particular generation list. By default python have a threshold of 700 for the youngest generation and 10 for each of the
23:04
older generations. You can check the number of objects in each generation using get count. Like here in generation zero you already have 596 objects but in generation one you have two and
23:21
generation two you have one. Now as you can see python creates a number of objects by default before you even start executing program. You can trigger a manual garbage collection process by using gc collect method. So here you have 595, two and one which existed even just you type
23:47
python in the console and you will get that. Now you will run gc collect method and running a garbage collection process clean up a huge amount of object because if you run the collect and then you again run the gc count you will get how much objects are remaining. Here there
24:06
are 577 objects in the first generation and three more in the older generation which are cleaned up. The best part here is that you can alter the thresholds of triggering the garbage collection here. This is something where you cannot change how the reference counting
24:23
reference counting work but you can change how frequently the garbage collection run in these generations. Now how are you going to modify the values here? There is a function
24:42
in gc called set threshold. Using this you can modify this value. In the example here we increase each of our thresholds from their default. Increasing the threshold will reduce the frequency at which garbage collector runs.
25:01
This will be less computationally sensitive in the program but the catch here is it will keep the dead objects around longer in the memory. So maybe there is a possibility that some here like I said like we have said the threshold that after 15 objects garbage
25:26
collection should run on generation two and if there are there are objects whose reference count has gone zero in generation two they will still lie around unless either the threshold of 15 is reached or there are some which keeps checking that okay there are you can just delete
25:46
those objects. Now one of the catch that I have noticed in not the catch I would say sometimes not always it is it happens very rare that people turn off the garbage collector altogether and manually manage it. That is also feasible but it is advisable not to do
26:07
that. With this we conclude our talk. I hope it is helpful to you. In the summary garbage collection is implemented in Python in two ways. Reference counting and generational.
26:26
When the reference count of an object reaches zero reference counting garbage collection algorithm cleans up the object immediately. If it has cycle reference count doesn't reach zero you wait for generational garbage collection algorithm to kick in and clean the object.
26:44
While as a programmer you don't have to think about garbage collection in Python it can be useful to understand what happens under the hood because maybe maybe you need some manual garbage collection to be run in your program. So with this thank you for your time.
27:08
Thank you very much. Hello. Thank you very much for your talk. We can still take a few questions if somebody has one so let's have a look if somebody is going up and yes making the way to
27:23
the microphone so please ask your question. Well many many things for your talk. I finally understood what is generational garbage collection. I never exactly understood it. I have one question. When we use Python in control systems and we tend to run Python processes for very
27:45
long times months and sometimes even a year or two and we have observed that whenever we have a high peak of CPU usage in the machine where it is running it's like the garbage like if the garbage collector stops working and we saw that we have a huge increase of memory
28:04
and then it never cleans up. Do you have an explanation for that? So this is my assumption because based on what the data you have provided you're whenever you have spike in CPU you notice that the garbage collection doesn't kick in.
28:22
Am I right? Doesn't? Excuse me what you said the garbage collector doesn't? I said that whenever there is a spike in CPU that means CPU is getting overloaded garbage collection doesn't have don't happen and the memory doesn't get free. Exactly yeah.
28:46
Exactly right because so here's the thing when you run a garbage collection you have this memory to be freed up and it needs its own CPU cycles. If you already have CPU cycles which are used for computation somewhere else then of course
29:05
it will get queued up in operating system that hey wait we have this computation already overloading the CPU and you have to wait for that so that's one of the possible reasons for it. So but yeah that's what I assume happening here and as long as the CPU cycles are available
29:26
the operating systems say that hey garbage collection let's go and clean up your you know garbage so that memory get free. But yeah it can be a cyclic you know deadlock CPU is not free the computation is blocking it garbage collection was not able to clean up that is
29:45
piling up the memory and piling up the memory the computation is not able to finish and CPU is getting spiked up. So in that case you have to figure out there must be some kind of memory leak or you have to restrict how you are how the CPU utilization is spiking up because
30:06
in production environment we say that if your CPU utilization is more than 70 percent that means you have to span a new instance so that the traffic get evenly distributed or the load get evenly distributed. Okay thank you very much that's all we have time for today
30:23
uh so let's have another round of applause for Patiba. Thank you.