We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Fun with cPython memory allocator

Formal Metadata

Title
Fun with cPython memory allocator
Title of Series
Part Number
113
Number of Parts
119
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date2014
LanguageEnglish
Production PlaceBerlin

Content Metadata

Subject Area
Genre
Abstract
Tomasz Paczkowski - Fun with cPython memory allocator Working with Python does not usually involve debugging memory problems: the interpreter takes care of allocating and releasing system memory and you get to enjoy working on real world issues. But what if you encounter such problems? What if your program never releases memory? How do you debug it? This talk describes some of the lesser known properties of cPython memory allocator and some ways to debug memory-related problems, all this based on real events. ----- Working with Python does not usually involve debugging memory problems: the interpreter takes care of allocating and releasing system memory and you get to enjoy working on real problems. But what if you encounter such problems? What if your program never releases memory? How do you debug it? I will tell a story of one programmer discovering such problems. The talk will take listeners on a journey of issues they can encounter, tools they can use to debug the problems and possible solutions to seek out. There will also be a brief mention of general memory management principles. cPython uses a combination of its own allocator, `malloc`, and `mmap` pools to manage memory of Python programs. It usually is smart enough, but there are some darker corners that are not well known by an average Joe Programmer (read: me). There are tools that can help debug memory problems, but those are also relatively unknown, and tend to have documentation that one might find lacking. I will describe one such tool, called `guppy`, which I have found particulary helpful.
Keywords