<\/span><\/h3>\n03:14:22 - CRITICAL - kernel: [99283.12] oom-kill: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0\n03:14:22 - CRITICAL - kernel: [99283.12] Out of memory: Killed process 14202 (python3) total-vm:32.4GB, anon-rss:31.1GB\n03:14:23 - ALERT - Monitoring: Service 'DataIngestor' down on Node-04.\n03:14:45 - INFO - SysEng: Initial investigation shows heap fragmentation and 98% RAM utilization.\n03:15:10 - INFO - SysEng: Tracing memory leak to a single variable: 'pending_records'.\n03:15:12 - INFO - SysEng: It's a python list. Again.\n<\/code><\/pre>\n
\n<\/span>SECTION 1: THE LIE OF THE ABSTRACTION<\/span><\/h3>\nI\u2019ve been awake for 48 hours. My coffee is cold, my eyes feel like they\u2019ve been rubbed with sandpaper, and I\u2019ve spent the last six hours staring at a heap dump that looks like a Jackson Pollock painting if he only used “unallocated” and “pointer” as colors. The culprit? A python list<\/strong>. <\/p>\nThe problem with modern developers\u2014the ones who think a “bootcamp” qualifies them to touch production\u2014is that they treat the python list<\/strong> as some sort of magical, infinite, weightless container. They think it\u2019s a linked list. They think it\u2019s a “collection.” They don’t realize that under the hood, it\u2019s a brutal, unforgiving C array of pointers that will happily eat your RAM and ask for seconds.<\/p>\nWhen you call my_list = []<\/code>, you aren’t just creating a “box.” You are invoking PyList_New<\/code> in Objects\/listobject.c<\/code>. You are asking the CPython runtime to manage a PyListObject<\/code> struct. This struct contains a pointer to a block of memory (ob_item<\/code>) and a count of how many items are currently in it (ob_size<\/code>), plus the total capacity (allocated<\/code>). <\/p>\nOn a 64-bit system running Python 3.11.4, an empty python list<\/strong> isn’t “empty.” It\u2019s 56 bytes of overhead just to exist. <\/p>\n$ python3.11\n>>> import sys\n>>> l = []\n>>> sys.getsizeof(l)\n56\n<\/code><\/pre>\nFifty-six bytes for nothing<\/em>. And that\u2019s just the start of the tragedy.<\/p>\n
\n<\/span>SECTION 2: THE OVER-ALLOCATION TAX<\/span><\/h3>\nLet\u2019s talk about how a python list<\/strong> grows. It doesn’t grow one element at a time. That would be too efficient for the OS but too slow for the CPU. Instead, Python uses a “growth pattern” to minimize the number of times it has to call realloc()<\/code>. <\/p>\nIf you look at listobject.c<\/code> in the CPython source, specifically the list_resize<\/code> function, you\u2019ll see the logic. The growth factor isn’t a simple doubling. It\u2019s a bit more “creative.” The formula for the new size is roughly:<\/p>\nnew_allocated = (size_t)newsize + (newsize >> 3) + (newsize < 9 ? 3 : 6);<\/code><\/p>\nThis means the python list<\/strong> over-allocates. It grabs more memory than it needs “just in case.” Here is the sequence of allocated slots as you append: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88…<\/p>\nWhen our “DataIngestor” service was pulling 10 million records into a python list<\/strong>, it wasn’t just using memory for 10 million records. It was using memory for the next jump in the allocation sequence. If you have 8,000,001 items, the python list<\/strong> has allocated space for significantly more. That “slack” space is dead RAM. It\u2019s a ghost in the machine, sitting there, doing nothing, while the OOM killer sharpens its axe.<\/p>\n# Monitoring the growth of a python list\n>>> import sys\n>>> l = []\n>>> for i in range(20):\n... l.append(i)\n... print(f"Len: {len(l)}, Size: {sys.getsizeof(l)}, Allocated: {(sys.getsizeof(l)-56)\/\/8}")\n...\nLen: 1, Size: 88, Allocated: 4\nLen: 2, Size: 88, Allocated: 4\nLen: 3, Size: 88, Allocated: 4\nLen: 4, Size: 88, Allocated: 4\nLen: 5, Size: 120, Allocated: 8\n...\nLen: 9, Size: 184, Allocated: 16\n<\/code><\/pre>\nLook at that. At length 5, the python list<\/strong> jumped from 88 bytes to 120 bytes. It\u2019s already wasting space for 3 extra pointers. On a 64-bit machine, each pointer is 8 bytes. Multiply that by 10 million records, and you\u2019re talking about megabytes of wasted space just to hold nothing<\/em>.<\/p>\n
\n<\/span>SECTION 3: THE POINTER INDIRECTION NIGHTMARE<\/span><\/h3>\nHere is the part where the junior devs usually start crying. A python list<\/strong> does not store your data. It stores pointers<\/em> to your data. <\/p>\nIf you have a python list<\/strong> of integers, you don’t have a contiguous block of integers in memory. You have a contiguous block of 8-byte memory addresses. Each of those addresses points to a PyObject<\/code> somewhere else on the heap. <\/p>\nLet’s do the math for 1,000,000 integers.
\nIn a sane language like C, an array of 1,000,000 32-bit integers takes 4,000,000 bytes. 4MB. Simple.<\/p>\n
In Python, for a python list<\/strong> of 1,000,000 integers:
\n1. The list itself: ~8,000,000 bytes for the pointers (8 bytes each).
\n2. The integer objects: Each int<\/code> in Python 3.11 is at least 28 bytes.
\n – 24 bytes for the PyObject<\/code> header (refcount, type pointer, size).
\n – 4 bytes for the actual value.<\/p>\nTotal: (8 bytes * 1,000,000) + (28 bytes * 1,000,000) = 36,000,000 bytes.
\n36MB. <\/p>\n
We are using 36MB to store 4MB of data. That is a 9x overhead. And that\u2019s assuming no fragmentation. When the “DataIngestor” was running, it wasn’t just storing integers; it was storing complex dictionaries and strings. The indirection kills cache locality. The CPU spends all its time waiting for the RAM to fetch the pointer, then waiting again to fetch the object the pointer points to. It\u2019s a game of “where\u2019s Waldo” played at 3 gigahertz, and the RAM is losing.<\/p>\n
\n<\/span>SECTION 4: THE O(N) INSERTION SUICIDE<\/span><\/h3>\nI found a line of code in the ingestor.py<\/code> file that made me want to throw my mechanical keyboard through the window. <\/p>\npending_records.insert(0, new_record)<\/code><\/p>\nThe developer who wrote this clearly thinks a python list<\/strong> is a linked list. It is not. As I\u2019ve already established, it\u2019s a dynamic array. When you insert at index 0, the python list<\/strong> has to move every single other pointer<\/em> one slot to the right in memory. <\/p>\nIf the list has 1,000,000 items, insert(0, x)<\/code> calls memmove()<\/code> for 8,000,000 bytes. Every. Single. Time.
\nIf you do this in a loop to ingest 1,000,000 items, you aren’t doing $O(n)$ work. You are doing $O(n^2)$ work. You are moving trillions of bytes. <\/p>\nI watched the CPU metrics for Node-04. It wasn’t doing “data processing.” It was spending 99% of its cycles in libc.so.6<\/code> moving memory around because some kid didn’t understand that a python list<\/strong> is a contiguous block. This is why the service was lagging. This is why the “heartbeat” failed. This is why I was paged at 3 AM.<\/p>\n
\n<\/span>SECTION 5: THE 64-BIT TAX AND OBJECT OVERHEAD<\/span><\/h3>\nWe are running on 64-bit hardware. Everything is bigger. Every pointer in that python list<\/strong> is 8 bytes. In Python 3.11, the object headers are still massive. <\/p>\nLet\u2019s look at the PyObject<\/code> struct:<\/p>\ntypedef struct _object {\n _PyObject_HEAD_EXTRA\n Py_ssize_t ob_refcnt;\n struct _typeobject *ob_type;\n} PyObject;\n<\/code><\/pre>\nEvery single item you put in a python list<\/strong> carries this baggage. If you have a list of small strings, the overhead is even more insulting. A 1-character string in Python is about 50 bytes. <\/p>\nWhen we hit 31GB of RSS (Resident Set Size), it wasn’t because we had 31GB of data. We had maybe 3GB of actual data and 28GB of Python’s “convenience.” The python list<\/strong> is the primary vehicle for this bloat. It encourages you to keep references to objects alive longer than they need to be. Because the list holds a reference, the garbage collector can’t touch the objects. Even if you “delete” the data elsewhere, if it\u2019s still in that python list<\/strong>, it\u2019s staying in RAM.<\/p>\nAnd don’t get me started on the cyclic garbage collector. The python list<\/strong> is a container object, which means it\u2019s tracked by the GC’s linked lists for cycle detection. This adds even more overhead. Every time the GC runs, it has to traverse these massive lists, checking ob_refcnt<\/code> and looking for cycles. With 10 million items, the GC pause times were hitting 500ms. The whole application just… stops.<\/p>\n
\n<\/span>SECTION 6: REMEDIATION FOR THE INCOMPETENT<\/span><\/h3>\nIf I see another python list<\/strong> used for high-throughput data ingestion, I am going to start revoking git permissions. We have tools for this. Tools that don’t treat RAM like an infinite resource.<\/p>\n<\/span>1. Use collections.deque<\/code> for Queues<\/span><\/h4>\nIf you need to add things to the beginning and end, use a double-ended queue. It\u2019s implemented as a linked list of blocks. It doesn’t require memmove()<\/code> for every insertion. It\u2019s $O(1)$ for appendleft()<\/code>, unlike the python list<\/strong> which is $O(n)$.<\/p>\n<\/span>2. Use array.array<\/code> for Homogeneous Data<\/span><\/h4>\nIf you are storing 10 million integers, for the love of God, use the array<\/code> module.<\/p>\nimport array\n# This uses 4 bytes per integer. Period. No PyObject overhead for each element.\ndata = array.array('i', [0] * 1000000)\n<\/code><\/pre>\nThe array.array<\/code> stores the actual raw C values in a contiguous block. It\u2019s compact. It\u2019s fast. It\u2019s what a python list<\/strong> pretends to be but isn’t.<\/p>\n<\/span>3. Use NumPy for Numerical Work<\/span><\/h4>\nIf you’re doing math, a python list<\/strong> is a joke. NumPy arrays are contiguous, they allow for SIMD instructions, and they don’t involve pointer indirection for every single addition.<\/p>\n<\/span>4. Pre-allocate if you must use a list<\/span><\/h4>\nIf you know you need 1,000,000 slots, don’t append()<\/code> in a loop. That triggers the resize logic multiple times. Pre-allocate: l = [None] * 1000000<\/code>. This sets the allocated<\/code> size once. It\u2019s still a list of pointers, but at least you aren’t calling realloc<\/code> and memmove<\/code> twenty times while the list grows.<\/p>\n
\n<\/span>SECTION 7: THE “JUNIOR-LEVEL” CODE VS. REALITY<\/span><\/h3>\nLet’s look at the “before” and “after” of the refactor I had to perform while hallucinating from sleep deprivation.<\/p>\n
THE CRIME (Original Code):<\/strong><\/p>\n# Found in ingestor.py\nclass DataIngestor:\n def __init__(self):\n self.pending_records = [] # The ticking time bomb\n\n def add_record(self, record):\n # O(n) insertion. Absolute madness.\n self.pending_records.insert(0, record) \n\n # Memory grows indefinitely until OOM\n if len(self.pending_records) > 1000000:\n self.process_batch()\n\n def process_batch(self):\n # Slicing creates a NEW list. Another memory allocation!\n batch = self.pending_records[:500000]\n self.pending_records = self.pending_records[500000:]\n return batch\n<\/code><\/pre>\nTHE REFACTOR (What I wrote at 4 AM):<\/strong><\/p>\nfrom collections import deque\n\nclass DataIngestor:\n def __init__(self):\n # Deque handles O(1) appends and pops from both ends\n self.pending_records = deque()\n\n def add_record(self, record):\n # O(1) operation. No memmove.\n self.pending_records.append(record)\n\n if len(self.pending_records) > 1000000:\n self.process_batch()\n\n def process_batch(self):\n # No slicing. No new list creation.\n batch = []\n for _ in range(500000):\n if self.pending_records:\n batch.append(self.pending_records.popleft())\n return batch\n<\/code><\/pre>\nThe refactored version dropped the memory usage of the process by 40% immediately. Why? Because we stopped slicing. In Python, self.pending_records[:500000]<\/code> creates a shallow copy<\/em>. That means it creates a new<\/em> python list<\/strong> and copies 500,000 pointers into it. For a brief moment, you have the original list and the slice in memory at the same time. If you\u2019re already at 80% RAM usage, that slice is the bullet that kills the process.<\/p>\n
\n<\/span>SECTION 8: FINAL GRIEVANCES<\/span><\/h3>\nI am tired of explaining that “Python is slow” is a self-fulfilling prophecy written by people who don’t understand how a python list<\/strong> works. Python isn’t inherently slow; your use of its abstractions is incompetent. <\/p>\nThe python list<\/strong> is a tool. It is a general-purpose, flexible, dynamic array of pointers. It is not a database. It is not a message queue. It is not a high-performance numerical buffer. When you treat it as such, you are offloading the mental burden of memory management onto the hardware. But the hardware has limits. <\/p>\nWe lost two hours of production data because someone thought list.insert(0, x)<\/code> was “cleaner” than using a deque<\/code>. We lost $4,000 in cloud compute over-provisioning because the heap was so fragmented by millions of int<\/code> objects that the OS couldn’t reclaim memory even after the list was cleared.<\/p>\nNext time someone suggests we “just add more RAM” to fix a memory leak caused by a python list<\/strong>, I am resigning. We don’t need more RAM. We need developers who have read listobject.c<\/code>. We need developers who understand that every time they type []<\/code>, they are making a contract with the CPython runtime\u2014a contract that says, “I am too lazy to manage memory, so please waste it for me.”<\/p>\nThe audit is complete. The server is back up. I am going home to sleep. If the “DataIngestor” crashes again because someone replaced the deque<\/code> with a python list<\/strong> because “it’s more Pythonic,” don’t call me. Call a priest.<\/p>\nEND OF REPORT.<\/strong><\/p>\n
\n<\/span>APPENDIX A: MEMORY BREAKDOWN (64-BIT)<\/span><\/h3>\n\n\n\n| Item<\/th>\n | Size (Bytes)<\/th>\n<\/tr>\n<\/thead>\n |
\n\nPyListObject<\/code> Header<\/td>\n| 56<\/td>\n<\/tr>\n | \nSingle Pointer in ob_item<\/code><\/td>\n| 8<\/td>\n<\/tr>\n | \nPyObject<\/code> Header (Integer)<\/td>\n| 24<\/td>\n<\/tr>\n | \n| Integer Value (Small)<\/td>\n | 4<\/td>\n<\/tr>\n | \nTotal for 1 Integer in List<\/strong><\/td>\n88 (List overhead + Pointer + Object)<\/strong><\/td>\n<\/tr>\n\nTotal for 1 Integer in array.array<\/code><\/strong><\/td>\n| 4<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Note: The 88 bytes for the first element includes the initial over-allocation of the list. Subsequent elements cost 8 bytes (pointer) + 28 bytes (object) until the next resize event.<\/em><\/p>\n<\/span>APPENDIX B: THE GROWTH CONSTANTS<\/span><\/h3>\nFor those who still don’t get it, here is the growth table for a python list<\/strong> in 3.11:<\/p>\n\n- 0 elements: 56 bytes (Allocated: 0)<\/li>\n
- 1-4 elements: 88 bytes (Allocated: 4)<\/li>\n
- 5-8 elements: 120 bytes (Allocated: 8)<\/li>\n
- 9-16 elements: 184 bytes (Allocated: 16)<\/li>\n
- 17-25 elements: 256 bytes (Allocated: 25)<\/li>\n
- 26-35 elements: 336 bytes (Allocated: 35)<\/li>\n<\/ul>\n
If you have 26 items, you are paying for 35. You are paying for 9 pointers to nowhere. Multiply that by the scale of our production environment, and you\u2019ll see why the CFO is asking why our AWS bill looks like a phone number.<\/p>\n SIGN-OFF:<\/strong> | | | | | | |