text
\n<— 2024-05-22 03:14:12.441 —>
\n<— Last few GCs —>
\n[28492:0x64c0000] 14201 ms: Mark-sweep 2038.4 (2082.1) -> 2038.1 (2082.1) MB, 1142.1 \/ 0.0 ms (average mu = 0.142, current mu = 0.002) allocation failure; GC in old space requested
\n[28492:0x64c0000] 15403 ms: Mark-sweep 2038.1 (2082.1) -> 2038.1 (2050.1) MB, 1201.8 \/ 0.0 ms (average mu = 0.078, current mu = 0.000) last resort GC in old space requested<\/p>\n
<— JS stacktrace —> When the javascript array<\/strong> transitions to Do you have any idea how many CPU cycles you\u2019re wasting because you were too lazy to use a In Node.js v20 and v21, the Maglev compiler tries to optimize some of these checks, but it can only do so much when the underlying data structure is fundamentally broken. Maglev is great for mid-tier optimization, but it\u2019s not a miracle worker. It still has to generate guards for those holes.<\/p>\n Table of Contents<\/p>\n
\nFATAL ERROR: Reached heap limit Allocation failed – JavaScript heap out of memory
\n 1: 0x104697e00 node::Abort() [\/usr\/local\/bin\/node]
\n 2: 0x104576f34 node::OnFatalError(char const, char const<\/em>) [\/usr\/local\/bin\/node]
\n 3: 0x1047f8e10 v8::Utils::ReportOOMFailure(v8::internal::Isolate, char const<\/em>, bool) [\/usr\/local\/bin\/node]
\n 4: 0x1047f8db0 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate, char const<\/em>, bool) [\/usr\/local\/bin\/node]
\n 5: 0x1049969a8 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [\/usr\/local\/bin\/node]
\n 6: 0x10499949c v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [\/usr\/local\/bin\/node]
\n 7: 0x104995a34 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [\/usr\/local\/bin\/node]
\n 8: 0x104992e40 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [\/usr\/local\/bin\/node]
\n 9: 0x1049887e0 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [\/usr\/local\/bin\/node]
\n10: 0x104989094 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [\/usr\/local\/bin\/node]
\n11: 0x104969f6c v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::Handle
\n…
\nProcess finished with exit code 134 (interrupted by signal 6: SIGABRT)<\/p>\n## The Crime Scene: Line 402 of the "Ingestor" Service\n\nI am staring at a piece of code that looks like it was written by someone who thinks memory is a magical, infinite resource provided by the cloud gods. It\u2019s 3:14 AM. My coffee is cold, and the production environment for our high-throughput telemetry service is a smoking crater. The culprit isn\u2019t a DDoS attack or a hardware failure. It\u2019s a **javascript array**.\n\nSpecifically, this line:\n`const buffer = []; buffer[data.id] = data.payload;`\n\nTo the uninitiated, this looks benign. To me, it looks like a suicide note. The `data.id` in question? A 64-bit integer from an external system that occasionally spikes into the millions. By using a high-index assignment on a standard **javascript array**, the developer triggered a catastrophic transition in the V8 engine that turned a simple list into a bloated, memory-guzzling hash table, eventually fragmenting the heap until Orinoco\u2014the V8 garbage collector\u2014gave up and died.\n\nThis isn't "clean code." This isn't "idiomatic JavaScript." This is technical malpractice.\n\n## The Lie of the High-Level Abstraction\n\nDevelopers today are pampered. You\u2019ve been told that the **javascript array** is a Swiss Army knife. You think it\u2019s a list, a stack, a queue, and a dictionary all rolled into one. You think you can just `push()` and `pop()` and index into it at random without paying the piper. You are wrong.\n\nUnder the hood, V8 (the engine powering Node.js v21.x, which we are supposedly running) treats arrays with extreme suspicion. A **javascript array** is not a contiguous block of memory like a C-style array. It is a complex object that transitions through various "elements kinds" based on what you do to it. \n\nWhen you create `[]`, V8 assumes you\u2019re going to be a good citizen. It assigns it a hidden class (a Map) and sets its elements kind to `PACKED_SMI_ELEMENTS`. "Smi" stands for Small Integer. This is the holy grail of performance. The values are stored unboxed, directly in the array's elements store. It\u2019s fast. It\u2019s lean. It\u2019s what I miss about writing C++.\n\nBut the moment you do something stupid, like `buffer[1000000] = 'oops'`, you\u2019ve created a "hole." You\u2019ve moved from `PACKED` to `HOLEY`. And in the world of V8, once you go holey, you never go back.\n\n## Holey Elements: The Silent Performance Killer\n\nLet\u2019s look at what happened to our heap using the `--allow-natives-syntax`. I reproduced the developer's "logic" in a controlled environment to show exactly how they ruined my night.\n\n```javascript\n\/\/ node --allow-natives-syntax\nconst arr = [1, 2, 3];\nconsole.log(%DebugPrint(arr));\n\/\/ Elements Kind: PACKED_SMI_ELEMENTS\n\narr[100] = 4;\nconsole.log(%DebugPrint(arr));\n\/\/ Elements Kind: HOLEY_SMI_ELEMENTS\n<\/code><\/pre>\nHOLEY_SMI_ELEMENTS<\/code>, the engine can no longer assume that every index contains a valid value. Every time you access an element, the engine now has to perform a “hole check.” It has to check if the value is the special “hole” marker. If it is, it doesn’t just return undefined<\/code>. No, it has to climb the prototype chain to see if Array.prototype<\/code> or Object.prototype<\/code> has a property at that index. <\/p>\nMap<\/code>? You\u2019re forcing the engine to do a recursive lookup on every single iteration. In a high-throughput environment like our telemetry ingestor, this is the equivalent of trying to run a marathon with lead boots.<\/p>\n