text
root@prod-legacy-srv-04:/# apt-get install python3-cryptography
Reading package lists… Done
Building dependency tree
Reading state information… Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python3-cryptography : Depends: libc6 (>= 2.28) but 2.24-11+deb9u4 is to be installed
Depends: libssl1.1 (>= 1.1.1) but 1.1.0l-1~deb9u1 is to be installed
E: Unable to correct problems, you have held broken packages.
Look at that. Look at it until your eyes bleed. That output is the tombstone of my 30s. It’s 3:14 AM, the data center floor is a balmy 58 degrees Fahrenheit, and I’m vibrating from a lethal combination of gas-station espresso and the sheer, unadulterated hatred I feel for whoever decided that "it worked on my machine" was an acceptable excuse for pushing code.
For twenty years, I’ve lived in the cold. I’ve swapped blown power supplies in racks that hummed like angry hornets. I’ve traced SCSI cables in the dark. And for most of that time, my primary job wasn't "engineering"—it was "dependency archeology." We spent decades trying to convince a single Linux kernel to host five different applications that all wanted a different version of the same shared library. It was a hostage situation where the hostages were our weekends.
Then came the containers. They didn't arrive as a gift; they arrived as a desperate, heavy-handed tactical retreat. We stopped trying to fix the environment and started burning it down and replacing it every time a developer changed a line of CSS.
## The Ghost of Dependency Hell Past
In 2005, if you wanted to run a new service, you prayed to the gods of `ldconfig`. You’d spend four days compiling `gcc` from source just so you could compile a specific version of `openssl` that wouldn't crash the legacy PHP 4.3 app running on the same box. We lived in a world of "Shared Library Hell."
The problem is fundamental to how Linux handles binaries. When you run a program, the dynamic linker (`ld-linux.so`) looks at the ELF header, sees what `.so` files it needs, and goes hunting in `/lib` and `/usr/lib`. If two apps need `libxml2`, but one needs version 2.9.1 and the other needs 2.9.4, and those versions aren't binary compatible? You’re dead. You’re either running two physical servers (expensive), two VMs (heavy), or you’re spending your night manually setting `LD_LIBRARY_PATH` and hoping the whole house of cards doesn't collapse when a cron job runs an `apt-get upgrade`.
"It worked on my machine." I’ve heard that sentence more often than I’ve heard my own mother’s voice. It’s the battle cry of the developer who doesn't realize their MacBook Pro is running a completely different kernel, a different libc, and a different filesystem than the Debian 10 box I’m trying to keep alive with duct tape and spite.
## The Illusion of Isolation
We tried to fix this before Docker. We had `chroot`. We called it "chroot jail," and it was about as secure as a screen door in a hurricane. It changed the root directory for a process, but it didn't isolate the process tree, the network stack, or the IPC. A process in a chroot jail could still see every other process on the system if it had the right permissions. It could still exhaust the global PID space. It could still saturate the network interface.
Then the kernel developers gave us Namespaces. This is the "magic" that people think Docker invented. Docker didn't invent isolation; it just put a UI on top of kernel features that were too annoying for humans to use manually.
When you start a container, you’re just calling `clone()` with a bunch of flags. You’re telling the kernel: "Give this process its own view of the world."
- **UTS Namespace:** Give it a different hostname.
- **PID Namespace:** Let it think it’s PID 1, even though on my host it’s PID 45029.
- **Net Namespace:** Give it its own virtual eth0 and routing table.
- **Mnt Namespace:** Give it its own mount points.
It’s a lie. The container is a lie. It’s just a process. If you run `top` on the host, you see the containerized process right there, naked and shivering. But inside the container, the process thinks it’s alone in the universe. That isolation is what saved us from the "it worked on my machine" plague.
## What Is a Container, Really?
A junior developer walked into my office yesterday. He had a broken Python environment. He’d installed some global package that nuked his `pip` and now nothing would run. He looked at me with those wide, un-jaded eyes and asked, "Should I just use Docker?"
I sighed so hard I think I displaced a lung. I told him, "Sit down. Let me explain **what is** actually happening when you run a container."
It isn't a virtual machine. There is no hypervisor. There is no guest OS kernel. When you run a container, you are running a process on *my* kernel, but you're wrapping it in a straightjacket of cgroups and namespaces.
"What is" a container? It’s a way to package the entire user-space environment—every library, every config file, every binary—into a single, immutable image. It’s a way to ensure that the `libc6` version 2.28 that your app needs is exactly what it gets, regardless of whether the host server is running Debian, RHEL, or some experimental distro I built in a fever dream.
I showed him this Dockerfile, the kind of thing we use now to avoid the 3 AM calls:
```dockerfile
# Docker v25.0 Syntax - Using multi-stage builds to keep the bloat down
FROM python:3.11-slim-bookworm AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt
FROM python:3.11-slim-bookworm AS runtime
# Copy only the installed site-packages from the builder
COPY --from=builder /root/.local /root/.local
COPY . /app
WORKDIR /app
ENV PATH=/root/.local/bin:$PATH
# No more "it worked on my machine." It works in this image.
ENTRYPOINT ["python", "app.py"]
I told him: “This is how we stop the bleeding. We don’t install things on the host anymore. We build an image. We test the image. We ship the image. If the image works on your laptop, it works on the production cluster, because the image is the environment.”
Table of Contents
The RAM Tax We All Agreed to Pay
Don’t get me wrong. I hate the overhead. I remember when we could run an entire web server, a database, and a mail server on 128MB of RAM. Now, a “Hello World” microservice in a container takes up 200MB just to exist.
We’ve agreed to pay a “RAM Tax.” We trade hardware efficiency for developer sanity. Because hardware is cheap, but my time—spent debugging why libcrypto.so.1.1 is missing a symbol—is expensive.
We use Control Groups (cgroups) to manage this tax. In the old days, a runaway process would fork-bomb the entire server and I’d have to hard-reboot the machine. Now, I can tell the kernel: “This container gets 512MB of RAM and 0.5 cores. If it tries to take more, kill it.”
# Checking cgroup v2 limits for a running container in Docker 25.0
root@prod-node-01:~# cat /sys/fs/cgroup/system.slice/docker-<ID>.scope/memory.max
536870912
root@prod-node-01:~# cat /sys/fs/cgroup/system.slice/docker-<ID>.scope/cpu.max
50000 100000
This is the “peace of mind” that synergy-loving managers talk about, but they don’t understand the cost. The cost is layers upon layers of abstraction. The cost is a network stack that involves bridges, veth pairs, and iptables NAT rules that look like a bowl of spaghetti.
Why Your 2GB Image is an Insult to My Ancestors
While I’ve accepted containers as a necessary evil, I haven’t accepted your 2GB Docker images. I see you, junior dev. I see you using FROM ubuntu:latest and then installing vim, curl, git, and a full Java Development Kit just to run a Node.js script.
Every line in a Dockerfile creates a layer. These layers are managed by the storage driver—usually overlay2 these days, since we finally buried aufs and devicemapper in the backyard.
When you build an image, Docker uses a Union File System. It stacks these layers on top of each other. If you change one byte in a top layer, it doesn’t affect the bottom layers. This is great for caching, but it’s a nightmare for disk space if you don’t know what you’re doing.
# Inspecting the layer bloat
root@prod-node-01:~# docker history my-bloated-app:latest
IMAGE CREATED CREATED BY SIZE COMMENT
<ID> 10 minutes ago pip install -r requirements.txt 850MB
<ID> 12 minutes ago apt-get update && apt-get install -y gcc ... 400MB
<ID> 15 minutes ago COPY . /app 100MB
<ID> 2 hours ago FROM python:3.11 900MB
You’re shipping a whole operating system to run a script that calculates the price of a taco. It’s offensive. We used to fit entire operating systems on floppy disks. Now you’re telling me you need a gigabyte of “base image” because you’re too lazy to use a multi-stage build or an Alpine-based distribution?
The overlay2 driver works by having a lowerdir (the read-only layers), an upperdir (the writable layer where your app lives), and a merged directory (what the process actually sees). When you write a file in a container, the kernel performs a “copy-up” operation. It copies the file from the lowerdir to the upperdir before you can modify it. If you’re doing heavy I/O inside a container without using volumes, you’re killing performance. But hey, at least you didn’t have to deal with apt-get conflicts, right?
The Storage Driver Graveyard
We need to talk about the transition to Docker v25.0 and the death of the old ways. For years, we struggled with storage drivers. AUFS was the original, but it was never in the main Linux kernel. We had to use devicemapper, which was a nightmare of loopback devices and thin-provisioning pools that would inevitably run out of space and corrupt your entire filesystem at 4:00 PM on a Friday.
Docker 25.0 has doubled down on overlay2 and containerd. We’ve moved away from the monolithic Docker daemon doing everything. Now, dockerd is just a thin wrapper that talks to containerd, which talks to runc. It’s a chain of command that would make a general blush.
The storage driver is where the rubber meets the road. If you’re still trying to use zfs or btrfs drivers because you think you’re smarter than the defaults, stop. overlay2 is the standard because it’s fast and it doesn’t break (mostly). It uses the kernel’s native overlay filesystem support, which means less overhead and fewer weird locking issues.
# Checking the storage driver in Docker 25.0
root@prod-node-01:~# docker info | grep "Storage Driver"
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
If d_type is false, your life is a lie and your performance will be garbage. This is the kind of technical minutiae that keeps me awake. Not “synergy.” Not “digital transformation.” Just the raw, cold reality of whether or not my filesystem supports the metadata required to make container layering efficient.
The Final Confession: We Didn’t Fix It, We Just Boxed It
At the end of the day, containers are a confession of failure. We failed to make Linux distributions stable enough to handle multiple conflicting requirements. We failed to make software that is truly portable. So, we gave up. We decided to just bundle the entire world into a tarball and ship it.
We use containers now because the alternative is a return to the dark ages of manual configuration management. I don’t want to go back to writing 2,000-line Bash scripts to provision a server, only to have the script fail because a mirror in Sweden went offline. I don’t want to spend my life debugging why libstdc++.so.6 is missing GLIBCXX_3.4.21.
Containers allow me to treat servers like cattle, not pets. If a server starts acting up, I don’t log in and fix it. I kill it. I let the orchestrator spin up a new one. The container image is the source of truth. It is the only thing that matters.
But remember this: when you run that container, you are still running on a kernel. You are still sharing resources. You are still vulnerable to kernel exploits. You haven’t escaped the reality of systems engineering; you’ve just put a very pretty, very expensive box around it.
I’m going back to my data center now. I have a rack of servers that need to be decommissioned, and the fans are calling my name. Use your containers, kid. Use them to keep your dependencies isolated. Use them to make your deployments repeatable. But for the love of all that is holy, keep your images small, and never, ever tell me that it “worked on your machine” unless you’re prepared to hand me that machine and walk away forever.
The “Bare-Metal Exile” isn’t a title I chose; it’s a sentence I’m serving. And in this prison, Docker is the only thing that keeps the walls from closing in.
# Final sanity check before I go back to the cold
root@prod-node-01:~# docker ps --format "table {{.ID}}\t{{.Status}}\t{{.Names}}"
CONTAINER ID STATUS NAMES
f3a2b1c0d9e8 Up 45 hours legacy-app-proxy
a1b2c3d4e5f6 Up 12 days database-master
7g8h9i0j1k2l Up 5 minutes junior-devs-broken-app
The junior dev’s app is up. For now. I’ll give it twenty minutes before it hits the OOM killer because he didn’t set a memory limit. I’ll be here. I’m always here. In the cold. Waiting for the next dependency to fail.
Related Articles
Explore more insights and best practices: