{"id":4872,"date":"2026-09-01T23:51:06","date_gmt":"2026-09-01T18:21:06","guid":{"rendered":"https:\/\/itsupportwale.com\/blog\/10-essential-aws-best-practices-for-cloud-optimization-4\/"},"modified":"2026-09-01T23:51:06","modified_gmt":"2026-09-01T18:21:06","slug":"10-essential-aws-best-practices-for-cloud-optimization-4","status":"publish","type":"post","link":"https:\/\/itsupportwale.com\/blog\/10-essential-aws-best-practices-for-cloud-optimization-4\/","title":{"rendered":"10 Essential AWS Best Practices for Cloud Optimization"},"content":{"rendered":"

text
\n[root@ops-bastion-01 ~]# aws ce get-cost-and-usage \\
\n –time-period Start=2024-05-18,End=2024-05-21 \\
\n –granularity DAILY \\
\n –metrics “UnblendedCost” \\
\n –group-by Type=DIMENSION,Key=SERVICE<\/p>\n

{
\n “ResultsByTime”: [
\n {
\n “TimePeriod”: {“Start”: “2024-05-18”, “End”: “2024-05-19”},
\n “Groups”: [
\n {“Keys”: [“Amazon Elastic Compute Cloud – Compute”], “Metrics”: {“UnblendedCost”: {“Amount”: “412.45”, “Unit”: “USD”}}},
\n {“Keys”: [“Amazon Simple Storage Service”], “Metrics”: {“UnblendedCost”: {“Amount”: “89.12”, “Unit”: “USD”}}},
\n {“Keys”: [“Amazon VPC”], “Metrics”: {“UnblendedCost”: {“Amount”: “12.50”, “Unit”: “USD”}}}
\n ]
\n },
\n {
\n “TimePeriod”: {“Start”: “2024-05-19”, “End”: “2024-05-20”},
\n “Groups”: [
\n {“Keys”: [“Amazon Elastic Compute Cloud – Compute”], “Metrics”: {“UnblendedCost”: {“Amount”: “84200.15”, “Unit”: “USD”}}},
\n {“Keys”: [“Amazon Simple Storage Service”], “Metrics”: {“UnblendedCost”: {“Amount”: “112450.30”, “Unit”: “USD”}}},
\n {“Keys”: [“Amazon VPC”], “Metrics”: {“UnblendedCost”: {“Amount”: “53120.44”, “Unit”: “USD”}}}
\n ]
\n }
\n ]
\n}
\n[root@ops-bastion-01 ~]# tail -n 50 \/var\/log\/k8s-autoscaler.log
\nE0519 03:14:22.123456 static_autoscaler.go:214] Failed to scale up: InsufficientInstanceCapacity for c5.9xlarge in us-east-1a
\nW0519 03:14:25.987654 cluster_state.go:120] Node group ‘worker-pool-heavy’ is at max size 500, but pending pods exist.
\n[root@ops-bastion-01 ~]# date
\nMon May 21 04:12:09 UTC 2024<\/p>\n

I haven't slept since Saturday. My keyboard is sticky with spilled espresso and the bitter residue of a "cloud-native" architecture designed by a consultant who charged $400 an hour to build a system that nearly bankrupted us in a single weekend. The Slack alerts started at 3:00 AM on Sunday\u2014the kind of alerts that don't just ping, they scream. PagerDuty was essentially a funeral dirge for our burn rate.\n\nThe "Incident" wasn't a hack. It wasn't a DDoS. It was a perfectly functioning, "infinitely scalable" pipeline doing exactly what it was told to do. It was a monument to the hubris of ignoring the underlying physics of networking and the predatory pricing models of managed services.\n\n## Ticket #9902: The $10,000-an-Hour NAT Gateway\n\nThe first thing I saw in the Cost Explorer was a vertical line for VPC costs. In a sane world, VPC costs are a rounding error. In this world\u2014the one the "Architects" built\u2014we were routing 400TB of raw telemetry data from a fleet of EKS worker nodes in a private subnet through a single pair of NAT Gateways to reach an S3 bucket in the same region.\n\nThe consultant didn't believe in VPC Endpoints. "Too much complexity," they said. "Just use the default route," they said.\n\nHere is the math they ignored: NAT Gateway processing charges are $0.045 per GB. That sounds small until you realize that 400TB through a NAT Gateway is $18,000 just in processing fees, not counting the standard data transfer rates. Because the worker nodes were in `us-east-1a` and the NAT Gateway was in `us-east-1b` (don't ask), we were also getting hit with cross-AZ transfer taxes of $0.01 per GB in both directions.\n\nI had to rip out the routing table and replace it with a Gateway Endpoint while the system was under load. This is the Terraform I had to hot-patch into production at 4:00 AM while my hands were shaking:\n\n```hcl\n# Terraform 1.7.x - Fixing the "Consultant Special"\nresource "aws_vpc_endpoint" "s3_gateway" {\n  vpc_id       = var.vpc_id\n  service_name = "com.amazonaws.us-east-1.s3"\n  vpc_endpoint_type = "Gateway"\n\n  # Attach to all private route tables to bypass the NAT Gateway\n  route_table_ids = data.aws_route_tables.private.ids\n\n  tags = {\n    Name        = "s3-gateway-endpoint"\n    Environment = "production"\n    Reason      = "Stopping the NAT Gateway bleeding"\n  }\n}\n\n# We also had to kill the NAT Gateways once the routes propagated\n# But first, we had to ensure the Interface Endpoints for STS and Kinesis were there\nresource "aws_vpc_endpoint" "kinesis_interface" {\n  vpc_id              = var.vpc_id\n  service_name        = "com.amazonaws.us-east-1.kinesis-streams"\n  vpc_endpoint_type   = "Interface"\n  private_dns_enabled = true\n\n  subnet_ids          = var.private_subnet_ids\n  security_group_ids  = [aws_security_group.vpc_endpoint_sg.id]\n}\n<\/code><\/pre>\n
The “aws best” practices\u2014the ones they actually put in the whitepapers but nobody reads because they’re too busy looking at “Serverless” infographics\u2014explicitly state that you should use Gateway Endpoints for S3 and DynamoDB. It\u2019s free. It\u2019s a routing table entry. But no, we paid $53,000 for a NAT Gateway to act as a glorified, expensive straw for our data.<\/p>\n
\n
Table of Contents<\/p>\n