Master Python from the inside out. Here, we don't just write code; we look under the hood at memory management, data types, and logic, all while applying the mindfulness and philosophy of the Bhagavad Gita to our development journey.
DISTRIBUTED DEEP LEARNING — PART 1/5 Day 1: Demystifying GPU Memory & The ZeRO Revolution — DeepSpeed, FSDP & State Sharding 25 min read Series: The Dharma of Development Distributed DL (Day 1 / 5) Level: Principal / Systems AI Engineer 💥 Context: You attempt to fine-tune or pretrain a 13-billion parameter dense transformer on an 80GB NVIDIA H100 or A100 GPU using standard PyTorch. You set your batch size to 1. You press enter. Within three seconds, your terminal explodes with the most dreaded error in artificial intelligence: torch.cuda.OutOfMemoryError: CUDA out of memory . How is this possible? In 16-bit precision, 13 billion parameters occupy only ~26 GB of disk space. Why can't an 80 GB state-of-the-art GPU train a 26 GB model? Because the naive mental model of machine learning memory is deeply flawed. Today, we demystify the true $16\Phi$ memory footprint of neural networks and master the architecture that made modern large language models possi...
Day 26: The Blind Architect — Observability Part 1
Series: Logic & Legacy
Day 26 / 50 (Part 1 of 3)
Level: Senior / SRE
⏳ Context: You deploy a highly concurrent FastAPI microservice. At 3:14 AM, the production alerts fire. You open your terminal, tail the logs, and see: Error 500: Database timeout while fetching user. Which user? Which request payload? You have 10,000 logs streaming per minute. Five different users hit the endpoint at the exact same millisecond. Without a way to group those logs together, you aren't debugging; you are guessing. Today, we cure your system's blindness. This is Part 1 of our Observability trilogy, where we manually wire the nervous system of our application.
Debugging in Asynchronous Python Applications
1. The Correlation ID (The Needle in the Haystack)
In a synchronous, single-threaded application (like old-school Django or Flask), you could almost get away with sequential logging. But FastAPI is asynchronous. While Request A is waiting for the database, the event loop switches to Request B, C, and D. Your log files are a jumbled, interwoven mess.
To fix this, every single log entry associated with a specific HTTP request must carry a Trace ID (or Correlation ID). But passing a trace_id string explicitly through 15 layers of functions (from endpoint -> service -> repository -> database adapter) is a massive anti-pattern that pollutes your clean code.
To see how to implement contextvars entirely from scratch (bypassing heavy third-party libraries), check out today's GitHub engine file. But for our application, here is how you build a production-grade interceptor.
import uuid
from contextvars import ContextVar
from fastapi import FastAPI, Request
# 1. Define the Context Variable. This is safe across async boundaries!
trace_id_ctx: ContextVar[str] = ContextVar("trace_id", default="-")
app = FastAPI()
@app.middleware("http")
async definject_observability(request: Request, call_next):
# Respect incoming tracing headers from API Gateways or Nginx
request_id = request.headers.get("X-Request-ID", str(uuid.uuid4()))
# Attach the ID to the current Asyncio Task
token = trace_id_ctx.set(request_id)
try:
response = awaitcall_next(request)
# Always return the ID to the client so they can report it to support
response.headers["X-Trace-ID"] = request_id
return response
finally:
# CRITICAL: Reset the context to prevent memory leaks
trace_id_ctx.reset(token)
2. Timing The Beast
Knowing what happened is only half the battle. You need to know how long it took. Before we can export metrics to fancy dashboards like Grafana, we must first capture the raw duration. time.perf_counter() is your best friend here—it's highly precise and ignores system clock updates (unlike time.time()).
The Shareable Quote:"Logs tell you that a fire happened. Observability tells you who dropped the match, what room they were in, and how fast the fire spread."
🛠️ Day 26 Project: The Structured JSON Logger
Parsing raw text logs is for amateurs. Your log aggregators want JSON. Build a custom logger that forces all outputs into JSON format, automatically including the trace_id.
Write a custom logging.Formatter subclass.
In the format() method, construct a dictionary containing timestamp, level, message, and trace_id (fetched from your ContextVar).
Use json.dumps() to return the formatted string.
Ensure your middleware logs the request duration (e.g., {"message": "Request finished", "duration_ms": 142.5, "trace_id": "..."}).
🔥 PRO UPGRADE / TEASER
Manual logs and prints are fine for a single server. But what happens when you have 50 pods running? Tomorrow in Part 2, we bring in the heavy artillery. We are going to expose these metrics to Prometheus, ship our JSON logs to Loki, and visualize the chaos in Grafana.
Comments
Post a Comment
?: "90px"' frameborder='0' id='comment-editor' name='comment-editor' src='' width='100%'/>