Skip to main content

Featured

GPU Memory Sharding with FSDP

GPU Memory Sharding with FSDP

DISTRIBUTED DEEP LEARNING — PART 1/5 Day 1: Demystifying GPU Memory & The ZeRO Revolution — DeepSpeed, FSDP & State Sharding 25 min read Series: The Dharma of Development Distributed DL (Day 1 / 5) Level: Principal / Systems AI Engineer 💥 Context: You attempt to fine-tune or pretrain a 13-billion parameter dense transformer on an 80GB NVIDIA H100 or A100 GPU using standard PyTorch. You set your batch size to 1. You press enter. Within three seconds, your terminal explodes with the most dreaded error in artificial intelligence: torch.cuda.OutOfMemoryError: CUDA out of memory . How is this possible? In 16-bit precision, 13 billion parameters occupy only ~26 GB of disk space. Why can't an 80 GB state-of-the-art GPU train a 26 GB model? Because the naive mental model of machine learning memory is deeply flawed. Today, we demystify the true $16\Phi$ memory footprint of neural networks and master the architecture that made modern large language models possi...

FastAPI Dependency Injection: Real-World Architecture & Scoped State (2026)

BACKEND SERIES

Day 24: Dependency Injection — The Art of Inversion

Series: Logic & Legacy
Day 24 / 50
Level: Senior / Architect

⏳ Context: I once spent a miserable Saturday refactoring a microservice where every single function signature had twelve parameters. We were manually passing database sessions, logger instances, and user IDs through ten layers of "Clean Architecture" calls. It was a 2 AM epiphany: I wasn't writing clean code; I was a glorified courier. FastAPI's Depends() solves this, but most devs treat it like a magic black box. Today, we move beyond the magic and look at how we actually use DI in a high-concurrency production environment.

The Kurukshetra Armory illustrates automated component resolution and chaining.  The Request-Scoped Cache shows efficient reuse of database connections within each request boundary.  The Safety Yield demonstrates guaranteed resource cleanup (teardown), even when application crashes occur.


1. Beyond the Magic: The Scoped Lifecycle

In programming, Dependency Injection means your code declares what it needs to work, and the system (FastAPI) takes care of "injecting" it. But at a senior level, you aren't just looking for "shared logic." You are looking for Lifecycle Management.

The most powerful feature of FastAPI's DI isn't just getting a variable; it's the Request-Scoped Cache. If five different sub-dependencies all ask for a database connection, FastAPI ensures they all get the exact same instance for that specific request, and then tears it down safely afterward.

For those who want to see the "Zero-Library" raw implementation of this mental model—where we build the DI engine from scratch using the inspect module—I've pushed the raw code to the repository below.

Raw Implementation (Under the Hood):

github.com/.../dependency_injection.py

2. Production Pattern: Chained Dependencies & Teardown

In production, you don't just return a dictionary. you need Resource Teardown. By using the yield keyword, you create a context manager that ensures your database connections are closed even if the request crashes mid-flight.

The "Warrior" Production Pattern (Annotated + Yield)
from typing import Annotated
from fastapi import Depends, FastAPI, HTTPException

app = FastAPI()

# LEVEL 0: Resource Management with Teardown
async def get_db():
    db = DatabasePool.connect()
    try:
        yield db # This is injected into the caller
    finally:
        db.disconnect() # This runs AFTER the response is sent

# LEVEL 1: Hierarchical Logic (Krishna / Charioteer)
async def get_current_warrior(db: Annotated[Database, Depends(get_db)]):
    warrior = await db.fetch_user("Arjuna")
    if not warrior:
        raise HTTPException(status_code=403)
    return warrior

# Type Aliases make your signatures clean and reusable
WarriorDep = Annotated[dict, Depends(get_current_warrior)]

@app.get("/battle/strike")
async def launch_astra(hero: WarriorDep, target: str):
    # hero is already validated, authenticated, and DB-connected.
    return {"msg": f"{hero['name']} targets {target}"}

The Shareable Quote: "Clean APIs aren't about REST purity; they are about predictability. Dependency injection ensures your business logic doesn't have to worry about where the world ends and the data begins."

🛠️ Day 24 Project: The Recursive Guard

Time to stop using simple strings. Let's build a secure, hierarchical permission system.

  • Create a get_settings dependency that reads a .env file.
  • Create a get_auth_service that depends on get_settings.
  • Create a get_current_user that depends on get_auth_service and validates a JWT.
  • Implement a require_admin dependency that depends on get_current_user. If the user isn't an admin, it must raise a 403 before the endpoint even starts.
🔥 PRO UPGRADE / TEASER

We injected state today. But what if you want to modify the request itself before it even hits the DI layer? Tomorrow, we draw the battle lines: Day 25: Middlewares vs. Dependencies.

Architectural Consulting

If you are building a data-intensive AI application and require a Senior Engineer to architect your secure, high-concurrency backend, I am available for direct contracting.

Explore Enterprise Engagements →

Comments