#llms

Production LLM Inference on Google Cloud

A production design for serving open-weight LLMs on GKE, covering routing, KV caches, autoscaling, rollouts, failures, and cost.

Speculative Decoding

How speculative decoding preserves a target model's distribution while replacing serial decode steps with parallel verification.

Low-Rank Adaptation (LoRA)

Why LoRA works, what its factorization changes, and the targeting, optimization, quantization, merging, and serving traps that matter.

Understanding the Internal Mechanics of LLMs

A grounded tour of tokenization, transformer blocks, causal attention, training, inference, KV caches, scaling, and quantization.