#ai
-
Production LLM Inference on Google Cloud
A production design for serving open-weight LLMs on GKE, covering routing, KV caches, autoscaling, rollouts, failures, and cost.
-
How speculative decoding preserves a target model's distribution while replacing serial decode steps with parallel verification.
-
Model Compression via Knowledge Distillation
How knowledge distillation trains a smaller model from a teacher, including the core loss, practical variants, code, and common failure modes.
-
Why LoRA works, what its factorization changes, and the targeting, optimization, quantization, merging, and serving traps that matter.
-
Mixture of Experts - Mathematical Foundations and Scaling
How sparse Mixture-of-Experts layers route tokens, balance expert load, scale capacity, and trade computation for communication.
-
AI and the Art of Subtle Control
How ranking and prediction systems shape attention and choices, where claims outrun evidence, and what meaningful user control looks like.
-
Understanding the Internal Mechanics of LLMs
A grounded tour of tokenization, transformer blocks, causal attention, training, inference, KV caches, scaling, and quantization.