#systems

Production LLM Inference on Google Cloud

A production design for serving open-weight LLMs on GKE, covering routing, KV caches, autoscaling, rollouts, failures, and cost.

Speculative Decoding

How speculative decoding preserves a target model's distribution while replacing serial decode steps with parallel verification.