#vllm

Production LLM Inference on Google Cloud

A production design for serving open-weight LLMs on GKE, covering routing, KV caches, autoscaling, rollouts, failures, and cost.