#neural-networks

Model Compression via Knowledge Distillation

How knowledge distillation trains a smaller model from a teacher, including the core loss, practical variants, code, and common failure modes.

Low-Rank Adaptation (LoRA)

Why LoRA works, what its factorization changes, and the targeting, optimization, quantization, merging, and serving traps that matter.

Mixture of Experts - Mathematical Foundations and Scaling

How sparse Mixture-of-Experts layers route tokens, balance expert load, scale capacity, and trade computation for communication.

AI and the Art of Subtle Control

How ranking and prediction systems shape attention and choices, where claims outrun evidence, and what meaningful user control looks like.