BLOG / TECHNICAL NOTES
Notes from
the workbench.
I write about the models, algorithms, and systems I'm working on or trying to understand. These notes are a way to think things through, one mechanism at a time.
ARCHITECTURES · OPTIMIZATION · SYSTEMS01 / ArchitecturesLONG-FORM ARTICLE
Understanding Kimi Delta Attention
From associative memory to parallel computation.
Deriving corrective writes, selective decay, WY-style matrix transitions and the triangular solve behind chunkwise KDA.
21 min readRead the article
02/
Muon from First Principles
From AdamW and matrix geometry to the polar factor, Newton–Schulz and practical GPU-efficient Muon updates.
19 min read