Commit Graph
1 Commits
Author SHA1 Message Date
shunyadandClaude Opus 5.5 d2755d60a1 fix(sm120): check Cake MegaMoE protocol errors at runtime
The Cake kernel reports an invalid call (e.g. its lossless return codec ran
out of overflow blocks) in a fail-closed flag, but DeepInfer read it only
once at warmup, so such a call silently returned wrong MoE outputs. The flag
is per call (the next launch clears it) and lives in the workspace all
layers share, so it has to be captured right after each call:

- CakeMoE ORs the flag into its own protocol_errors tensor after every
  call, on the stream (no host sync, also inside CUDA graphs), and names
  itself after its MoE layer.
- userbuffer.check_cake_moe_errors(), called once per step from
  DeepseekForCausalLM.sample, copies the layers' flags to pinned memory
  behind the step and checks the previous step's copy, so an error stops
  the run one step later without a host synchronization.
- check_protocol reads the same flag tensor.

Tests: tests/worker/test_cake_moe_errors.py (CPU).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 22:46:30 -08:00