Commit Graph
4 Commits
Author SHA1 Message Date
shunyadandClaude Opus 5.5 addd6e6c3e fix(eval/cvbench): do not count missing sources as 0% in the overall score
summarize_extra averaged ADE20K/COCO/Omni3D with .get(source, 0.0), so a run
that scored no rows of a source (e.g. --limit 200, which keeps only ADE20K
rows because records are taken in file order) reported an overall CV-Bench
score of at most a quarter of the real accuracy.

The overall/2D/3D scores are now reported only when every source they
average has scored rows; otherwise summary.json lists
cvbench_missing_sources and the scorer warns. Per-source and per-task
accuracies are reported either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 22:22:15 -08:00
shunyadandClaude Opus 5.5 bdb6014ca1 fix(eval): default MTP varlen off on SM120 and record the effective value
run_eval.py forced DEEPINFER_ENABLE_MTP_VARLEN=1, which the engine refuses on
SM120, so `run_eval.py --bench aime|hle|gpqa` (and their driver scripts)
failed at LLMEngine init unless the user remembered --env ...=0. meta.json
also assumed mtp_varlen=True whenever --env did not mention it.

- One _default_engine_env() supplies the defaults for both the rank processes
  and meta.json: MTP varlen on, except on SM120.
- meta.json's mtp / mtp_varlen and bench run_metadata now see the same
  defaults-then---env environment the ranks run with.
- README.sm120.md no longer asks for --env DEEPINFER_ENABLE_MTP_VARLEN=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 22:22:14 -08:00
Zihua WuandClaude Opus 5.5 efecbb5ba6 feat(sm120): offline inference on NVIDIA Blackwell SM120
Run DeepSeek-V4.1-Flash offline inference on SM120 GPUs (RTX PRO 6000
Blackwell) with the same entry points as SM100. SM100 and Ascend paths
are unchanged.

- Build: SM120 is detected from the first visible GPU
  (scripts/nvidia_arch.sh). It builds DeepEP and DeepGEMM from its nv_dev
  branch (vendor/DeepGEMM-sm120), and no FlashMLA or DeepSelect;
  FlashInfer is installed from a pinned commit
  (requirements-nvidia-sm120.txt).
- Attention: V4.1 sparse MLA runs on FlashInfer's SM120 sparse MLA
  kernels, reading the inline-scale KV records as stored; KV blocks are
  aligned to 512 bytes.
- Indexer: FlashInfer's fused DSA indexer scores the index keys and
  selects their top-k in one call, for decode on every indexer layer
  (candidate source and consumer layers included) and for prefill outside
  the candidate layers. The remaining selections (prefill of the
  candidate layers, the sampler) use FlashInfer's top-k behind
  DeepSelect's interface (model_executor/layers/topk.py), in
  deterministic mode.
- MoE EP: DeepEP over NCCL GIN by default, or NCCL collectives
  (DEEPINFER_SM120_EP_BACKEND=nccl); NCCL settings known to break the EP
  paths are rejected. EPLB, including decode-only draft-layer EPLB, and
  per-layer YoCo MoE dispatch.
- Engram tables live in page-locked host shared memory; killed workers
  leave nothing in /dev/shm.
- Features: attention DP + MoE EP, DSpark/nextn speculative decoding,
  CUDA graphs, async scheduling, decoder SWA bounded replay, structured
  outputs, image inputs, multi-node deployments. Features without an SM120
  implementation (CP > 1, MTP varlen, batch-invariant mode, GPU-resident
  Engram tables) are refused at engine start.
- Eval: chat GSM8K and CV-Bench; tests for the SM120 kernels and their
  DeepInfer call sites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 18:32:01 -08:00
Jiashi Li d585e6542c initial commit 2026-10-20 12:09:42 +08:00