DGX Spark (GB10) — Qwen3.6-27B-AEON benchmarks: engines × quantization × DSpark-DFlash

Single-stream inference benchmarks for AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored (BF16 and NVFP4) on a single NVIDIA GB10 / DGX Spark, comparing three serving engines and the Hikari07jp/DSpark-Qwen3.6-27B-AEON-draft speculative draft head. Measured with llama-benchy — same methodology as the sister studies (qwen3.5-122b, nemotron-120b, nemotron-puzzle-75b, gemma4-31b).

Headline: engines are identical on decode (bandwidth-bound); NVFP4 = 2× decode over BF16 (8.9 vs 4.5 tok/s); DSpark-DFlash n=8 on the NVFP4 target hits 21–26 tok/s (up to 3×) through 8k context, then collapses below baseline past ~8k (7.5 vs 8.6 tok/s @12k, down to 5.7 vs 7.7 @64k) — a sharp cliff consistent with the draft head running past its effective context. Full analysis in comparison.md.

Environment

Target model	AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored — Qwen3.6-27B (hybrid attention + Gated-DeltaNet), abliterated; BF16 (~51 GB) and NVFP4 (compressed-tensors, ~23 GB) checkpoints
Draft head	Hikari07jp/DSpark-Qwen3.6-27B-AEON-draft — 2B DFlash block-diffusion + VanillaMarkov head, distilled to AEON logits; K≤10, run at n=8 (README-recommended) and n=1, draft_sample_method=probabilistic, --mamba-cache-dtype float32
Engines	stock vllm/vllm-openai:latest (0.24.0) · eugr/spark-vllm:latest (0.23.1rc1.dev899) · ghcr.io/aeon-7/aeon-vllm-ultimate:2026-06-18-v0.23.0-dflashfix (0.23.0+aeon.sm121a.dflash; required for the draft)
Draft patches	the draft repo's vllm_patches/{qwen3_dflash.py, llm_base_proposer.py} copied into the aeon image at container start, plus a 2-line shim (_draft_kv_cache_group_ids = (), _draft_layer_to_kv_cache_gid = {} class defaults in dflash.py) to bridge the stock-0.23-vs-aeon base-proposer skew — see scripts/qwen36_dflash_driver.sh
GPU	NVIDIA GB10 (DGX Spark class), SM 12.1, 128 GB / 119 GiB unified LPDDR5X, ~273 GB/s
Driver / CUDA	580.159.03 / CUDA 13.0
OS / kernel	Ubuntu 24.04.4 LTS, kernel 6.17.0-1026-nvidia, aarch64 (Grace); host zgx-5457
Benchmark tool	llama-benchy 0.4.0
Serving params (fixed)	--max-model-len 131072 --gpu-memory-utilization 0.80, prefix caching off, chunked prefill on, TP=1; DFlash configs add --attention-backend flash_attn
Benchmark params	--pp 512 --tg 256 --exact-tg --runs 2 --latency-mode generation, depths 0 512 1024 2048 3072 4096 6144 8192 12288 16384 24576 32768 49152 65536
Date	2026-07-09/10

Configs tested

config	checkpoint	engine	speculation
Q_bf16_stock_n0	BF16	vLLM 0.24 stock	none
Q_bf16_eugr_n0	BF16	eugr/spark-vllm	none
Q_bf16_aeon_n0	BF16	aeon 0.23	none
Q_nvfp4_stock_n0	NVFP4	vLLM 0.24 stock	none
Q_nvfp4_eugr_n0	NVFP4	eugr/spark-vllm	none
Q_nvfp4_aeon_n0	NVFP4	aeon 0.23	none
Q_bf16_dfl1	BF16	aeon 0.23	DSpark-DFlash n=1
Q_bf16_dfl8	BF16	aeon 0.23	DSpark-DFlash n=8
Q_nvfp4_dfl8	NVFP4	aeon 0.23	DSpark-DFlash n=8 (draft card's reference pairing)

Contents

• comparison.md — verdict, tables, graphs, deployment war stories
• dataset.csv / dataset.json — every datapoint (9 configs × 14 depths)
• graphs/ — decode/TTFT/prefill vs context, grouped bars
• results/ — raw llama-benchy JSON
• scripts/ — the two benchmark drivers (incl. the working DFlash deploy recipe) + aggregation

Reproduce

python3 -m venv benchy-venv && ./benchy-venv/bin/pip install -U llama-benchy
bash scripts/qwen36_bench_driver.sh    # engine/quant matrix (6 no-spec configs)
bash scripts/qwen36_dflash_driver.sh   # DFlash configs (applies draft vllm_patches + shim)
python3 scripts/aggregate_and_plot.py results .

Caveats

Single-stream (concurrency=1), runs=2. Decode with DFlash is content-dependent (acceptance varies). The --enable-prefix-caching pass on the DFlash config hung (DSpark proposer × aeon prefix-align patch incompatibility) — no cached-TTFT data; prefix caching should be considered unsupported with this draft head on this stack. llama-benchy drives near-greedy generation; the draft card's acceptance gains are larger under T=1.0 sampling, so the ~8k crossover may sit slightly further out in sampled chat workloads. Numbers are specific to these images/models/hardware/dates.