TD-028-frontier-moe-out-of-scope-catalog-ceiling.md

id	TD-028
title	Frontier-scale MoEs (Kimi K2, Qwen3-235B) are out of scope at 30 GiB; the fitting adds are vision + embeddings
status	accepted
date	2026-06-22
supersedes
superseded_by
tags	catalog ram-ceiling moe kimi-k2 qwen3-235b internvl3 embeddings scope

TD-028: Frontier MoEs are out of scope at 30 GiB; the real adds are vision + embeddings

Context

A recurring "what bigger model should I run?" suggestion list (Kimi K2, Qwen3-235B-A22B, DeepSeek V3, dense 32B reasoners). This TD records the scope boundary so the same too-big candidates aren't re-proposed — the binding constraint is 30 GiB unified RAM and bandwidth-bound decode (TD-004), not model availability.

The arithmetic (why the headline picks don't fit)

Candidate	Size reality	Fits 30 GiB?
Kimi K2	~1T total / 32B active; Q4 ≈ ~500 GB, even ~1.8-bit dynamic ≈ ~250 GB	❌ no usable quant exists at this size
Qwen3-235B-A22B	235B total; Q4 ≈ ~133 GB, 2-bit ≈ ~88 GB	❌ not close
DeepSeek V3 / "Coder V3"	671B-class	❌ no
Dense 32B (R1-Distill-32B, Qwen3-32B)	Q4 ≈ ~19 GB — fits — but dense on this iGPU ≈ ~2.5–3 tok/s (cf. dense 14B=5.8, 24B=3.8, 27B=3.1)	⚠️ fits, but against the MoE thesis (TD-004); the existing MoE qwen3-30b-thinking (~15 t/s) dominates it

Decision

1. Do not chase frontier-scale MoEs in the catalog. They do not fit 30 GiB at any usable quant; "if it fits in RAM" is false for K2/235B/V3 here. (Mirrors TD-003: record the dead-end so it isn't re-litigated.)
2. Do not add dense 32B reasoners as drivers — they fit but run ~3 t/s and are strictly worse on this silicon than the MoE thinker already shipped.
3. The fitting upgrades are small/dense and complementary, not bigger generalists:
   • InternVL3 (8B/14B) — strong vision/OCR/UI-screenshot model; fits, worth pulling alongside the existing qwen3-vl-8b.
   • The embeddings already in carbon.toml (qwen3-embed-4b, bge-m3) — just carbon pull them; they feed RAG (TD-020) and the Agent Mind (TD-024).

Rationale

At 30 GiB the box is already at the best model class that fits — a 30B-A3B MoE — which is the README's own "honest ceiling". The leverage at this point is not a bigger model (none fits); it's better context use (TD-024/TD-020/TD-023) and filling capability gaps (vision, retrieval) with models that fit. Quality jumps from here come from a bigger box, not a bigger download.

Consequences

• Positive: stops wasted pulls/disk on un-runnable weights; redirects effort to the persistent-context roadmap and to fitting, useful additions.
• Negative: the absolute quality ceiling on this laptop stays at 30B-class — accepted and already documented (README: "no model that fits in 30 GiB reliably converges fix-until-green").

Related

• TD-004 (MoE-over-dense thesis), TD-003 (recorded dead-end pattern), TD-020 (embeddings/RAG consumer of the pulls).