Summary
The 5.5 GiB activation reserve floor in UnifiedMemoryCap.resolvedActivationReserveBytes is sized for the worst-case model (gemma-4-26b at B=8: 5.05 GiB peak). Operators serving only gpt-oss-20b (B=8 peak: 2.56 GiB) cannot lower it via DARKBLOOM_ACTIVATION_RESERVE_GB because the env override is clamped to max(env, default).
Reproduction
On a 32 GB Mac with gpt-oss-20b as the only model:
darkbloom doctor
# [FAIL] model fits in RAM — gpt-oss-20b needs ~20.0 GB but only 14.5 GB is usable
The 20.0 GB breaks down as:
- Model weights (on-disk × 1.2 overhead): ~13.5 GB
- Activation reserve: 5.5 GiB (sized for gemma-4 B=8)
- Minimum serveable KV: 1.0 GiB
But gpt-oss-20b's measured B=8 activation peak is only 2.56 GiB (v0.8.0 benchmark data). A 2.75 GiB reserve (peak + slack) is sufficient, which would drop the model's need from 20.0 → 17.2 GB.
Root cause
resolvedActivationReserveBytes enforces a floor:
return max(bytes, defaultActivationReserveBytes) // line 307
The env var can only raise the reserve, never lower it. The comment says this prevents a "legacy 3" from recreating the B=8 OOM — but for operators serving only gpt-oss-20b, the floor is unnecessarily conservative and blocks models that would fit with a model-appropriate reserve.
Impact
On 32 GB machines (the most common provider hardware), gpt-oss-20b fails the doctor check and refuses to load, even when the system has enough free memory. The operator sees "this box's RAM is too small" when the real issue is a one-size-fits-all reserve sized for a different model.
Proposed fix
Allow DARKBLOOM_ACTIVATION_RESERVE_GB to set the reserve directly (remove the max floor). Operators serving a single model can size the reserve against their model's measured peak. The default remains 5.5 GiB for operators who don't set the env var.
The coordinator must be updated in lockstep when changing this on a fleet provider (servabilityActivationFloorGB in servability.go).
Additional findings
darkbloom status reports "Inference memory: 28 GB available" (static physical - 4), while darkbloom doctor reports ~14.5 GB usable (live OS free+inactive minus load reserve). These are inconsistent and confusing to operators. The status number should also reflect actual system memory.
Summary
The 5.5 GiB activation reserve floor in
UnifiedMemoryCap.resolvedActivationReserveBytesis sized for the worst-case model (gemma-4-26b at B=8: 5.05 GiB peak). Operators serving only gpt-oss-20b (B=8 peak: 2.56 GiB) cannot lower it viaDARKBLOOM_ACTIVATION_RESERVE_GBbecause the env override is clamped tomax(env, default).Reproduction
On a 32 GB Mac with gpt-oss-20b as the only model:
The 20.0 GB breaks down as:
But gpt-oss-20b's measured B=8 activation peak is only 2.56 GiB (v0.8.0 benchmark data). A 2.75 GiB reserve (peak + slack) is sufficient, which would drop the model's need from 20.0 → 17.2 GB.
Root cause
resolvedActivationReserveBytesenforces a floor:The env var can only raise the reserve, never lower it. The comment says this prevents a "legacy 3" from recreating the B=8 OOM — but for operators serving only gpt-oss-20b, the floor is unnecessarily conservative and blocks models that would fit with a model-appropriate reserve.
Impact
On 32 GB machines (the most common provider hardware), gpt-oss-20b fails the doctor check and refuses to load, even when the system has enough free memory. The operator sees "this box's RAM is too small" when the real issue is a one-size-fits-all reserve sized for a different model.
Proposed fix
Allow
DARKBLOOM_ACTIVATION_RESERVE_GBto set the reserve directly (remove themaxfloor). Operators serving a single model can size the reserve against their model's measured peak. The default remains 5.5 GiB for operators who don't set the env var.The coordinator must be updated in lockstep when changing this on a fleet provider (
servabilityActivationFloorGBinservability.go).Additional findings
darkbloom statusreports "Inference memory: 28 GB available" (staticphysical - 4), whiledarkbloom doctorreports ~14.5 GB usable (live OS free+inactive minus load reserve). These are inconsistent and confusing to operators. The status number should also reflect actual system memory.