# E21 follow-up: KV budget vs ubatch at 512K, and the 1M budget (Qwen3.5-4B Q4_K_M, dedicated test rig RTX 3080 10GB)
Measured 2026-10-28 ~17:10-17:25 ET, after the 9B sweep had stopped (GPU idle, 368-553 MiB desktop use).
Method as e21_4b_512k_vram_budget.md: llama-server, -ngl 99 (all layers on GPU), -fa on, -np 1, YaRN, llama.cpp's own buffer report (-v).
Usable ceiling: llama.cpp reports 9073 MiB free on CUDA0. Weights (CUDA0 model buffer) 2603.50 MiB. Recurrent state 50.25 MiB.
Budget = 9073 - 2603.50 - 50.25 - compute. bits/value = budget_MiB*8*2^20 / values.
Script: C:\AGI\agi-research\bench\e21_ub_1m.py; raw: C:\AGI\agi-research\e21_budget\ub_1m_results.jsonl and *.log.

## 1. 512K (8,589,934,592 KV values), q4_0 KV on GPU, load only
| ubatch | compute MiB | KV budget MiB | bits/value | with 300 MiB margin | q4_0 (4608 MiB) fits? | idle load used / shared |
|---|---|---|---|---|---|---|
| 512 (prior) | 2620.29 | 3799 | 3.71 | 3.42 | no, 808 over | 9881 / 1370 |
| 256 | 2334.28 | 4085 | 3.99 | 3.70 | no, 523 over | 9910 / 658 SPILL |
| 128 | 2191.27 | 4228 | 4.13 | 3.84 | no, 380 over | 9893 / 396 SPILL |

The compute buffer barely shrinks with ubatch (-286 MiB at 256, -429 at 128), far from the ~1.5 GB hoped for. Most of it scales with context length, not ubatch. The target loosens only from 3.71 to 4.13 bits/value. **Plain q4_0 still does not fit at 512K at any ubatch.**

**Prefill time at 512K: NOT MEASURED.** Every 512K config with q4_0 KV already spills at idle load (shared 396-658 MiB), so a timed prefill would be a spill run, which is invalid under the GPU-fit gate. Stock llama.cpp has no FA-supported KV type below 4.5 bits, so no stock config is fully resident at 512K. I stopped the script before its prefill stage. For reference, earlier E21 measured 4B prefill at 262K at about 2380 tok/s.

## 2. 1M (1,048,576 ctx, 17,179,869,184 KV values)
q4_0 KV would be 9216 MiB, larger than the whole usable ceiling, so I measured compute with --no-kv-offload (KV in host RAM, load only). At 512K this method matched on-GPU compute to within 8 MiB, but that hasn't been verified at 1M.
| ubatch | weights | compute (activations) | fixed state | weights+compute+state | KV budget MiB | bits/value | with 300 margin |
|---|---|---|---|---|---|---|---|
| 512 | 2603.50 | 6332.00 | 50.25 | 8985.75 | 87 | 0.04 | negative (does not fit) |
| 256 | 2603.50 | 5790.00 | 50.25 | 8443.75 | 629 | 0.31 | 0.16 |
| 128 | 2603.50 | 5519.00 | 50.25 | 8172.75 | 900 | 0.44 | 0.29 |

Weights plus activations alone do fit under 9073 at 1M, but only just. At the default ubatch only 87 MiB is left, and with a 300 MiB margin an empty KV cache won't fit at all. Even these KV-off loads showed shared usage (1130 / 604 / 342 MiB) from host-side buffers.
Activations at 1M (5.5-6.3 GB) are now the dominant cost, bigger than the weights. **At 1M, the compute buffer is the main problem rather than KV bits.** Getting there needs activation/compute-buffer reduction (for example chunked or streamed attention that doesn't scale with n_ctx) together with KV at roughly 0.3-0.45 bits/value.