Question-first at 704K: Qwen3.5-4B found the right passcode out of four in 9 of 10 fresh trials on a 10 GB RTX 3080

After question-first beat question-last 9 to 2 at 512K (E21n), I ran ten new, unselected multikey needle trials at about 710K tokens, the largest context this 4B model holds on a 10 GB card. Question-first passed 9 of 10 (exact 95% interval 55% to 100%), meeting the bar of 8 of 10 set before the run. The one miss picked a decoy. This is a known prompting technique, measured here on my own setup.

Experiment	E21O
Date	2026-10-29
Benchmark	RULER-style multikey needle: one target passcode sentence plus three decoy passcodes for other vault colors, in shuffled paragraphs from the MATH training set, filled to about 706K to 713K tokens. Greedy decoding, 64-token answer limit. A run passes only if the first code in the answer is the target code.
Model	Qwen3.5-4B, Q4_K_M GGUF, YaRN context extension x2.75 to 720,896 tokens, flash attention on, K and V cache at q4_0, ubatch 64
Hardware	One of my dedicated test rigs: RTX 3080 10 GB
Verdict	PASS

What I tested

10 trials with seeds 2300 to 2309, random passcodes, random target color and target depths from 10% to 90%. Every prompt states the question before the document and again after it. Question-last was not rerun in this round. llama.cpp built with upstream PR 29827.

Pass bar, set before the run

Set before the run (descriptive): report the pass rate with an exact 95% interval; call 704K usable for this task if at least 8 of 10 pass.

Result

Trial	Depth	Prompt tokens	Answer	Peak VRAM	Time
1	10%	711,360	silver code (pass)	9,818 MiB	18.1 min
2	90%	707,413	orange code (pass)	9,828 MiB	18.1 min
3	30%	712,964	orange code (pass)	9,828 MiB	18.3 min
4	70%	713,017	yellow code (pass)	9,828 MiB	18.3 min
5	50%	705,915	purple code (pass)	9,818 MiB	17.9 min
6	15%	706,739	blue code (pass)	9,875 MiB	18.1 min
7	85%	708,254	orange code (pass)	9,856 MiB	18.1 min
8	40%	711,452	black decoy (fail)	9,866 MiB	18.3 min
9	60%	706,876	red code (pass)	9,866 MiB	18.1 min
10	20%	708,110	red code (pass)	9,856 MiB	18.1 min
Total			9 of 10 pass	9,818 to 9,875 MiB

Exact (Clopper-Pearson) 95% interval for 9 of 10 is 55.5% to 99.7%. Ten trials is a small sample.

The one miss was trial 8 at 40% depth: the target was silver, and the model answered with the black decoy code.

Shared (system) memory read 170 MiB in every run, against 146 MiB at 512K. Part of that is the embedding table, which stays in system RAM in this setup, so the runs are not 100% on the GPU, and I have not yet shown whether the extra 24 MiB at 704K is a small spill or driver overhead.

Prefill ran at about 660 tok/s and each run took about 18 minutes, so a 704K prompt is slow on this card.

A paired question-last run on the same ten documents (E21p) started right after this one, so the size of the question-first effect at 704K can be tested directly.

What happened

E21i had shown this 4B model failing two of three multikey trials at 704K with the question only at the end, even though the same context fits on the card. E21n then showed that also asking the question before the document fixed most failures at 512K. Here the question-first prompt held up at 704K on ten fresh trials: 9 of 10 correct, with one decoy pick. So with this prompt, the model's usable context on this task reaches about the largest context it can hold on a 10 GB card. The memory saving that makes this context size fit is upstream llama.cpp PR 29827, so credit goes to its authors. The 1M target still needs a new way to shrink the cache, because 704K is the limit with a 4-bit cache.

How to rerun

powershell -ExecutionPolicy Bypass -File e21o_chain.ps1

Code

All code for E21O: e21o_chain.ps1, results.jsonl.

Prior work

No novelty claim. Stating the question before a long context is a known prompting technique; this is a measurement of it on a standard RULER-style multikey needle test.

Written by Andrew Rose with help from AI tools. Every number comes from the run logs.