Is it where the question sits, or that it is asked twice? Multikey needle, 4B model, 256K
On the same eight 256K-token documents, asking the question at both ends found the right passcode 8 of 8 times, asking it only at the end 6 of 8, and asking it only at the start 2 of 8. A same-length neutral instruction at the start changed nothing (6 of 8). So the question has to be at the end, and repeating it at the start helped in two trials, which is too few to call at this sample size. The earlier question-first gains were really a both-ends versus end-only comparison.
| Experiment | E22P |
| Date | 2026-10-30 |
| Benchmark | RULER-style multikey needle: one target passcode sentence plus three decoy passcodes for other vault colors, in shuffled paragraphs from the MATH training set, 253K to 259K tokens per prompt, target depth spread from 10% to 90%. Greedy decoding, 64-token answer limit. A run passes only if the first code in the answer is the target code. |
| Model | Qwen3.5-4B, Q4_K_M GGUF, native 262,144-token context, flash attention on, K and V cache at q4_0, ubatch 64 |
| Hardware | One of my dedicated test rigs: RTX 3080 10 GB |
| Verdict | BASELINE |
What I tested
Four prompt layouts on the same document in each of 8 trials: (1) question at the end only, (2) question at the start only, (3) the same question at both ends, (4) a neutral instruction with the same token count as the question at the start, plus the question at the end. Arm order was randomized per trial and each request ran on a freshly started server, so no cache was reused. Design suggested by u/magicdoorai on Reddit, to separate the position of the question from repeating it.
Pass bar, set before the run
Set before the run: position matters if start-only beats end-only by at least 3 of 8; repetition matters if both-ends beats start-only by at least 3 of 8. The neutral-instruction arm controls for the extra tokens at the start. With n=8 this is a screen; any separation gets a confirm run at n=16.
Result
| Layout | Passed | 95% interval (exact binomial) | Failures |
|---|---|---|---|
| Question at both ends | 8 of 8 | 63% to 100% | none |
| Question at the end only | 6 of 8 | 35% to 97% | picked a decoy at depth 30% and 20% |
| Same-length neutral instruction at start, question at end | 6 of 8 | 35% to 97% | picked a decoy in the same two trials |
| Question at the start only | 2 of 8 | 3% to 65% | 6 answers with no passcode at all (a made-up number) |
Position: start-only lost to end-only by 4 of 8, the opposite of the position hypothesis.
Repetition: both-ends beat start-only by 6 of 8, which clears the pre-set bar, but most of that is the question being at the end at all. Against end-only, both-ends gained 2 of 8 (the two decoy trials), below the 3 of 8 screen.
The neutral filler arm matched end-only trial for trial, so the extra tokens at the start do not explain anything.
This changes how I read my earlier question-first posts (E21n to E21p): those compared both-ends against end-only, so the gain there was from adding the question at the start on top of the end, not from moving it.
Peak GPU memory was about 5.46 GB with at most 114 MiB in shared system memory, so the cache stayed on the GPU. Each request took about 3.3 minutes, 105 minutes in total.
I could not score whether the model stopped cleanly after the answer, because this llama-server build does not return the end-of-sequence flag. Later runs log the stop reason.
Per-trial records, including the answers, are in results.jsonl in the code folder.
What happened
All 32 requests ran without errors on the dedicated test rig. The result corrects my own earlier framing: asking the question first is not what helps. The question needs to be at the end, and a copy at the start may add a little on top, which a confirm run at n=16 will check. Thanks to u/magicdoorai for the design.
How to rerun
powershell -ExecutionPolicy Bypass -File e22p_chain.ps1Code
All code for E22P: e22p_chain.ps1, results.jsonl.
Prior work
No novelty claim. Where to put the question in a long prompt is a known prompting question; this is a paired control measurement on a standard RULER-style multikey needle test.
Written by Andrew Rose with help from AI tools. Every number comes from the run logs.
Questions or ideas? me@andrewroseit.com or @AndrewRoseIT.