# E22 code and measurements

- qcap.py: real-query attention capture. Loads Qwen3.5-4B (HF bf16 weights as bitsandbytes NF4, compute bf16), prefills WikiText-2 raw test text in chunks of 2048, captures post-norm, post-RoPE queries at 64 positions spread from 1024 to ctx-1 for all 16 query heads on the 8 full-attention layers (3, 7, ..., 31), and computes attention over the full causal prefix.
- For each of the 8,192 rows (64 positions x 16 heads x 8 layers) it records the fraction and count of keys needed for 90% and 99% of the mass, top-1/8/64 mass, self mass, first-4-token mass and last-64-token mass.
- attn_concentration_real_*.json: the summary output for the 4K and 32K runs.
- GPU memory was capped at 6 GiB; shared GPU memory was logged during the runs and stayed at its idle baseline (no spill). The 64K run hit a CUDA out-of-memory error; 128K was not attempted.

Run: python qcap.py <ctx>   (for example 4096 or 32768). Needs torch, transformers, bitsandbytes, datasets.