A tiny draft model made Q6_K about 35% faster

Speculative decoding with Qwen2.5-0.5B as the draft took Q6_K from 14.13 to 19.09 tok/s on my RTX 3080. Quality held. N-gram guessing did nothing, so it's dropped.

Round E04. Same setup as always: Qwen2.5-14B on one RTX 3080, running in llama.cpp.

First I wanted a cleaner quality check on Q6_K against full-size F16.

Test	Q6_K	F16
MMLU (500 questions)	0.772	0.770
GSM8K (32 problems)	0.6875	0.6875
Speed	14.13 tok/s	1.98 tok/s

So Q6_K scores the same as F16 here and runs about 7× faster on this card. Good baseline.

Then I tried speculative decoding. To be clear, this isn't mine. It's a feature already built into llama.cpp. A small model (Qwen2.5-0.5B at Q8_0) guesses the next few tokens, and the big model checks them all in one pass. When the guesses are right you get several tokens for the price of one.

Setup	Speed	GSM8K (32)
Q6_K alone	14.13 tok/s	0.6875
Q6_K + 0.5B draft	19.09 tok/s	0.71875

About 35% faster. The GSM8K bump is one problem out of 32, so I'm calling that the same, not better.

Why I care: more tokens per second from the same GPU means lower cost per token. This was measured on a regular gaming PC, so it's a real option for cheap serving today, no new hardware.

I also tried n-gram self-speculation, where it guesses from text already in the context instead of using a draft model. 13.72 tok/s, with only about 0 to 3% of guesses accepted. No gain, so it's dropped.