https://andrewroseit.com/Independent AI researcher. I want AI cheap enough to run that it can actually make companies profitable, from one gaming GPU up to data centers. Every result posted, failures included.enhttps://andrewroseit.com/posts/op-offload/https://andrewroseit.com/posts/op-offload/Tue, 27 Oct 2026 12:00:00 +0000Turning off op offload took Qwen2.5-14B Q6_K from 60.2 to 135.0 tok/s at 32 users on my RTX 3080. Single user didn't change. It's a setting, not a new method.https://andrewroseit.com/posts/spec-decoding/https://andrewroseit.com/posts/spec-decoding/Tue, 27 Oct 2026 12:00:00 +0000Speculative decoding with Qwen2.5-0.5B as the draft took Q6_K from 14.13 to 19.09 tok/s on my RTX 3080. Quality held. N-gram guessing did nothing, so it's dropped.https://andrewroseit.com/posts/32-user-dip/https://andrewroseit.com/posts/32-user-dip/Tue, 27 Oct 2026 12:00:00 +0000Full-size F16 in llama.cpp gets 28.4 tok/s total at 16 users, 19.1 at 32, then 35.3 at 64. That dip shouldn't be there.https://andrewroseit.com/posts/q6k-baseline/https://andrewroseit.com/posts/q6k-baseline/Mon, 26 Oct 2026 12:00:00 +0000llama.cpp's Q6_K runs Qwen2.5-14B at 13.28 tok/s on my RTX 3080 and only loses 0.72% quality. That's the bar now.https://andrewroseit.com/posts/ssx-dropped/https://andrewroseit.com/posts/ssx-dropped/Mon, 26 Oct 2026 12:00:00 +0000My exact-correction idea matched the full model's quality exactly. It was also way too slow, so it's gone.https://andrewroseit.com/posts/dropped-ideas/https://andrewroseit.com/posts/dropped-ideas/Mon, 26 Oct 2026 12:00:00 +0000Before building anything I search the papers. Several of my ideas were already out there, so I dropped them.