Running Qwen3.8 9B on a 16 GB M4: 36.4 tok/s Decode and a 57x Prefill Cut
Which quantisation fits 16 GB, why prefill is already at a hardware limit, and the two defects that stop prompt caching from working. Prefill on a 4,312 token prompt falls …

Tag archive
Notes connected by the Quantisation tag.
Which quantisation fits 16 GB, why prefill is already at a hardware limit, and the two defects that stop prompt caching from working. Prefill on a 4,312 token prompt falls …
No entries are filed here yet.
Browse all notes →