Running Qwen3.8 9B on a 16 GB M4: 36.4 tok/s Decode and a 57x Prefill Cut
Which quantisation fits 16 GB, why prefill is already at a hardware limit, and the two defects that stop prompt caching from working. Prefill on a 4,312 token prompt falls …
Tag archive
#Profiling
Notes connected by the Profiling tag.
4
entries
→
Cutting a Cold cargo check From 23.77s to 0.47s on an M4
A cold cargo check of rust-analyzer goes from 23.77s to 0.47s. Three changes across cargo and rustc, what each one bought, and how two of them ship as a separate tool that …
Taking the 2B Qwen3-TTS from 0.876 RTF to 0.253 RTF on Apple Silicon M4
A day of profiling a two-billion-parameter speech model on an M4 MacBook Pro, told as six wrong predictions. The largest win was nine lines of sorting, and one of my two …
Tuning CosyVoice3 in Rust: Where the Speed Actually Was
The same speech model made fast a second time, in Rust on Metal, from RTF 2.96 to 0.697. Scheduling beat kernels, and two of the measurements I steered by were wrong.
No entries are filed here yet.
Browse all notes →