Running Qwen3.8 9B on a 16 GB M4: 36.4 tok/s Decode and a 57x Prefill Cut
Which quantisation fits 16 GB, why prefill is already at a hardware limit, and the two defects that stop prompt caching from working. Prefill on a 4,312 token prompt falls …
Category archive
Performance
Field notes collected under Performance.
7
entries
→
Cutting a Cold cargo check From 23.77s to 0.47s on an M4
A cold cargo check of rust-analyzer goes from 23.77s to 0.47s. Three changes across cargo and rustc, what each one bought, and how two of them ship as a separate tool that …
Taking the 2B Qwen3-TTS from 0.876 RTF to 0.253 RTF on Apple Silicon M4
A day of profiling a two-billion-parameter speech model on an M4 MacBook Pro, told as six wrong predictions. The largest win was nine lines of sorting, and one of my two …
Tuning CosyVoice3 in Rust: Where the Speed Actually Was
The same speech model made fast a second time, in Rust on Metal, from RTF 2.96 to 0.697. Scheduling beat kernels, and two of the measurements I steered by were wrong.
How I Made CosyVoice3 2.2× Faster on Apple Silicon
How I bounded PyTorch MPS memory growth, cut CosyVoice3 latency below real time, and moved only the dispatch-bound LLM stage to MLX.
Finding an 800ms Token Callout in a Gravitee API Gateway
A service token lived for more than ten hours, but our Gravitee gateway fetched it on every request. Here is how we measured the delay, tested the obvious cache fix, and found …
Making a Truly Fast Website
In this short piece, I want to share how you can make a website that performs well for users. Speed is very essential.
No entries are filed here yet.
Browse all notes →