For most of Codictate's life, Whisper transcription meant one thing: shell out to whisper-cli, the reference command line tool from whisper.cpp. It worked. It was also the only thing that binary could ever do.
We have replaced it with CrispASR, a whisper.cpp fork that ships as a single prebuilt binary with 107 compiled backends behind one --backend flag. On an M4 Max it was faster in 9 out of 9 comparisons, median 1.58x, and used up to 26% less peak memory at the same accuracy.
The speed is nice. The backends are the actual reason.
The same model, a different process running it
Worth separating two things that are easy to conflate. The Speech Engine is Whisper. The ASR Harness is the process that loads the weights and runs inference. Swapping the harness does not change which model you picked in Settings, does not re-download anything, and does not move a single byte off your machine. Same GGML weights, same flags, same local-only pipeline.
whisper-cli runs Whisper. That is the whole feature set. Point it at weights that are not whisper.cpp GGML and it exits.
CrispASR takes the same flags Codictate was already passing (-m -t --language -f --no-prints -nt), returns the same stdout shape, and adds --backend. That flag is what turns "add support for a new model family" from a packaging project into a string.
Speed: faster in 9 of 9, median 1.58x
Everything below ran on an Apple M4 Max with 36 GB of RAM, macOS 26.5.1. Three models (large-v3-q5_0, large-v3-turbo-q5_0, medium.en-q5_0) across three conditions: English clean and English noisy from LibriSpeech test-clean and test-other, plus Spanish from FLEURS es_419. 17 scored utterances per combination, 3 warmup runs before measurement.
Numbers are milliseconds of processing per second of audio, whisper-cli first.
| Model | English (clean) | English (noisy) | Spanish |
|---|---|---|---|
| Large v3 q5_0 | 149 to 109 ms (1.37x) | 209 to 140 ms (1.49x) | 130 to 83 ms (1.57x) |
| Large v3 Turbo q5_0 | 117 to 74 ms (1.58x) | 137 to 97 ms (1.41x) | 88 to 51 ms (1.73x) |
| Medium English q5_0 | 102 to 63 ms (1.62x) | 147 to 82 ms (1.79x) | 73 to 46 ms (1.59x) |
Nine comparisons, nine wins, ranging from 1.36x to 1.80x. There is no cell where the old harness came out ahead.
None of these models were anywhere near falling behind real time to begin with. Large v3 at 149 ms per second of audio has plenty of headroom. What the extra margin buys you is a shorter wait between releasing the shortcut and seeing text appear, and less work done on battery for the same sentence.
Memory: up to 26% off peak RSS
| Model | whisper-cli | CrispASR | Change |
|---|---|---|---|
| Large v3 q5_0 | 1,990 MB | 1,505 MB | -24% |
| Medium English q5_0 | 1,124 MB | 833 MB | -26% |
| Large v3 Turbo q5_0 | 802 MB | 738 MB | -8% |
Average peak resident set size. The pattern is that the bigger the model, the more the new harness saves, which is the useful direction. Large v3 dropping half a gigabyte matters on a 16 GB machine and matters a lot on an 8 GB one. Turbo, already the lean option, only picks up 64 MB.
Accuracy: unchanged, and I am not going to dress that up
Mean word error rate came out at 4.68% for CrispASR against 4.78% for whisper-cli. That is a gap of 0.11 percentage points on a sample of 17 utterances per combination.
That is not an improvement. It is parity, and 17 utterances is a small enough sample that I would not defend the direction of that gap either. Broken down pair by pair, CrispASR was better on 2, level on 3, and worse on 3. (Medium English on Spanish is not a real comparison in either direction. An English-only model transcribing Spanish produces garbage whichever harness runs it, so it is excluded from that count.)
The honest summary is: we got a speed and memory win for free, and accuracy stayed where it was. If accuracy had moved meaningfully in either direction on this sample size, the first thing I would have suspected is the benchmark.
The real payoff: --backend cohere and a Danish model nothing else can load
Danish has been the weak spot in every benchmark we have published. In our full model benchmark, Whisper Large managed 87.1% on Danish while Small dropped to 64.3%. In the Tiny and Base triage, Base hit 39.5% and Tiny 14.2%. If you dictate in Danish, the only workable answer so far has been "run the biggest model you can afford."
syvai built something better. syvai/hviske-v5-tiny is a Danish speech model whose card reports Danish WER around 10.5, at tiny-model size. The quantizations run roughly 39x realtime for f16 and 56x realtime for q4_k on an M4.
Here is the catch that makes this post a story about harnesses rather than about a model. Those weights are GGUF, and they load under exactly one thing: CrispASR's cohere backend. Not whisper.cpp. Not llama.cpp.
Under the old setup, offering that model would have meant vendoring a second ASR binary, codesigning and notarizing it, and wiring up a second process path with its own flags, its own error handling and its own failure modes. Under CrispASR it is one more argument on a command line we already build:
crispasr --backend cohere -m hviske-v5-tiny-f16.gguf -f audio.wav --language da
That is the difference between a project and a parameter.
One licensing note, since it shapes what we can do: hviske-v5-tiny is CC BY-NC 4.0, non-commercial. Codictate is free, has no paid tier, and is not going to get one, which is what makes non-commercial use of the model appropriate here rather than a problem to argue about later.
What we did not choose
Three options were on the table, and two of them were worse for reasons worth writing down.
Add CrispASR only for hviske and keep whisper-cli for Whisper. The conservative choice, and the one that quietly costs the most. Two ASR binaries to vendor, sign and notarize forever, two code paths to debug, and no evidence either way about which one is actually better.
Replace whisper-cli outright on faith. Tempting, because it verified as a drop-in immediately: a single 16 MB arm64 binary ran the existing ggml-large-v3-turbo-q5_0.bin with Codictate's exact flag set, exit 0, identical output shape. But "it launched and looked right" is not the same as "it transcribes as well," and the shipping transcription path is not the place to find out.
Implement both, decide with the benchmark. This is what we did. The harness became a dimension in the benchmark runner alongside model and dataset, so "already benchmarked" is now a property of a harness-model-dataset combination rather than of a model. CrispASR sat behind a dev-only flag with no user-facing picker until the numbers came in. If it had lost, nothing user-visible would have needed walking back.
There is a build-time bonus that had nothing to do with the benchmark. Upstream whisper.cpp publishes no macOS CLI binary, only Windows, Ubuntu, and an xcframework, so whisper-cli had to be compiled from source: 3 to 5 minutes on a fresh clone or CI run, plus a cmake dependency on every contributor's machine. CrispASR publishes prebuilt binaries for both macOS and Windows. That build step is gone.
Known limitations
Small and Tiny are unmeasured. Three models were tested, all of them medium or large. Whether the speed advantage holds at the small end, where per-run overhead is a bigger share of total time, is an open question.
Windows is unmeasured. CrispASR ships Windows binaries and Codictate builds both platforms in the same pass, but every number here is Apple Silicon.
CrispASR is a young project with essentially one maintainer. It is on the 0.8.x line and moving fast. That is a real dependency risk on the shipping transcription path, and it is the single strongest argument that the cautious option was the right one. We took it anyway, because the benchmark said the code is good.
Nothing about this changes the part that matters. Audio never leaves your machine, there is no account, and there are no analytics. We just replaced the local process that reads it.
The benchmark data behind these numbers is on the benchmarks page. CrispASR is at github.com/CrispStrobe/CrispASR, and if you work with Danish audio, go look at syvai's models. That work deserves the attention more than our command line flag does.