Benchmark run
Catch-up run: recordings 401 to 950, 13 models
Partial run that extends 13 Whisper and Parakeet models from 400 to 950 recordings per language. The pooled results are on the benchmarks page.
Sep 18, 2026
Run note: Clips 401-950, 13 multilingual models, matching the Flow comparison depth
- Models
- 13
- Samples per dataset
- 550
- Warmup runs
- 3
- Hardware
- Apple M4 Max / 36 GB / macOS 26.6.2
Full results
Accuracy is the share of words transcribed correctly per language condition, so higher is better; speed is milliseconds per second of audio, so lower is better. The best value in each column is emphasised, and a cell reading a dash says on hover why there is no figure in it.
Accuracy (%) per language, and speed in ms per second of audio, so lower is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 | 502 | 3.0 GB | 3.5 GB | 149 ms | 92.8%Every word from every language counted together, not an average of per-language scores. | 96.1%Every word from every language counted together, not an average of per-language scores. | 90.7%Every word from every language counted together, not an average of per-language scores. | 97.2% | 95.0% | 97.0% | 87.8% | 85.7% |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Whisper Large V3 Q5_0 | 502 | 1.1 GB | 1.5 GB | 98 ms | 92.7%Every word from every language counted together, not an average of per-language scores. | 96.1%Every word from every language counted together, not an average of per-language scores. | 90.7%Every word from every language counted together, not an average of per-language scores. | 97.2% | 94.9% | 97.0% | 87.7% | 85.8% |
| Whisper Large V3 Turbo Q8_0 | 502 | 834 MB | 1.1 GB | 65 ms | 92.1%Every word from every language counted together, not an average of per-language scores. | 95.8%Every word from every language counted together, not an average of per-language scores. | 89.8%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.6% | 96.8% | 87.1% | 83.6% |
| Whisper Large V3 Turbo | 502 | 1.6 GB | 1.9 GB | 82 ms | 92.1%Every word from every language counted together, not an average of per-language scores. | 95.8%Every word from every language counted together, not an average of per-language scores. | 89.8%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.5% | 96.8% | 87.1% | 83.7% |
| Whisper Large V3 Turbo Q5_0 | 502 | 574 MB | 781 MB | 59 ms | 91.8%Every word from every language counted together, not an average of per-language scores. | 95.7%Every word from every language counted together, not an average of per-language scores. | 89.4%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.3% | 96.6% | 86.5% | 83.3% |
| Whisper Large V2 | 502 | 3.0 GB | 3.5 GB | 148 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.5%Every word from every language counted together, not an average of per-language scores. | 88.4%Every word from every language counted together, not an average of per-language scores. | 96.6% | 94.2% | 96.5% | 85.6% | 80.9% |
| Whisper Large V2 Q8_0 | 502 | 1.6 GB | 2.1 GB | 110 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.4%Every word from every language counted together, not an average of per-language scores. | 88.4%Every word from every language counted together, not an average of per-language scores. | 96.6% | 94.1% | 96.5% | 85.6% | 80.9% |
| Whisper Large V2 Q5_0 | 502 | 1.1 GB | 1.5 GB | 97 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.5%Every word from every language counted together, not an average of per-language scores. | 88.3%Every word from every language counted together, not an average of per-language scores. | 96.7% | 94.1% | 96.6% | 85.4% | 80.9% |
| Parakeet TDT 0.6B v3 | 502 | 500 MB | 80 MB | 7 ms | 89.8%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 86.6%Every word from every language counted together, not an average of per-language scores. | 96.3% | 93.6% | 95.8% | 80.3% | 81.9% |
| Whisper Large V1 | 502 | 3.0 GB | 3.5 GB | 149 ms | 89.5%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 86.1%Every word from every language counted together, not an average of per-language scores. | 96.5% | 93.3% | 96.1% | 82.4% | 77.2% |
| Whisper Medium | 502 | 1.5 GB | 1.9 GB | 85 ms | 88.1%Every word from every language counted together, not an average of per-language scores. | 94.9%Every word from every language counted together, not an average of per-language scores. | 84.0%Every word from every language counted together, not an average of per-language scores. | 96.6% | 93.1% | 96.1% | 79.3% | 73.5% |
| Whisper Medium Q8_0 | 502 | 766 MB | 1.2 GB | 65 ms | 88.1%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 83.9%Every word from every language counted together, not an average of per-language scores. | 96.6% | 93.1% | 96.1% | 79.1% | 73.4% |
| Whisper Medium Q5_0 | 502 | 514 MB | 880 MB | 59 ms | 88.0%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 83.7%Every word from every language counted together, not an average of per-language scores. | 96.7% | 93.1% | 96.1% | 78.8% | 73.0% |
At a glance
Ratings computed from benchmark data, scaled 1 to 10, and written with their scale in every cell. Accuracy is based on Word Error Rate (WER) and does not include punctuation yet.
Ratings run 1 to 10, higher is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 Q5_0 | All languages | Yes | 7/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
|---|---|---|---|---|
| Whisper Large V3 | All languages | Yes | 6/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q5_0 | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Parakeet TDT 0.6B v3 | 25 languages | No | 10/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V1 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q5_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q8_0 | All languages | Yes | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q8_0 | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q5_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q8_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
Charts
The plots generated alongside this run. Select a chart to open it full size.


