Benchmark run
2026 09 v4 multilingual catchup
Clips 401-950, 13 multilingual models, matching the Flow comparison depth
Sep 18, 2026
- Models
- 13
- Samples per dataset
- 550
- Warmup runs
- 3
- Hardware
- Apple M4 Max / 36 GB / macOS 26.6.2
Full results
Accuracy is the share of words transcribed correctly per language condition, so higher is better; speed is milliseconds per second of audio, so lower is better. The best value in each column is emphasised, and a cell reading a dash says on hover why there is no figure in it.
Accuracy (%) per language, and speed in ms per second of audio, so lower is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 | 502 | 3.0 GB | 3.5 GB | 149 ms | 92.8%Every word from every language counted together, not an average of per-language scores. | 96.1%Every word from every language counted together, not an average of per-language scores. | 90.7%Every word from every language counted together, not an average of per-language scores. | 97.2% | 95.0% | 97.0% | 87.8% | 85.7% |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Whisper Large V3 Q5_0 | 502 | 1.1 GB | 1.5 GB | 98 ms | 92.7%Every word from every language counted together, not an average of per-language scores. | 96.1%Every word from every language counted together, not an average of per-language scores. | 90.7%Every word from every language counted together, not an average of per-language scores. | 97.2% | 94.9% | 97.0% | 87.7% | 85.8% |
| Whisper Large V3 Turbo Q8_0 | 502 | 834 MB | 1.1 GB | 65 ms | 92.1%Every word from every language counted together, not an average of per-language scores. | 95.8%Every word from every language counted together, not an average of per-language scores. | 89.8%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.6% | 96.8% | 87.1% | 83.6% |
| Whisper Large V3 Turbo | 502 | 1.6 GB | 1.9 GB | 82 ms | 92.1%Every word from every language counted together, not an average of per-language scores. | 95.8%Every word from every language counted together, not an average of per-language scores. | 89.8%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.5% | 96.8% | 87.1% | 83.7% |
| Whisper Large V3 Turbo Q5_0 | 502 | 574 MB | 781 MB | 59 ms | 91.8%Every word from every language counted together, not an average of per-language scores. | 95.7%Every word from every language counted together, not an average of per-language scores. | 89.4%Every word from every language counted together, not an average of per-language scores. | 96.9% | 94.3% | 96.6% | 86.5% | 83.3% |
| Whisper Large V2 | 502 | 3.0 GB | 3.5 GB | 148 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.5%Every word from every language counted together, not an average of per-language scores. | 88.4%Every word from every language counted together, not an average of per-language scores. | 96.6% | 94.2% | 96.5% | 85.6% | 80.9% |
| Whisper Large V2 Q8_0 | 502 | 1.6 GB | 2.1 GB | 110 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.4%Every word from every language counted together, not an average of per-language scores. | 88.4%Every word from every language counted together, not an average of per-language scores. | 96.6% | 94.1% | 96.5% | 85.6% | 80.9% |
| Whisper Large V2 Q5_0 | 502 | 1.1 GB | 1.5 GB | 97 ms | 91.1%Every word from every language counted together, not an average of per-language scores. | 95.5%Every word from every language counted together, not an average of per-language scores. | 88.3%Every word from every language counted together, not an average of per-language scores. | 96.7% | 94.1% | 96.6% | 85.4% | 80.9% |
| Parakeet TDT 0.6B v3 | 502 | 500 MB | 80 MB | 7 ms | 89.8%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 86.6%Every word from every language counted together, not an average of per-language scores. | 96.3% | 93.6% | 95.8% | 80.3% | 81.9% |
| Whisper Large V1 | 502 | 3.0 GB | 3.5 GB | 149 ms | 89.5%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 86.1%Every word from every language counted together, not an average of per-language scores. | 96.5% | 93.3% | 96.1% | 82.4% | 77.2% |
| Whisper Medium | 502 | 1.5 GB | 1.9 GB | 85 ms | 88.1%Every word from every language counted together, not an average of per-language scores. | 94.9%Every word from every language counted together, not an average of per-language scores. | 84.0%Every word from every language counted together, not an average of per-language scores. | 96.6% | 93.1% | 96.1% | 79.3% | 73.5% |
| Whisper Medium Q8_0 | 502 | 766 MB | 1.2 GB | 65 ms | 88.1%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 83.9%Every word from every language counted together, not an average of per-language scores. | 96.6% | 93.1% | 96.1% | 79.1% | 73.4% |
| Whisper Medium Q5_0 | 502 | 514 MB | 880 MB | 59 ms | 88.0%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 83.7%Every word from every language counted together, not an average of per-language scores. | 96.7% | 93.1% | 96.1% | 78.8% | 73.0% |
At a glance
Ratings computed from benchmark data, scaled 1 to 10, and written with their scale in every cell. Accuracy is based on Word Error Rate (WER) and does not include punctuation yet.
Ratings run 1 to 10, higher is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 Q5_0 | All languages | Yes | 7/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
|---|---|---|---|---|
| Whisper Large V3 | All languages | Yes | 6/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q5_0 | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Parakeet TDT 0.6B v3 | 25 languages | No | 10/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V1 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q5_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q8_0 | All languages | Yes | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q8_0 | All languages | No | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q5_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q8_0 | All languages | Yes | 8/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
Charts
The plots generated alongside this run. Select a chart to open it full size.
