Benchmark run
34 Whisper and Parakeet models: the screening run
Every Whisper size from Tiny to Large V3 Turbo, quantized and full precision, plus Parakeet TDT 0.6B v3, on 50 recordings per language on an Apple M4 Max.
May 9, 2026
Run note: Initial model benchmarks to see if there are some models that we should leave out for a more extensive benchmark that will include way more samples. This does not include the Tiny/Base models which has been tested initially already
- Models
- 34
- Samples per dataset
- 50
- Warmup runs
- 3
- Hardware
- Apple M4 Max / 36 GB / macOS 26.4.1
Full results
Accuracy is the share of words transcribed correctly per language condition, so higher is better; speed is milliseconds per second of audio, so lower is better. The best value in each column is emphasised, and a cell reading a dash says on hover why there is no figure in it.
Accuracy (%) per language, and speed in ms per second of audio, so lower is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 | 47 | 3.0 GB | 4.0 GB | 183 ms | 92.9%Every word from every language counted together, not an average of per-language scores. | 94.8%Every word from every language counted together, not an average of per-language scores. | 91.8%Every word from every language counted together, not an average of per-language scores. | 96.3% | 93.8% | 96.5% | 87.3% | 89.5% |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Whisper Large V3 Q5_0 | 47 | 1.1 GB | 2.0 GB | 146 ms | 92.5%Every word from every language counted together, not an average of per-language scores. | 94.6%Every word from every language counted together, not an average of per-language scores. | 91.4%Every word from every language counted together, not an average of per-language scores. | 96.3% | 93.6% | 96.5% | 86.9% | 88.3% |
| Whisper Large V3 Turbo Q8_0 | 47 | 834 MB | 1.1 GB | 108 ms | 91.9%Every word from every language counted together, not an average of per-language scores. | 95.1%Every word from every language counted together, not an average of per-language scores. | 90.2%Every word from every language counted together, not an average of per-language scores. | 95.5% | 94.8% | 96.7% | 83.2% | 87.5% |
| Whisper Large V3 Turbo Q5_0 | 47 | 574 MB | 800 MB | 101 ms | 91.9%Every word from every language counted together, not an average of per-language scores. | 94.8%Every word from every language counted together, not an average of per-language scores. | 90.3%Every word from every language counted together, not an average of per-language scores. | 95.1% | 94.6% | 96.8% | 84.5% | 86.5% |
| Whisper Large V3 Turbo | 47 | 1.6 GB | 1.9 GB | 109 ms | 91.9%Every word from every language counted together, not an average of per-language scores. | 95.0%Every word from every language counted together, not an average of per-language scores. | 90.1%Every word from every language counted together, not an average of per-language scores. | 95.4% | 94.7% | 96.7% | 83.2% | 87.4% |
| Whisper Large V2 Q8_0 | 47 | 1.6 GB | 2.5 GB | 154 ms | 91.5%Every word from every language counted together, not an average of per-language scores. | 94.9%Every word from every language counted together, not an average of per-language scores. | 89.6%Every word from every language counted together, not an average of per-language scores. | 95.0% | 94.9% | 96.1% | 84.6% | 85.2% |
| Whisper Large V2 | 47 | 3.0 GB | 4.0 GB | 185 ms | 91.5%Every word from every language counted together, not an average of per-language scores. | 94.8%Every word from every language counted together, not an average of per-language scores. | 89.6%Every word from every language counted together, not an average of per-language scores. | 95.0% | 94.7% | 96.1% | 84.5% | 85.3% |
| Whisper Large V2 Q5_0 | 47 | 1.1 GB | 2.0 GB | 146 ms | 91.4%Every word from every language counted together, not an average of per-language scores. | 94.9%Every word from every language counted together, not an average of per-language scores. | 89.4%Every word from every language counted together, not an average of per-language scores. | 95.3% | 94.6% | 96.1% | 83.9% | 85.3% |
| Whisper Large V1 | 47 | 3.0 GB | 4.0 GB | 184 ms | 89.9%Every word from every language counted together, not an average of per-language scores. | 94.1%Every word from every language counted together, not an average of per-language scores. | 87.5%Every word from every language counted together, not an average of per-language scores. | 94.7% | 93.7% | 95.8% | 80.7% | 82.2% |
| Parakeet TDT 0.6B v3 | 47 | 500 MB | 80 MB | 19 ms | 89.4%Every word from every language counted together, not an average of per-language scores. | 93.9%Every word from every language counted together, not an average of per-language scores. | 86.8%Every word from every language counted together, not an average of per-language scores. | 95.7% | 92.7% | 94.4% | 78.0% | 84.5% |
| Whisper Medium | 47 | 1.5 GB | 2.1 GB | 116 ms | 88.5%Every word from every language counted together, not an average of per-language scores. | 94.0%Every word from every language counted together, not an average of per-language scores. | 85.4%Every word from every language counted together, not an average of per-language scores. | 94.8% | 93.4% | 95.2% | 78.2% | 78.3% |
| Whisper Medium Q8_0 | 47 | 766 MB | 1.4 GB | 106 ms | 88.3%Every word from every language counted together, not an average of per-language scores. | 94.0%Every word from every language counted together, not an average of per-language scores. | 85.1%Every word from every language counted together, not an average of per-language scores. | 95.0% | 93.4% | 95.2% | 77.9% | 77.9% |
| Whisper Medium Q5_0 | 47 | 514 MB | 1.1 GB | 99 ms | 88.3%Every word from every language counted together, not an average of per-language scores. | 94.1%Every word from every language counted together, not an average of per-language scores. | 85.0%Every word from every language counted together, not an average of per-language scores. | 95.3% | 93.3% | 95.2% | 77.1% | 78.2% |
| Whisper Small | 47 | 465 MB | 807 MB | 59 ms | 82.1%Every word from every language counted together, not an average of per-language scores. | 92.9%Every word from every language counted together, not an average of per-language scores. | 76.0%Every word from every language counted together, not an average of per-language scores. | 94.3% | 92.0% | 94.0% | 64.2% | 61.8% |
| Whisper Small Q8_0 | 47 | 252 MB | 558 MB | 58 ms | 81.9%Every word from every language counted together, not an average of per-language scores. | 92.9%Every word from every language counted together, not an average of per-language scores. | 75.7%Every word from every language counted together, not an average of per-language scores. | 94.1% | 92.1% | 94.0% | 63.9% | 61.0% |
| Whisper Small Q5_1 | 47 | 181 MB | 476 MB | 58 ms | 81.7%Every word from every language counted together, not an average of per-language scores. | 93.3%Every word from every language counted together, not an average of per-language scores. | 75.2%Every word from every language counted together, not an average of per-language scores. | 94.5% | 92.4% | 93.8% | 63.6% | 59.8% |
| Whisper Base | 47 | 142 MB | 334 MB | 57 ms | 70.4%Every word from every language counted together, not an average of per-language scores. | 91.1%Every word from every language counted together, not an average of per-language scores. | 58.8%Every word from every language counted together, not an average of per-language scores. | 91.8% | 90.6% | 88.4% | 39.5% | 35.3% |
| Whisper Base Q8_0 | 47 | 78 MB | 247 MB | 57 ms | 70.3%Every word from every language counted together, not an average of per-language scores. | 91.0%Every word from every language counted together, not an average of per-language scores. | 58.6%Every word from every language counted together, not an average of per-language scores. | 92.4% | 90.1% | 88.6% | 38.4% | 35.5% |
| Whisper Base Q5_1 | 47 | 57 MB | 219 MB | 57 ms | 70.1%Every word from every language counted together, not an average of per-language scores. | 90.2%Every word from every language counted together, not an average of per-language scores. | 58.8%Every word from every language counted together, not an average of per-language scores. | 90.8% | 89.9% | 88.9% | 37.6% | 36.4% |
| Whisper Tiny | 47 | 75 MB | 224 MB | 57 ms | 59.9%Every word from every language counted together, not an average of per-language scores. | 86.9%Every word from every language counted together, not an average of per-language scores. | 44.7%Every word from every language counted together, not an average of per-language scores. | 87.6% | 86.4% | 85.0% | 14.2% | 16.9% |
| Whisper Tiny Q8_0 | 47 | 42 MB | 174 MB | 57 ms | 59.6%Every word from every language counted together, not an average of per-language scores. | 86.9%Every word from every language counted together, not an average of per-language scores. | 44.1%Every word from every language counted together, not an average of per-language scores. | 87.2% | 86.8% | 85.0% | 11.2% | 17.9% |
| Whisper Tiny Q5_1 | 47 | 31 MB | 157 MB | 58 ms | 58.4%Every word from every language counted together, not an average of per-language scores. | 86.4%Every word from every language counted together, not an average of per-language scores. | 42.6%Every word from every language counted together, not an average of per-language scores. | 87.1% | 86.0% | 83.5% | 14.4% | 11.7% |
| Whisper Base Q5_1 en | 47 | 57 MB | 217 MB | 58 ms | 30.7%Every word from every language counted together, not an average of per-language scores. | 92.3%Every word from every language counted together, not an average of per-language scores. | -4.2%Every word from every language counted together, not an average of per-language scores. | 92.7% | 92.1% | 3.8% | -9.6% | -10.2% |
| Whisper Base Q8_0 en | 47 | 78 MB | 247 MB | 58 ms | 30.6%Every word from every language counted together, not an average of per-language scores. | 92.0%Every word from every language counted together, not an average of per-language scores. | -4.2%Every word from every language counted together, not an average of per-language scores. | 92.4% | 91.8% | 3.2% | -7.9% | -10.9% |
| Whisper Base en | 47 | 142 MB | 333 MB | 59 ms | 29.7%Every word from every language counted together, not an average of per-language scores. | 92.3%Every word from every language counted together, not an average of per-language scores. | -5.7%Every word from every language counted together, not an average of per-language scores. | 93.1% | 91.8% | 0.8% | -7.0% | -13.8% |
| Whisper Medium en | 47 | 1.5 GB | 2.1 GB | 116 ms | 29.6%Every word from every language counted together, not an average of per-language scores. | 94.4%Every word from every language counted together, not an average of per-language scores. | -7.0%Every word from every language counted together, not an average of per-language scores. | 95.7% | 93.5% | 10.8% | -13.9% | -25.6% |
| Whisper Small en | 47 | 465 MB | 807 MB | 61 ms | 29.4%Every word from every language counted together, not an average of per-language scores. | 93.8%Every word from every language counted together, not an average of per-language scores. | -7.1%Every word from every language counted together, not an average of per-language scores. | 95.3% | 92.8% | 9.0% | -15.3% | -21.9% |
| Whisper Small Q8_0 en | 47 | 252 MB | 558 MB | 60 ms | 28.7%Every word from every language counted together, not an average of per-language scores. | 93.7%Every word from every language counted together, not an average of per-language scores. | -8.1%Every word from every language counted together, not an average of per-language scores. | 95.3% | 92.7% | 10.1% | -19.3% | -23.0% |
| Whisper Medium Q8_0 en | 47 | 766 MB | 1.4 GB | 103 ms | 28.5%Every word from every language counted together, not an average of per-language scores. | 94.1%Every word from every language counted together, not an average of per-language scores. | -8.6%Every word from every language counted together, not an average of per-language scores. | 95.7% | 93.0% | 10.0% | -16.3% | -27.4% |
| Whisper Medium Q5_0 en | 47 | 514 MB | 1.1 GB | 96 ms | 28.3%Every word from every language counted together, not an average of per-language scores. | 94.1%Every word from every language counted together, not an average of per-language scores. | -8.9%Every word from every language counted together, not an average of per-language scores. | 95.7% | 93.1% | 9.2% | -11.4% | -32.0% |
| Whisper Small Q5_1 en | 47 | 181 MB | 475 MB | 62 ms | 27.3%Every word from every language counted together, not an average of per-language scores. | 94.0%Every word from every language counted together, not an average of per-language scores. | -10.3%Every word from every language counted together, not an average of per-language scores. | 95.4% | 93.0% | 5.7% | -17.2% | -26.3% |
| Whisper Tiny en | 47 | 75 MB | 224 MB | 59 ms | 23.5%Every word from every language counted together, not an average of per-language scores. | 88.2%Every word from every language counted together, not an average of per-language scores. | -13.1%Every word from every language counted together, not an average of per-language scores. | 88.9% | 87.8% | -1.2% | -18.7% | -24.4% |
| Whisper Tiny Q5_1 en | 47 | 31 MB | 157 MB | 59 ms | 22.4%Every word from every language counted together, not an average of per-language scores. | 88.0%Every word from every language counted together, not an average of per-language scores. | -14.7%Every word from every language counted together, not an average of per-language scores. | 88.8% | 87.5% | -1.0% | -26.0% | -23.2% |
| Whisper Tiny Q8_0 en | 47 | 42 MB | 173 MB | 58 ms | 21.8%Every word from every language counted together, not an average of per-language scores. | 88.5%Every word from every language counted together, not an average of per-language scores. | -15.9%Every word from every language counted together, not an average of per-language scores. | 89.1% | 88.1% | -2.3% | -20.7% | -30.7% |
At a glance
Ratings computed from benchmark data, scaled 1 to 10, and written with their scale in every cell. Accuracy is based on Word Error Rate (WER) and does not include punctuation yet.
Ratings run 1 to 10, higher is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Large V3 Q5_0 | All languages | Yes | 6/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
|---|---|---|---|---|
| Whisper Large V3 | All languages | Yes | 5/10 | 10/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q5_0 | All languages | No | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Parakeet TDT 0.6B v3 | 25 languages | No | 10/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V1 | All languages | Yes | 5/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 | All languages | Yes | 5/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q5_0 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V2 Q8_0 | All languages | Yes | 6/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo | All languages | No | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Large V3 Turbo Q8_0 | All languages | No | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium | All languages | Yes | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q5_0 | All languages | Yes | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q8_0 | All languages | Yes | 7/10 | 9/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small Q5_1 | All languages | Yes | 8/10 | 7/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small | All languages | Yes | 8/10 | 7/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small Q8_0 | All languages | Yes | 8/10 | 7/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q5_1 | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q8_0 | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q5_1 | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q8_0 | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q5_1 en | English only | No | 9/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q8_0 en | English only | No | 9/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium en | English only | No | 7/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q5_0 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Medium Q8_0 en | English only | No | 7/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small Q5_1 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Small Q8_0 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q5_1 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q8_0 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
Charts
The plots generated alongside this run. Select a chart to open it full size.


