Benchmark run
Tiny base triage
Triage of Tiny and Base model families (all quantization and language variants) to evaluate whether these smaller models are worth further benchmarking.
May 9, 2026
- Models
- 12
- Samples per dataset
- 50
- Warmup runs
- 3
- Hardware
- Apple M4 Max / 36 GB / macOS 26.4.1
Full results
Accuracy is the share of words transcribed correctly per language condition, so higher is better; speed is milliseconds per second of audio, so lower is better. The best value in each column is emphasised, and a cell reading a dash says on hover why there is no figure in it.
Accuracy (%) per language, and speed in ms per second of audio, so lower is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Base | 47 | 142 MB | 334 MB | 57 ms | 70.4%Every word from every language counted together, not an average of per-language scores. | 91.1%Every word from every language counted together, not an average of per-language scores. | 58.8%Every word from every language counted together, not an average of per-language scores. | 91.8% | 90.6% | 88.4% | 39.5% | 35.3% |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Whisper Base Q8_0 | 47 | 78 MB | 247 MB | 57 ms | 70.3%Every word from every language counted together, not an average of per-language scores. | 91.0%Every word from every language counted together, not an average of per-language scores. | 58.6%Every word from every language counted together, not an average of per-language scores. | 92.4% | 90.1% | 88.6% | 38.4% | 35.5% |
| Whisper Base Q5_1 | 47 | 57 MB | 219 MB | 57 ms | 70.1%Every word from every language counted together, not an average of per-language scores. | 90.2%Every word from every language counted together, not an average of per-language scores. | 58.8%Every word from every language counted together, not an average of per-language scores. | 90.8% | 89.9% | 88.9% | 37.6% | 36.4% |
| Whisper Tiny | 47 | 75 MB | 224 MB | 57 ms | 59.9%Every word from every language counted together, not an average of per-language scores. | 86.9%Every word from every language counted together, not an average of per-language scores. | 44.7%Every word from every language counted together, not an average of per-language scores. | 87.6% | 86.4% | 85.0% | 14.2% | 16.9% |
| Whisper Tiny Q8_0 | 47 | 42 MB | 174 MB | 57 ms | 59.6%Every word from every language counted together, not an average of per-language scores. | 86.9%Every word from every language counted together, not an average of per-language scores. | 44.1%Every word from every language counted together, not an average of per-language scores. | 87.2% | 86.8% | 85.0% | 11.2% | 17.9% |
| Whisper Tiny Q5_1 | 47 | 31 MB | 157 MB | 58 ms | 58.4%Every word from every language counted together, not an average of per-language scores. | 86.4%Every word from every language counted together, not an average of per-language scores. | 42.6%Every word from every language counted together, not an average of per-language scores. | 87.1% | 86.0% | 83.5% | 14.4% | 11.7% |
| Whisper Base Q5_1 en | 47 | 57 MB | 217 MB | 58 ms | 30.7%Every word from every language counted together, not an average of per-language scores. | 92.3%Every word from every language counted together, not an average of per-language scores. | -4.2%Every word from every language counted together, not an average of per-language scores. | 92.7% | 92.1% | 3.8% | -9.6% | -10.2% |
| Whisper Base Q8_0 en | 47 | 78 MB | 247 MB | 58 ms | 30.6%Every word from every language counted together, not an average of per-language scores. | 92.0%Every word from every language counted together, not an average of per-language scores. | -4.2%Every word from every language counted together, not an average of per-language scores. | 92.4% | 91.8% | 3.2% | -7.9% | -10.9% |
| Whisper Base en | 47 | 142 MB | 333 MB | 59 ms | 29.7%Every word from every language counted together, not an average of per-language scores. | 92.3%Every word from every language counted together, not an average of per-language scores. | -5.7%Every word from every language counted together, not an average of per-language scores. | 93.1% | 91.8% | 0.8% | -7.0% | -13.8% |
| Whisper Tiny en | 47 | 75 MB | 224 MB | 59 ms | 23.5%Every word from every language counted together, not an average of per-language scores. | 88.2%Every word from every language counted together, not an average of per-language scores. | -13.1%Every word from every language counted together, not an average of per-language scores. | 88.9% | 87.8% | -1.2% | -18.7% | -24.4% |
| Whisper Tiny Q5_1 en | 47 | 31 MB | 157 MB | 59 ms | 22.4%Every word from every language counted together, not an average of per-language scores. | 88.0%Every word from every language counted together, not an average of per-language scores. | -14.7%Every word from every language counted together, not an average of per-language scores. | 88.8% | 87.5% | -1.0% | -26.0% | -23.2% |
| Whisper Tiny Q8_0 en | 47 | 42 MB | 173 MB | 58 ms | 21.8%Every word from every language counted together, not an average of per-language scores. | 88.5%Every word from every language counted together, not an average of per-language scores. | -15.9%Every word from every language counted together, not an average of per-language scores. | 89.1% | 88.1% | -2.3% | -20.7% | -30.7% |
At a glance
Ratings computed from benchmark data, scaled 1 to 10, and written with their scale in every cell. Accuracy is based on Word Error Rate (WER) and does not include punctuation yet.
Ratings run 1 to 10, higher is better. A cell reading a dash was never measured, and hovering any cell says what is behind it.
| Whisper Base | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
|---|---|---|---|---|
| Whisper Base Q5_1 | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q8_0 | All languages | Yes | 9/10 | 5/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q5_1 | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q8_0 | All languages | Yes | 9/10 | 3/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q5_1 en | English only | No | 9/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Base Q8_0 en | English only | No | 9/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q5_1 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
| Whisper Tiny Q8_0 en | English only | No | 8/10 | 1/10Every word from every language counted together, not an average of per-language scores. |
Charts
The plots generated alongside this run. Select a chart to open it full size.
