Initial model benchmarks to see if there are some models that we should leave out for a more extensive benchmark that will include way more samples. This does not include the Tiny/Base models which has been tested initially already
May 9, 2026Apple M4 Max / 36 GB / macOS 26.4.134 models50 samples per dataset
| Model | Samples | Disk | RAM | Speed | Avg overall | Avg EN | Avg multi | EN | EN noisy | ES | DA | HU |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Large V3 | 50 | 3.0 GB | 4.0 GB | 183 ms | 92.7% | 95.0% | 91.1% | 96.3% | 93.8% | 96.5% | 87.3% | 89.5% |
| Large V3 q5_0 | 50 | 1.1 GB | 2.0 GB | 146 ms | 92.3% | 94.9% | 90.6% | 96.3% | 93.6% | 96.5% | 86.9% | 88.3% |
| Large V3 Turbo q8_0 | 50 | 834 MB | 1.1 GB | 108 ms | 91.6% | 95.2% | 89.2% | 95.5% | 94.8% | 96.7% | 83.2% | 87.5% |
| Large V3 Turbo q5_0 | 50 | 574 MB | 800 MB | 101 ms | 91.5% | 94.9% | 89.3% | 95.1% | 94.6% | 96.8% | 84.5% | 86.5% |
| Large V3 Turbo | 50 | 1.6 GB | 1.9 GB | 109 ms | 91.5% | 95.1% | 89.1% | 95.4% | 94.7% | 96.7% | 83.2% | 87.4% |
| Large V2 | 50 | 3.0 GB | 4.0 GB | 185 ms | 91.1% | 94.8% | 88.6% | 95.0% | 94.7% | 96.1% | 84.5% | 85.3% |
| Large V2 q8_0 | 50 | 1.6 GB | 2.5 GB | 154 ms | 91.1% | 94.9% | 88.6% | 95.0% | 94.9% | 96.1% | 84.6% | 85.2% |
| Large V2 q5_0 | 50 | 1.1 GB | 2.0 GB | 146 ms | 91.0% | 94.9% | 88.4% | 95.3% | 94.6% | 96.1% | 83.9% | 85.3% |
| Large V1 | 50 | 3.0 GB | 4.0 GB | 184 ms | 89.4% | 94.2% | 86.3% | 94.7% | 93.7% | 95.8% | 80.7% | 82.2% |
| Parakeet TDT v3 | 50 | 500 MB | 80 MB | 19 ms | 89.1% | 94.2% | 85.6% | 95.7% | 92.7% | 94.4% | 78.0% | 84.5% |
| Medium full | 50 | 1.5 GB | 2.1 GB | 116 ms | 88.0% | 94.1% | 83.9% | 94.8% | 93.4% | 95.2% | 78.2% | 78.3% |
| Medium q8_0 | 50 | 766 MB | 1.4 GB | 106 ms | 87.9% | 94.2% | 83.6% | 95.0% | 93.4% | 95.2% | 77.9% | 77.9% |
| Medium q5_0 | 50 | 514 MB | 1.1 GB | 99 ms | 87.8% | 94.3% | 83.5% | 95.3% | 93.3% | 95.2% | 77.1% | 78.2% |
| Small full | 50 | 465 MB | 807 MB | 59 ms | 81.2% | 93.1% | 73.3% | 94.3% | 92.0% | 94.0% | 64.2% | 61.8% |
| Small q8_0 | 50 | 252 MB | 558 MB | 58 ms | 81.0% | 93.1% | 73.0% | 94.1% | 92.1% | 94.0% | 63.9% | 61.0% |
| Small q5_1 | 50 | 181 MB | 476 MB | 58 ms | 80.8% | 93.5% | 72.4% | 94.5% | 92.4% | 93.8% | 63.6% | 59.8% |
| Base full | 50 | 142 MB | 334 MB | 57 ms | 69.1% | 91.2% | 54.4% | 91.8% | 90.6% | 88.4% | 39.5% | 35.3% |
| Base q8_0 | 50 | 78 MB | 247 MB | 57 ms | 69.0% | 91.2% | 54.2% | 92.4% | 90.1% | 88.6% | 38.4% | 35.5% |
| Base q5_1 | 50 | 57 MB | 219 MB | 57 ms | 68.7% | 90.3% | 54.3% | 90.8% | 89.9% | 88.9% | 37.6% | 36.4% |
| Tiny full | 50 | 75 MB | 224 MB | 57 ms | 58.0% | 87.0% | 38.7% | 87.6% | 86.4% | 85.0% | 14.2% | 16.9% |
| Tiny q8_0 | 50 | 42 MB | 174 MB | 57 ms | 57.6% | 87.0% | 38.0% | 87.2% | 86.8% | 85.0% | 11.2% | 17.9% |
| Tiny q5_1 | 50 | 31 MB | 157 MB | 58 ms | 56.5% | 86.5% | 36.5% | 87.1% | 86.0% | 83.5% | 14.4% | 11.7% |
| Base q5_1 en | 50 | 57 MB | 217 MB | 58 ms | 33.7% | 92.4% | -5.3% | 92.7% | 92.1% | 3.8% | -9.6% | -10.2% |
| Base q8_0 en | 50 | 78 MB | 247 MB | 58 ms | 33.7% | 92.1% | -5.2% | 92.4% | 91.8% | 3.2% | -7.9% | -10.9% |
| Base full en | 50 | 142 MB | 333 MB | 59 ms | 33.0% | 92.4% | -6.7% | 93.1% | 91.8% | 0.8% | -7.0% | -13.8% |
| Medium full en | 50 | 1.5 GB | 2.1 GB | 116 ms | 32.1% | 94.6% | -9.6% | 95.7% | 93.5% | 10.8% | -13.9% | -25.6% |
| Small full en | 50 | 465 MB | 807 MB | 61 ms | 32.0% | 94.0% | -9.4% | 95.3% | 92.8% | 9.0% | -15.3% | -21.9% |
| Small q8_0 en | 50 | 252 MB | 558 MB | 60 ms | 31.2% | 94.0% | -10.7% | 95.3% | 92.7% | 10.1% | -19.3% | -23.0% |
| Medium q8_0 en | 50 | 766 MB | 1.4 GB | 103 ms | 31.0% | 94.3% | -11.2% | 95.7% | 93.0% | 10.0% | -16.3% | -27.4% |
| Medium q5_0 en | 50 | 514 MB | 1.1 GB | 96 ms | 30.9% | 94.4% | -11.4% | 95.7% | 93.1% | 9.2% | -11.4% | -32.0% |
| Small q5_1 en | 50 | 181 MB | 475 MB | 62 ms | 30.1% | 94.2% | -12.6% | 95.4% | 93.0% | 5.7% | -17.2% | -26.3% |
| Tiny full en | 50 | 75 MB | 224 MB | 59 ms | 26.5% | 88.4% | -14.8% | 88.9% | 87.8% | -1.2% | -18.7% | -24.4% |
| Tiny q5_1 en | 50 | 31 MB | 157 MB | 59 ms | 25.2% | 88.2% | -16.7% | 88.8% | 87.5% | -1.0% | -26.0% | -23.2% |
| Tiny q8_0 en | 50 | 42 MB | 173 MB | 58 ms | 24.7% | 88.6% | -17.9% | 89.1% | 88.1% | -2.3% | -20.7% | -30.7% |
At a glance
Ratings computed from benchmark data, scaled 1 to 10. Accuracy is based on Word Error Rate (WER) and does not include punctuation yet.
| Name | Lang | Translate | Speed | Accuracy |
|---|---|---|---|---|
| Large V3 | all | ✔ | 5 | 10 |
| Large V3 Turbo q5_0 | all | ✘ | 7 | 9 |
| Large V3 q5_0 | all | ✔ | 6 | 9 |
| Parakeet TDT v3 | 25 | ✘ | 10 | 9 |
| Large V1 | all | ✔ | 5 | 9 |
| Large V2 | all | ✔ | 5 | 9 |
| Large V2 q5_0 | all | ✔ | 6 | 9 |
| Large V2 q8_0 | all | ✔ | 6 | 9 |
| Large V3 Turbo | all | ✘ | 7 | 9 |
| Large V3 Turbo q8_0 | all | ✘ | 7 | 9 |
| Medium full | all | ✔ | 7 | 9 |
| Medium q5_0 | all | ✔ | 7 | 9 |
| Medium q8_0 | all | ✔ | 7 | 9 |
| Small q5_1 | all | ✔ | 8 | 7 |
| Small full | all | ✔ | 8 | 7 |
| Small q8_0 | all | ✔ | 8 | 7 |
| Base full | all | ✔ | 9 | 5 |
| Base q5_1 | all | ✔ | 9 | 5 |
| Base q8_0 | all | ✔ | 9 | 5 |
| Tiny full | all | ✔ | 9 | 3 |
| Tiny q8_0 | all | ✔ | 9 | 3 |
| Tiny q5_1 | all | ✔ | 9 | 2 |
| Base full en | en | ✘ | 8 | 1 (en: 9) |
| Base q5_1 en | en | ✘ | 9 | 1 (en: 9) |
| Base q8_0 en | en | ✘ | 9 | 1 (en: 9) |
| Medium full en | en | ✘ | 7 | 1 (en: 10) |
| Medium q5_0 en | en | ✘ | 8 | 1 (en: 10) |
| Medium q8_0 en | en | ✘ | 7 | 1 (en: 10) |
| Small full en | en | ✘ | 8 | 1 (en: 10) |
| Small q5_1 en | en | ✘ | 8 | 1 (en: 10) |
| Small q8_0 en | en | ✘ | 8 | 1 (en: 10) |
| Tiny full en | en | ✘ | 8 | 1 (en: 9) |
| Tiny q5_1 en | en | ✘ | 8 | 1 (en: 9) |
| Tiny q8_0 en | en | ✘ | 8 | 1 (en: 9) |
Charts
