⚡
Performance benchmark
How fast is the model?
Raw throughput on real hardware – how many tokens a model generates per second, how quickly it processes the prompt and how short the time to first token is. Higher is better.
⚡ Generation (tok/s)🔄 Prefill⏱️ TTFT👥 Concurrency
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | gemma-4-E2B-it5BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 668,00 tok/s TG Prefill 11.047 · TTFT 1.802 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 2 | gemma-4-E2B-it5BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 600,60 tok/s TG Prefill 11.197 · TTFT 1.774 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 3 | gemma-4-E2B-it5BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 462,78 tok/s TG Prefill 10.812 · TTFT 917 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 4 | gemma-4-E2B-it5BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 429,79 tok/s TG Prefill 10.850 · TTFT 916 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 5 | gemma-4-E4B-it8BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 428,56 tok/s TG Prefill 6.452 · TTFT 3.073 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 6 | gpt-oss-20b20BMXFP4OpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 378,51 tok/s TG Prefill 6.766 · TTFT 2.939 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 7 | gemma-4-E4B-it8BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 375,94 tok/s TG Prefill 6.660 · TTFT 2.979 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 8 | Qwen3-Coder-30B-A3B-Instruct30BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 344,35 tok/s TG Prefill 10.942 · TTFT 2.759 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 9 | Meta-Llama-3.1-8B-Instruct8BQ4_K_MMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 343,70 tok/s TG Prefill 9.999 · TTFT 3.230 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 10 | Qwen3-30B-A3B-Instruct-250730BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 339,73 tok/s TG Prefill 10.614 · TTFT 2.771 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 11 | Ornith-1.0-9B9BQ4_K_MDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 300,99 tok/s TG Prefill 4.071 · TTFT 4.790 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 12 | Qwen3.6-35B-A3B35BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 286,68 tok/s TG Prefill 4.535 · TTFT 4.311 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 13 | gemma-4-E4B-it8BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 283,39 tok/s TG Prefill 6.346 · TTFT 1.565 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 14 | Nemotron-3-Nano-4B4BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 277,39 tok/s TG Prefill 5.993 · TTFT 3.672 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 15 | Meta-Llama-3.1-8B-Instruct8BQ8_0Meta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 276,49 tok/s TG Prefill 9.874 · TTFT 3.124 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 16 | gpt-oss-20b20BMXFP4OpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 273,57 tok/s TG Prefill 6.722 · TTFT 1.473 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 17 | Qwen3-Coder-30B-A3B-Instruct30BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 266,26 tok/s TG Prefill 8.447 · TTFT 1.538 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 18 | Qwen3-30B-A3B-Instruct-250730BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 260,37 tok/s TG Prefill 8.681 · TTFT 1.498 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 19 | gemma-4-E4B-it8BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 254,04 tok/s TG Prefill 6.450 · TTFT 1.538 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 20 | Nemotron-Cascade-2-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 250,32 tok/s TG Prefill 5.865 · TTFT 3.782 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 21 | Nemotron-3-Nano-4B4BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 248,17 tok/s TG Prefill 4.462 · TTFT 2.412 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 22 | Qwen3.6-35B-A3B35BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 239,95 tok/s TG Prefill 4.270 · TTFT 2.268 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 23 | Meta-Llama-3.1-8B-Instruct8BQ4_K_MMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 237,04 tok/s TG Prefill 7.169 · TTFT 1.831 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 24 | Ornith-1.0-9B9BQ8_0DeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 235,85 tok/s TG Prefill 4.228 · TTFT 4.615 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 25 | Nemotron-Cascade-2-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 219,79 tok/s TG Prefill 4.791 · TTFT 2.262 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 26 | Ornith-1.0-9B9BQ4_K_MDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 209,92 tok/s TG Prefill 3.882 · TTFT 2.491 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 27 | Nemotron-3-Nano-4B4BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 208,19 tok/s TG Prefill 6.153 · TTFT 3.583 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 28 | Meta-Llama-3.1-8B-Instruct8BQ8_0Meta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 197,34 tok/s TG Prefill 6.940 · TTFT 1.849 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 29 | gemma-4-E2B-it5BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 194,48 tok/s TG Prefill 8.378 · TTFT 236 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 30 | Nemotron-3-Nano-4B4BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 187,29 tok/s TG Prefill 4.534 · TTFT 2.375 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 31 | Ornith-1.0-9B9BQ8_0DeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 172,32 tok/s TG Prefill 4.017 · TTFT 2.410 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 32 | Nemotron-3-Nano-4B4BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 166,48 tok/s TG Prefill 4.080 · TTFT 529 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 33 | gpt-oss-20b20BMXFP4OpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 165,76 tok/s TG Prefill 5.419 · TTFT 365 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 34 | Qwen3-Coder-30B-A3B-Instruct30BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 161,57 tok/s TG Prefill 5.583 · TTFT 437 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 35 | Qwen3-30B-A3B-Instruct-250730BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 160,98 tok/s TG Prefill 5.616 · TTFT 434 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 36 | Mistral-Small-3.1-24B-Instruct-250324BQ4_K_MMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 159,85 tok/s TG Prefill 3.304 · TTFT 8.318 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 37 | Nemotron-Cascade-2-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 157,01 tok/s TG Prefill 4.273 · TTFT 508 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 38 | gemma-4-12B-it12BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 156,81 tok/s TG Prefill 2.576 · TTFT 3.993 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 39 | gemma-4-E2B-it5BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 153,69 tok/s TG Prefill 8.308 · TTFT 239 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 40 | Qwen3.6-35B-A3B35BQ4_K_MQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 131,05 tok/s TG Prefill 3.671 · TTFT 526 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 41 | Nemotron-3-Nano-4B4BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 124,60 tok/s TG Prefill 4.092 · TTFT 528 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 42 | gemma-4-E4B-it8BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 119,55 tok/s TG Prefill 5.239 · TTFT 378 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 43 | Meta-Llama-3.1-8B-Instruct8BQ4_K_MMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 102,47 tok/s TG Prefill 4.866 · TTFT 508 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 44 | Mistral-Small-3.1-24B-Instruct-250324BQ4_K_MMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 94,61 tok/s TG Prefill 2.657 · TTFT 4.592 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 45 | Ornith-1.0-9B9BQ4_K_MDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 93,36 tok/s TG Prefill 3.523 · TTFT 547 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 46 | gemma-4-E4B-it8BQ8_0Google Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 87,28 tok/s TG Prefill 5.229 · TTFT 378 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 47 | Meta-Llama-3.1-8B-Instruct8BQ8_0Meta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 66,99 tok/s TG Prefill 5.119 · TTFT 482 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 48 | Ornith-1.0-9B9BQ8_0DeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 63,48 tok/s TG Prefill 3.602 · TTFT 536 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 49 | gemma-4-12B-it12BQ4_K_MGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 61,61 tok/s TG Prefill 2.561 · TTFT 774 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → | |
| 50 | Mistral-Small-3.1-24B-Instruct-250324BQ4_K_MMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 38,44 tok/s TG Prefill 1.790 · TTFT 1.199 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cli | Details → |