Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
ll
Modellserver / Inferenz-Engine

llama.cpp

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

3089Benchmarks
62Models
Best generation
2.491,2 tok/s
gemma-4-E2B-it
∅ Generation
251,8 tok/s
Schnitt aus 3089 Läufen
Best prefill
21.012 tok/s
Prompt processing
Best TTFT
20 ms
Time to First Token

Top-Modelle auf llama.cpp

Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.

gemma-4-E2B-it5B
2.491,2 tok/s
Nemotron-3-Nano-4B4B
2.182,7 tok/s
gpt-oss-20b20B
1.995,4 tok/s
Ornith-1.0-35B35B
1.799,4 tok/s
Nemotron-3-Nano-Omni-30B-A3B-Reasoning30B
1.791,5 tok/s
Nemotron-3-Nano-30B-A3B30B
1.773,5 tok/s
Nemotron-Cascade-2-30B-A3B30B
1.768,1 tok/s
Laguna-XS-2.1
1.765,0 tok/s
gemma-4-E4B-it8B
1.758,6 tok/s
Qwen3-Coder-30B-A3B-Instruct30B
1.716,6 tok/s

Auf welcher Hardware laeuft llama.cpp am schnellsten?

Bester Durchsatz je Grafikkarte mit diesem Modellserver.

NVIDIA GeForce RTX 5090
2.491,2 tok/s
NVIDIA RTX PRO 6000 Blackwell Workstation Edition
2.414,4 tok/s
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
2.143,6 tok/s
NVIDIA GeForce RTX 5070 Ti
1.744,0 tok/s
NVIDIA GeForce RTX 3090 Ti
1.492,8 tok/s
NVIDIA GB10
903,3 tok/s
Performance profile

Durchsatz mit llama.cpp

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

2.8902.5042.1181.7311.3458.56710.44812.32914.211Prefill (tok/s)Generation (tok/s)AMD Ryzen 7 5800X3D 8-Core Processor - 2.491,2 tok/s Generation, 9.882 tok/s Prefill, TTFT 7.279 ms (19 Laufe)AMD Ryzen 7 5800X3D 8...AMD Ryzen 9 9950X 16-Core Processor - 2.414,4 tok/s Generation, 11.453 tok/s Prefill, TTFT 6.708 ms (22 Laufe)AMD Ryzen 9 9950X 16-...AMD Ryzen Threadripper PRO 9965WX 24-Cores - 2.143,6 tok/s Generation, 12.016 tok/s Prefill, TTFT 7.089 ms (37 Laufe)AMD Ryzen Threadrippe...AMD Ryzen Threadripper PRO 5975WX 32-Cores - 1.744,0 tok/s Generation, 12.895 tok/s Prefill, TTFT 5.836 ms (2 Laufe)AMD Ryzen Threadrippe...
AMD Ryzen 7 5800X3D 8-Core Processor 2.491,2 tok/sAMD Ryzen 9 9950X 16-Core Processor 2.414,4 tok/sAMD Ryzen Threadripper PRO 9965WX 24-Cores 2.143,6 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 1.744,0 tok/s

GPUnach Grafikkarte

2.8902.5042.1181.7311.3458.56710.44812.32914.211Prefill (tok/s)Generation (tok/s)NVIDIA GeForce RTX 5090 - 2.491,2 tok/s Generation, 9.882 tok/s Prefill, TTFT 7.279 ms (19 Laufe)NVIDIA GeForce RTX 50...NVIDIA RTX PRO 6000 Blackwell Workstation Edition - 2.414,4 tok/s Generation, 11.453 tok/s Prefill, TTFT 6.708 ms (22 Laufe)NVIDIA RTX PRO 6000 B...NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition - 2.143,6 tok/s Generation, 12.016 tok/s Prefill, TTFT 7.089 ms (37 Laufe)NVIDIA RTX PRO 6000 B...NVIDIA GeForce RTX 5070 Ti - 1.744,0 tok/s Generation, 12.895 tok/s Prefill, TTFT 5.836 ms (2 Laufe)NVIDIA GeForce RTX 50...
NVIDIA GeForce RTX 5090 2.491,2 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 2.414,4 tok/sNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 2.143,6 tok/sNVIDIA GeForce RTX 5070 Ti 1.744,0 tok/s

MODnach Modell

2.9342.4702.0061.5411.0772.5618.44414.32720.210Prefill (tok/s)Generation (tok/s)gemma-4-E2B-it - 2.491,2 tok/s Generation, 15.103 tok/s Prefill, TTFT 4.996 ms (5 Laufe)gemma-4-E2B-itNemotron-3-Nano-4B - 2.182,7 tok/s Generation, 9.711 tok/s Prefill, TTFT 6.274 ms (6 Laufe)Nemotron-3-Nano-4Bgpt-oss-20b - 1.995,4 tok/s Generation, 16.290 tok/s Prefill, TTFT 5.603 ms (7 Laufe)gpt-oss-20bOrnith-1.0-35B - 1.799,4 tok/s Generation, 7.639 tok/s Prefill, TTFT 8.199 ms (3 Laufe)Ornith-1.0-35BNemotron-3-Nano-Omni-30B-A3B-Reasoning - 1.791,5 tok/s Generation, 8.883 tok/s Prefill, TTFT 7.317 ms (5 Laufe)Nemotron-3-Nano-Omni-...Nemotron-3-Nano-30B-A3B - 1.773,5 tok/s Generation, 8.971 tok/s Prefill, TTFT 7.271 ms (5 Laufe)Nemotron-3-Nano-30B-A...Nemotron-Cascade-2-30B-A3B - 1.768,1 tok/s Generation, 9.158 tok/s Prefill, TTFT 7.267 ms (5 Laufe)Nemotron-Cascade-2-30...Laguna-XS-2.1 - 1.765,0 tok/s Generation, 10.120 tok/s Prefill, TTFT 7.608 ms (18 Laufe)Laguna-XS-2.1gemma-4-E4B-it - 1.758,6 tok/s Generation, 13.020 tok/s Prefill, TTFT 6.144 ms (2 Laufe)gemma-4-E4B-itQwen3-Coder-30B-A3B-Instruct - 1.716,6 tok/s Generation, 15.882 tok/s Prefill, TTFT 6.791 ms (4 Laufe)Qwen3-Coder-30B-A3B-I...Qwen3-30B-A3B-Thinking-2507 - 1.683,8 tok/s Generation, 12.314 tok/s Prefill, TTFT 7.260 ms (4 Laufe)Qwen3-30B-A3B-Thinkin...Qwen3-Omni-30B-A3B-Thinking - 1.673,6 tok/s Generation, 11.468 tok/s Prefill, TTFT 7.357 ms (3 Laufe)Qwen3-Omni-30B-A3B-Th...Qwen3-30B-A3B-Instruct-2507 - 1.671,0 tok/s Generation, 15.288 tok/s Prefill, TTFT 6.912 ms (3 Laufe)Qwen3-30B-A3B-Instruc...Qwen3-VL-30B-A3B-Instruct - 1.655,9 tok/s Generation, 12.155 tok/s Prefill, TTFT 7.081 ms (2 Laufe)Qwen3-VL-30B-A3B-Inst...North-Mini-Code-1.0 - 1.621,0 tok/s Generation, 7.442 tok/s Prefill, TTFT 8.031 ms (2 Laufe)North-Mini-Code-1.0Meta-Llama-3.1-8B-Instruct - 1.593,1 tok/s Generation, 17.119 tok/s Prefill, TTFT 6.537 ms (2 Laufe)Meta-Llama-3.1-8B-Ins...Gemma-4-26B-A4B-it - 1.589,1 tok/s Generation, 5.652 tok/s Prefill, TTFT 9.060 ms (2 Laufe)Gemma-4-26B-A4B-itOrnith-1.0-9B - 1.560,5 tok/s Generation, 10.558 tok/s Prefill, TTFT 6.806 ms (1 Lauf)Ornith-1.0-9BQwen3.5-35B-A3B - 1.520,4 tok/s Generation, 7.808 tok/s Prefill, TTFT 7.689 ms (1 Lauf)Qwen3.5-35B-A3B
gemma-4-E2B-it 2.491,2 tok/sNemotron-3-Nano-4B 2.182,7 tok/sgpt-oss-20b 1.995,4 tok/sOrnith-1.0-35B 1.799,4 tok/sNemotron-3-Nano-Omni-30B-A3B-Reasoning 1.791,5 tok/sNemotron-3-Nano-30B-A3B 1.773,5 tok/sNemotron-Cascade-2-30B-A3B 1.768,1 tok/sLaguna-XS-2.1 1.765,0 tok/sgemma-4-E4B-it 1.758,6 tok/sQwen3-Coder-30B-A3B-Instruct 1.716,6 tok/sQwen3-30B-A3B-Thinking-2507 1.683,8 tok/sQwen3-Omni-30B-A3B-Thinking 1.673,6 tok/sQwen3-30B-A3B-Instruct-2507 1.671,0 tok/sQwen3-VL-30B-A3B-Instruct 1.655,9 tok/sNorth-Mini-Code-1.0 1.621,0 tok/sMeta-Llama-3.1-8B-Instruct 1.593,1 tok/sGemma-4-26B-A4B-it 1.589,1 tok/sOrnith-1.0-9B 1.560,5 tok/sQwen3.5-35B-A3B 1.520,4 tok/s
All measurements

80 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit llama.cpp.

← Alle Engines
Metric:
Size
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.491,20 tok/s TG
Prefill 15.814 · TTFT 4.382 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
2gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.414,37 tok/s TG
Prefill 20.277 · TTFT 4.107 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
3Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.182,67 tok/s TG
Prefill 10.599 · TTFT 5.598 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
4gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2.143,55 tok/s TG
Prefill 13.632 · TTFT 4.819 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
5gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.995,37 tok/s TG
Prefill 16.158 · TTFT 5.309 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
6gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.993,51 tok/s TG
Prefill 16.318 · TTFT 5.323 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
7Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.937,67 tok/s TG
Prefill 9.452 · TTFT 6.288 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
8Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.911,59 tok/s TG
Prefill 10.116 · TTFT 6.245 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
9Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.896,98 tok/s TG
Prefill 10.984 · TTFT 6.227 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
10gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.890,69 tok/s TG
Prefill 14.771 · TTFT 5.489 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
11Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.878,73 tok/s TG
Prefill 7.717 · TTFT 6.690 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
12gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.835,73 tok/s TG
Prefill 15.972 · TTFT 5.760 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
13Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.830,31 tok/s TG
Prefill 9.399 · TTFT 6.598 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
14gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.819,53 tok/s TG
Prefill 19.732 · TTFT 5.578 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
15gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.801,97 tok/s TG
Prefill 18.226 · TTFT 5.605 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
16Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.799,41 tok/s TG
Prefill 9.192 · TTFT 8.876 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
17Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.791,48 tok/s TG
Prefill 7.972 · TTFT 6.996 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
18Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.773,53 tok/s TG
Prefill 8.720 · TTFT 6.861 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
19Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.768,14 tok/s TG
Prefill 8.360 · TTFT 6.952 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
20Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.764,99 tok/s TG
Prefill 8.542 · TTFT 7.776 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
21gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.758,55 tok/s TG
Prefill 13.889 · TTFT 5.964 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
22Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.757,37 tok/s TG
Prefill 9.025 · TTFT 7.343 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
23Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.753,82 tok/s TG
Prefill 9.267 · TTFT 7.329 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
24gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.743,96 tok/s TG
Prefill 13.641 · TTFT 5.790 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_MDetails →
25gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.743,14 tok/s TG
Prefill 12.148 · TTFT 5.881 ms
10×NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAMllama.cppgodclawQ4_K_MDetails →
26gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.742,13 tok/s TG
Prefill 12.852 · TTFT 6.156 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
27Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.740,43 tok/s TG
Prefill 9.653 · TTFT 7.258 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
28Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.740,12 tok/s TG
Prefill 8.571 · TTFT 7.786 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
29Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.738,38 tok/s TG
Prefill 8.872 · TTFT 7.722 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
30Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.722,05 tok/s TG
Prefill 7.387 · TTFT 7.500 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
31Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.719,17 tok/s TG
Prefill 8.044 · TTFT 7.376 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
32Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.716,59 tok/s TG
Prefill 12.604 · TTFT 6.861 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
33Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.706,69 tok/s TG
Prefill 7.511 · TTFT 7.506 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
34Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.689,95 tok/s TG
Prefill 14.458 · TTFT 6.609 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
35Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.683,83 tok/s TG
Prefill 10.169 · TTFT 7.361 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
36Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.673,59 tok/s TG
Prefill 9.186 · TTFT 7.567 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
37Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.671,02 tok/s TG
Prefill 11.920 · TTFT 7.068 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
38Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.670,49 tok/s TG
Prefill 10.926 · TTFT 7.140 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
39Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.658,09 tok/s TG
Prefill 13.451 · TTFT 6.808 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
40Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.655,92 tok/s TG
Prefill 10.520 · TTFT 7.269 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
41Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.655,21 tok/s TG
Prefill 10.591 · TTFT 7.004 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
42Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.654,57 tok/s TG
Prefill 10.472 · TTFT 7.000 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
43Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.653,28 tok/s TG
Prefill 10.581 · TTFT 7.259 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
44Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.650,98 tok/s TG
Prefill 11.364 · TTFT 6.872 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
45Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.650,72 tok/s TG
Prefill 11.398 · TTFT 6.887 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
46Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.649,85 tok/s TG
Prefill 10.323 · TTFT 7.066 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
47Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.639,00 tok/s TG
Prefill 13.790 · TTFT 6.893 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
48Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.629,91 tok/s TG
Prefill 11.384 · TTFT 7.322 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
49North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.621,00 tok/s TG
Prefill 8.046 · TTFT 7.551 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawUD-Q4_K_MDetails →
50Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.620,45 tok/s TG
Prefill 10.729 · TTFT 7.428 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
51Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.620,41 tok/s TG
Prefill 12.519 · TTFT 7.197 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
52Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.615,26 tok/s TG
Prefill 13.041 · TTFT 7.166 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
53Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.611,02 tok/s TG
Prefill 12.725 · TTFT 7.223 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
54Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.609,12 tok/s TG
Prefill 10.883 · TTFT 7.452 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
55Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.608,39 tok/s TG
Prefill 11.312 · TTFT 7.410 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
56Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.607,66 tok/s TG
Prefill 10.675 · TTFT 7.505 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
57Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.607,48 tok/s TG
Prefill 7.448 · TTFT 7.492 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
58Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.606,11 tok/s TG
Prefill 12.690 · TTFT 7.203 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
59gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.606,09 tok/s TG
Prefill 12.150 · TTFT 6.323 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
60Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.593,10 tok/s TG
Prefill 16.376 · TTFT 6.849 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
61Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.589,11 tok/s TG
Prefill 6.805 · TTFT 8.073 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawUD-Q4_K_XLDetails →
62Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.560,53 tok/s TG
Prefill 10.558 · TTFT 6.806 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
63North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.557,27 tok/s TG
Prefill 6.839 · TTFT 8.511 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliUD-Q4_K_MDetails →
64Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.556,71 tok/s TG
Prefill 6.275 · TTFT 8.228 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliQ4_K_MDetails →
65Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.554,46 tok/s TG
Prefill 7.954 · TTFT 8.418 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
66Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.554,37 tok/s TG
Prefill 10.928 · TTFT 6.920 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
67Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.545,29 tok/s TG
Prefill 19.346 · TTFT 6.758 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
68Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.542,64 tok/s TG
Prefill 7.398 · TTFT 8.118 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
69Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.540,50 tok/s TG
Prefill 7.349 · TTFT 8.077 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
70Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.540,46 tok/s TG
Prefill 4.499 · TTFT 10.048 ms
10×NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAMllama.cppcodex_cliUD-Q4_K_XLDetails →
71Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.534,12 tok/s TG
Prefill 7.243 · TTFT 8.145 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
72Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.533,70 tok/s TG
Prefill 6.831 · TTFT 8.783 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
73Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.532,95 tok/s TG
Prefill 17.861 · TTFT 6.225 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
74Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.532,89 tok/s TG
Prefill 7.482 · TTFT 8.633 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
75Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.531,89 tok/s TG
Prefill 17.119 · TTFT 6.936 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
76Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.521,37 tok/s TG
Prefill 12.779 · TTFT 7.364 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
77Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.520,35 tok/s TG
Prefill 7.808 · TTFT 7.689 ms
10×NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAMllama.cppgodclawQ4_K_MDetails →
78Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.518,42 tok/s TG
Prefill 15.383 · TTFT 7.173 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
79Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.515,72 tok/s TG
Prefill 20.491 · TTFT 6.859 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →
80Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.512,43 tok/s TG
Prefill 14.637 · TTFT 7.246 ms
10×3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAMllama.cppgodclawQ4_K_MDetails →