Qwen (Alibaba)
Qwen2.5-72B-Instruct72B
Benchmark profile and published results.
Dense72B
Position in the field
Best values compared
Best metrics of this model against the minimum, average and maximum of all published systems.
Generation239,2 tok/s
Min 0,0Ø 518,2Max 2.414,4
Unter dem Durchschnitt · 244 Systeme im Feld
Prefill2.668 tok/s
Min 7Ø 6.972Max 39.029
Unter dem Durchschnitt · 244 Systeme im Feld
Time to First Token1.412 ms
Min 114Ø 27.182Max 354.674
Ueber dem Durchschnitt · 244 Systeme im Feld
Performance profile
Throughput by hardware & engine
Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.
GPUby graphics card
NVIDIA RTX PRO 6000 Blackwell Workstation Edition 239,2 tok/sNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 203,8 tok/sNVIDIA GeForce RTX 5070 Ti 5,8 tok/sNVIDIA GeForce RTX 3090 Ti 4,9 tok/sNVIDIA GeForce RTX 5090 3,0 tok/sNVIDIA GeForce RTX 2060 2,3 tok/sCPU-only 2,3 tok/sAMD Radeon AI PRO R9700 1,8 tok/s
CPUby processor
AMD Ryzen 9 9950X 16-Core Processor 239,2 tok/sAMD Ryzen Threadripper PRO 9965WX 24-Cores 203,8 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 5,8 tok/sAMD Ryzen 9 8945HX with Radeon Graphics 4,9 tok/sAMD Ryzen 7 5800X3D 8-Core Processor 3,0 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 2,3 tok/sIntel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz 2,3 tok/sAMD Ryzen Threadripper PRO 7955WX 16-Cores 1,8 tok/s
Throughput & latency
Performance benchmark
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 239,17 tok/s TG Prefill 2.414 · TTFT 43.060 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 2 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 203,75 tok/s TG Prefill 3.332 · TTFT 47.827 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 3 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 202,24 tok/s TG Prefill 2.746 · TTFT 49.827 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 4 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 168,46 tok/s TG Prefill 1.711 · TTFT 62.671 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 5 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 123,74 tok/s TG Prefill 2.668 · TTFT 13.115 ms | 5× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 6 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 107,05 tok/s TG Prefill 3.476 · TTFT 13.786 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 7 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 104,63 tok/s TG Prefill 2.633 · TTFT 14.998 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 8 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 86,86 tok/s TG Prefill 1.631 · TTFT 19.862 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 9 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 30,80 tok/s TG Prefill 1.743 · TTFT 1.412 ms | 1× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 10 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 29,55 tok/s TG Prefill 2.137 · TTFT 1.149 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 11 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 29,44 tok/s TG Prefill 1.610 · TTFT 1.524 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 12 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 25,49 tok/s TG Prefill 1.149 · TTFT 2.146 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 13 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 5,78 tok/s TG Prefill 173 · TTFT 53.622 ms | 5× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 14 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 5,77 tok/s TG Prefill 156 · TTFT 56.974 ms | 10× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 15 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,90 tok/s TG Prefill 239 · TTFT 40.388 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 16 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,84 tok/s TG Prefill 239 · TTFT 40.453 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 17 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,96 tok/s TG Prefill 158 · TTFT 53.509 ms | 5× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 18 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,83 tok/s TG Prefill 166 · TTFT 56.632 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 19 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,52 tok/s TG Prefill 148 · TTFT 16.554 ms | 1× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 20 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkPerformancetest Small 1.0 | 2,28 tok/s TG Prefill 81 · TTFT 569 ms | 1× | 2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 21 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmark | 2,28 tok/s TG Prefill 81 · TTFT 569 ms | 1× | Keine GPU (CPU-only)51x Intel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz · 504 GB RAM | godclaw | Details → | |
| 22 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,81 tok/s TG Prefill 15 · TTFT 141.412 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 23 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,79 tok/s TG Prefill 15 · TTFT 143.151 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 24 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,78 tok/s TG Prefill 15 · TTFT 143.025 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 25 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,76 tok/s TG Prefill 20 · TTFT 393.908 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 26 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,72 tok/s TG Prefill 21 · TTFT 372.410 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 27 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,72 tok/s TG Prefill 21 · TTFT 376.885 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 28 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,71 tok/s TG Prefill 21 · TTFT 376.282 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 29 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,70 tok/s TG Prefill 21 · TTFT 377.119 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 30 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,63 tok/s TG Prefill 21 · TTFT 377.401 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 31 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1,56 tok/s TG Prefill 151 · TTFT 14.186 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 32 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 0,82 tok/s TG Prefill 116 · TTFT 18.256 ms | 1× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → |
Agent & chat rating
Harness benchmark
Noch keine Harnessbenchmarks fuer dieses Modell.