Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
Qwen (Alibaba)

Qwen2.5-7B-Instruct7B

Benchmark profile and published results.

Dense7B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon RX 7900 XTXAll GPUs
Generation442,0 tok/s
Min 0,0Ø 247,7Max 3.484,3
Ueber dem Durchschnitt · 4867 Systeme im Feld
Prefill16.333 tok/s
Min 4Ø 4.890Max 112.346
Ueber dem Durchschnitt · 4849 Systeme im Feld
Time to First Token411 ms
Min 20Ø 32.030Max 535.235
Ueber dem Durchschnitt · 4864 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

54342230118059,72.0075.0778.14711.216Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 442,0 tok/s Generation, 9.575 tok/s Prefill, TTFT 1.732 ms (6 Laufe)AMD Radeon RX 7900 XTXAMD Radeon AI PRO R9700 - 250,6 tok/s Generation, 6.580 tok/s Prefill, TTFT 2.039 ms (3 Laufe)AMD Radeon AI PRO R97...NVIDIA GeForce RTX 2060 - 160,3 tok/s Generation, 3.649 tok/s Prefill, TTFT 3.816 ms (3 Laufe)NVIDIA GeForce RTX 20...
AMD Radeon RX 7900 XTX 442,0 tok/sAMD Radeon AI PRO R9700 250,6 tok/sNVIDIA GeForce RTX 2060 160,3 tok/s

CPUby processor

54342230118059,72.0075.0778.14711.216Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 442,0 tok/s Generation, 9.575 tok/s Prefill, TTFT 1.732 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 3 3100 4-Core Processor - 250,6 tok/s Generation, 6.580 tok/s Prefill, TTFT 2.039 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 160,3 tok/s Generation, 3.649 tok/s Prefill, TTFT 3.816 ms (3 Laufe)Intel(R) Xeon(R) CPU ...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 442,0 tok/sAMD Ryzen 3 3100 4-Core Processor 250,6 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 160,3 tok/s

ENGby engine

5224373522661814.0127.24910.48513.722Prefill (tok/s)Generation (tok/s)vLLM - 442,0 tok/s Generation, 11.912 tok/s Prefill, TTFT 1.633 ms (3 Laufe)vLLMllama.cpp - 261,4 tok/s Generation, 5.822 tok/s Prefill, TTFT 2.562 ms (9 Laufe)llama.cpp
vLLM 442,0 tok/sllama.cpp 261,4 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
442,02 tok/s TG
Prefill 16.333 · TTFT 3.286 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
2Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
326,73 tok/s TG
Prefill 13.426 · TTFT 1.202 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
3Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
261,39 tok/s TG
Prefill 10.101 · TTFT 2.985 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
4Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
250,57 tok/s TG
Prefill 9.085 · TTFT 3.442 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
5Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
178,27 tok/s TG
Prefill 8.322 · TTFT 1.724 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
6Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,09 tok/s TG
Prefill 6.490 · TTFT 2.083 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
7Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
160,26 tok/s TG
Prefill 4.979 · TTFT 6.510 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
8Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,79 tok/s TG
Prefill 3.816 · TTFT 3.796 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
9Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
76,83 tok/s TG
Prefill 3.292 · TTFT 785 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
10Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
68,23 tok/s TG
Prefill 5.975 · TTFT 411 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
11Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
58,79 tok/s TG
Prefill 4.166 · TTFT 593 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
12Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,92 tok/s TG
Prefill 2.151 · TTFT 1.142 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.