Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
Qwen (Alibaba)

Qwen2.5-3B-Instruct3B

Benchmark profile and published results.

Dense3B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon AI PRO R9700All GPUs
Generation364,1 tok/s
Min 0,0Ø 77,4Max 667,0
Ueber dem Durchschnitt · 975 Systeme im Feld
Prefill15.658 tok/s
Min 4Ø 2.287Max 35.719
Ueber dem Durchschnitt · 975 Systeme im Feld
Time to First Token324 ms
Min 31Ø 58.005Max 535.235
Ueber dem Durchschnitt · 973 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

4733552361180,005.88311.76617.649Prefill (tok/s)Generation (tok/s)AMD Radeon AI PRO R9700 - 364,1 tok/s Generation, 11.369 tok/s Prefill, TTFT 1.151 ms (3 Laufe)AMD Radeon AI PRO R97...AMD Radeon RX 7900 XTX - 358,2 tok/s Generation, 14.238 tok/s Prefill, TTFT 1.134 ms (9 Laufe)AMD Radeon RX 7900 XTXNVIDIA GeForce RTX 2060 - 259,9 tok/s Generation, 6.911 tok/s Prefill, TTFT 1.971 ms (6 Laufe)NVIDIA GeForce RTX 20...CPU-only - 3,5 tok/s Generation, 30 tok/s Prefill, TTFT 277.455 ms (3 Laufe)CPU-only
AMD Radeon AI PRO R9700 364,1 tok/sAMD Radeon RX 7900 XTX 358,2 tok/sNVIDIA GeForce RTX 2060 259,9 tok/sCPU-only 3,5 tok/s

CPUby processor

4733552361180,006.47312.94619.419Prefill (tok/s)Generation (tok/s)AMD Ryzen 3 3100 4-Core Processor - 364,1 tok/s Generation, 11.369 tok/s Prefill, TTFT 1.151 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...AMD Ryzen Threadripper PRO 3955WX 16-Cores - 358,2 tok/s Generation, 15.665 tok/s Prefill, TTFT 1.104 ms (6 Laufe)AMD Ryzen Threadrippe...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 333,2 tok/s Generation, 8.401 tok/s Prefill, TTFT 1.712 ms (9 Laufe)Intel(R) Xeon(R) CPU ...4 x Cortex-A78C + 4 x Cortex-A55 - 3,5 tok/s Generation, 30 tok/s Prefill, TTFT 277.455 ms (3 Laufe)4 x Cortex-A78C + 4 x...
AMD Ryzen 3 3100 4-Core Processor 364,1 tok/sAMD Ryzen Threadripper PRO 3955WX 16-Cores 358,2 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 333,2 tok/s4 x Cortex-A78C + 4 x Cortex-A55 3,5 tok/s

ENGby engine

4023813603393185.80210.51615.22919.943Prefill (tok/s)Generation (tok/s)llama.cpp - 364,1 tok/s Generation, 8.438 tok/s Prefill, TTFT 47.449 ms (18 Laufe)llama.cppvLLM - 355,9 tok/s Generation, 17.308 tok/s Prefill, TTFT 1.253 ms (3 Laufe)vLLM
llama.cpp 364,1 tok/svLLM 355,9 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
364,13 tok/s TG
Prefill 15.658 · TTFT 1.918 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
2Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
358,15 tok/s TG
Prefill 19.410 · TTFT 1.565 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
3Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
355,86 tok/s TG
Prefill 24.866 · TTFT 2.634 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
4Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
333,18 tok/s TG
Prefill 17.060 · TTFT 1.879 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
5Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
289,42 tok/s TG
Prefill 10.849 · TTFT 1.212 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
6Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
259,86 tok/s TG
Prefill 9.253 · TTFT 3.361 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
7Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
253,98 tok/s TG
Prefill 19.902 · TTFT 780 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
8Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
225,82 tok/s TG
Prefill 9.739 · TTFT 3.277 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
9Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
213,88 tok/s TG
Prefill 16.055 · TTFT 898 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
10Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
207,49 tok/s TG
Prefill 12.047 · TTFT 1.151 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
11Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
187,50 tok/s TG
Prefill 6.698 · TTFT 2.090 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
12Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
168,19 tok/s TG
Prefill 7.879 · TTFT 1.850 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
13Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
108,65 tok/s TG
Prefill 5.041 · TTFT 551 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
14Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,40 tok/s TG
Prefill 3.776 · TTFT 651 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
15Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
104,76 tok/s TG
Prefill 6.603 · TTFT 399 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
16Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
93,07 tok/s TG
Prefill 7.599 · TTFT 324 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
17Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
73,93 tok/s TG
Prefill 4.119 · TTFT 596 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
18Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
52,34 tok/s TG
Prefill 7.155 · TTFT 346 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
19Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3,47 tok/s TG
Prefill 33 · TTFT 369.084 ms
5×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
20Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3,22 tok/s TG
Prefill 24 · TTFT 103.262 ms
1×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
21Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
0,07 tok/s TG
Prefill 33 · TTFT 360.019 ms
10×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.