Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
Meta

Llama-3.2-3B-Instruct3B

Benchmark profile and published results.

Dense3B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon RX 7900 XTXAll GPUs
Generation465,7 tok/s
Min 0,0Ø 247,7Max 3.484,3
Ueber dem Durchschnitt · 4867 Systeme im Feld
Prefill23.398 tok/s
Min 4Ø 4.890Max 112.346
Ueber dem Durchschnitt · 4849 Systeme im Feld
Time to First Token302 ms
Min 20Ø 32.030Max 535.235
Ueber dem Durchschnitt · 4864 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

6054543021510,005.86611.73217.598Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 465,7 tok/s Generation, 14.196 tok/s Prefill, TTFT 1.049 ms (9 Laufe)AMD Radeon RX 7900 XTXAMD Radeon AI PRO R9700 - 397,2 tok/s Generation, 11.893 tok/s Prefill, TTFT 1.135 ms (3 Laufe)AMD Radeon AI PRO R97...NVIDIA GeForce RTX 2060 - 226,6 tok/s Generation, 6.927 tok/s Prefill, TTFT 2.004 ms (6 Laufe)NVIDIA GeForce RTX 20...CPU-only - 2,5 tok/s Generation, 29 tok/s Prefill, TTFT 270.941 ms (3 Laufe)CPU-only
AMD Radeon RX 7900 XTX 465,7 tok/sAMD Radeon AI PRO R9700 397,2 tok/sNVIDIA GeForce RTX 2060 226,6 tok/sCPU-only 2,5 tok/s

CPUby processor

6054543021510,006.36012.71919.079Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 465,7 tok/s Generation, 15.391 tok/s Prefill, TTFT 992 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 3 3100 4-Core Processor - 397,2 tok/s Generation, 11.893 tok/s Prefill, TTFT 1.135 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 354,2 tok/s Generation, 8.553 tok/s Prefill, TTFT 1.723 ms (9 Laufe)Intel(R) Xeon(R) CPU ...4 x Cortex-A78C + 4 x Cortex-A55 - 2,5 tok/s Generation, 29 tok/s Prefill, TTFT 270.941 ms (3 Laufe)4 x Cortex-A78C + 4 x...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 465,7 tok/sAMD Ryzen 3 3100 4-Core Processor 397,2 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 354,2 tok/s4 x Cortex-A78C + 4 x Cortex-A55 2,5 tok/s

ENGby engine

5264794313843376.07410.51114.94819.386Prefill (tok/s)Generation (tok/s)vLLM - 465,7 tok/s Generation, 16.879 tok/s Prefill, TTFT 1.016 ms (3 Laufe)vLLMllama.cpp - 397,2 tok/s Generation, 8.581 tok/s Prefill, TTFT 46.369 ms (18 Laufe)llama.cpp
vLLM 465,7 tok/sllama.cpp 397,2 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
465,73 tok/s TG
Prefill 23.398 · TTFT 1.915 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
2Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
397,16 tok/s TG
Prefill 14.911 · TTFT 2.001 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
3Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
360,12 tok/s TG
Prefill 19.755 · TTFT 1.579 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
4Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
354,21 tok/s TG
Prefill 17.373 · TTFT 1.825 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
5Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,54 tok/s TG
Prefill 19.091 · TTFT 830 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
6Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
261,48 tok/s TG
Prefill 13.512 · TTFT 1.061 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
7Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
241,23 tok/s TG
Prefill 15.447 · TTFT 931 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
8Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
226,59 tok/s TG
Prefill 9.526 · TTFT 3.488 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
9Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
215,97 tok/s TG
Prefill 9.775 · TTFT 3.283 ms
10×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
10Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
196,57 tok/s TG
Prefill 12.975 · TTFT 1.133 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
11Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
173,08 tok/s TG
Prefill 7.158 · TTFT 2.024 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
12Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
158,90 tok/s TG
Prefill 7.616 · TTFT 1.908 ms
5×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
13Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
109,96 tok/s TG
Prefill 5.072 · TTFT 529 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
14Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
107,15 tok/s TG
Prefill 6.505 · TTFT 397 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
15Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
106,38 tok/s TG
Prefill 7.256 · TTFT 344 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ8_0Details →
16Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,62 tok/s TG
Prefill 3.646 · TTFT 677 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
17Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
70,69 tok/s TG
Prefill 3.840 · TTFT 645 ms
1×2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ8_0Details →
18Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
69,35 tok/s TG
Prefill 8.149 · TTFT 302 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
19Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
2,46 tok/s TG
Prefill 35 · TTFT 360.878 ms
5×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
20Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1,99 tok/s TG
Prefill 22 · TTFT 104.643 ms
1×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
21Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
0,09 tok/s TG
Prefill 31 · TTFT 347.304 ms
10×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.