Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
Meta

Llama-3.2-1B-Instruct1B

Benchmark profile and published results.

Dense1B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon RX 7900 XTXAll GPUs
Generation717,6 tok/s
Min 0,1Ø 161,0Max 717,6
Spitzenwert im Feld · 338 Systeme im Feld
Prefill52.503 tok/s
Min 39Ø 5.997Max 52.503
Spitzenwert im Feld · 338 Systeme im Feld
Time to First Token156 ms
Min 156Ø 18.924Max 354.928
Spitzenwert im Feld · 338 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

9306984652330,0011.67723.35435.031Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 717,6 tok/s Generation, 28.265 tok/s Prefill, TTFT 548 ms (9 Laufe)AMD Radeon RX 7900 XTXAMD Liverpool (PlayStation 4) - 43,2 tok/s Generation, 237 tok/s Prefill, TTFT 10.796 ms (2 Laufe)AMD Liverpool (PlaySt...CPU-only - 13,2 tok/s Generation, 94 tok/s Prefill, TTFT 134.150 ms (3 Laufe)CPU-only
AMD Radeon RX 7900 XTX 717,6 tok/sAMD Liverpool (PlayStation 4) 43,2 tok/sCPU-only 13,2 tok/s

CPUby processor

9306984652330,0013.27626.55239.828Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 717,6 tok/s Generation, 32.133 tok/s Prefill, TTFT 493 ms (6 Laufe)AMD Ryzen Threadrippe...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 609,5 tok/s Generation, 20.528 tok/s Prefill, TTFT 659 ms (3 Laufe)Intel(R) Xeon(R) CPU ...AMD Jaguar 8-Core (PlayStation 4) - 43,2 tok/s Generation, 237 tok/s Prefill, TTFT 10.796 ms (2 Laufe)AMD Jaguar 8-Core (Pl...4 x Cortex-A78C + 4 x Cortex-A55 - 13,2 tok/s Generation, 94 tok/s Prefill, TTFT 134.150 ms (3 Laufe)4 x Cortex-A78C + 4 x...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 717,6 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 609,5 tok/sAMD Jaguar 8-Core (PlayStation 4) 43,2 tok/s4 x Cortex-A78C + 4 x Cortex-A55 13,2 tok/s

ENGby engine

7947507076646216.92518.86130.79842.734Prefill (tok/s)Generation (tok/s)vLLM - 717,6 tok/s Generation, 36.389 tok/s Prefill, TTFT 511 ms (3 Laufe)vLLMllama.cpp - 696,8 tok/s Generation, 13.270 tok/s Prefill, TTFT 38.858 ms (11 Laufe)llama.cpp
vLLM 717,6 tok/sllama.cpp 696,8 tok/s
Throughput & latency

Performance benchmark

Metric:
GPUs
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
717,61 tok/s TG
Prefill 52.503 · TTFT 1.022 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
2Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
696,81 tok/s TG
Prefill 41.362 · TTFT 722 ms
10×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
3Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
609,48 tok/s TG
Prefill 31.648 · TTFT 962 ms
10×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
4Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
479,20 tok/s TG
Prefill 40.760 · TTFT 356 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
5Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
443,74 tok/s TG
Prefill 31.085 · TTFT 445 ms
5×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
6Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
376,49 tok/s TG
Prefill 22.247 · TTFT 615 ms
5×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
7Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
209,03 tok/s TG
Prefill 11.184 · TTFT 257 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
8Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
207,12 tok/s TG
Prefill 7.688 · TTFT 400 ms
1×2x AMD Radeon RX 7900 XTX i4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAMllama.cppgodclawQ4_K_MDetails →
9Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
98,26 tok/s TG
Prefill 15.904 · TTFT 156 ms
1×2x AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMvLLMgodclawDetails →
10Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
43,17 tok/s TG
Prefill 281 · TTFT 8.805 ms
1×AMD Liverpool (PlayStation 4)AMD Jaguar 8-Core (PlayStation 4) · 5 GB RAMllama.cppgodclawQ4_K_MDetails →
11Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
13,17 tok/s TG
Prefill 122 · TTFT 228.054 ms
10×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
12Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
12,93 tok/s TG
Prefill 95 · TTFT 136.105 ms
5×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
13Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
7,33 tok/s TG
Prefill 65 · TTFT 38.290 ms
1×Keine GPU (CPU-only) i4 x Cortex-A78C + 4 x Cortex-A55 · 15 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
14Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
3,91 tok/s TG
Prefill 193 · TTFT 12.786 ms
1×AMD Liverpool (PlayStation 4)AMD Jaguar 8-Core (PlayStation 4) · 5 GB RAMllama.cppgodclawQ4_K_MDetails →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.