Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
NVIDIA

Llama-3.1-Nemotron-Nano-4B-v1.14.5B

Benchmark profile and published results.

Dense4.5B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon RX 7900 XTXAll GPUs
Generation419,4 tok/s
Min 0,0Ø 242,0Max 2.491,2
Ueber dem Durchschnitt · 4261 Systeme im Feld
Prefill11.855 tok/s
Min 4Ø 4.687Max 55.260
Ueber dem Durchschnitt · 4261 Systeme im Feld
Time to First Token457 ms
Min 20Ø 32.760Max 535.235
Ueber dem Durchschnitt · 4259 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

4614404193983778.2198.5698.9199.268Prefill (tok/s)Generation (tok/s)AMD Radeon RX 7900 XTX - 419,4 tok/s Generation, 8.744 tok/s Prefill, TTFT 1.556 ms (6 Laufe)AMD Radeon RX 7900 XTX
AMD Radeon RX 7900 XTX 419,4 tok/s

CPUby processor

4614404193983778.2198.5698.9199.268Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 3955WX 16-Cores - 419,4 tok/s Generation, 8.744 tok/s Prefill, TTFT 1.556 ms (6 Laufe)AMD Ryzen Threadrippe...
AMD Ryzen Threadripper PRO 3955WX 16-Cores 419,4 tok/s

ENGby engine

4614404193983778.2198.5698.9199.268Prefill (tok/s)Generation (tok/s)llama.cpp - 419,4 tok/s Generation, 8.744 tok/s Prefill, TTFT 1.556 ms (6 Laufe)llama.cpp
llama.cpp 419,4 tok/s
Throughput & latency

Performance benchmark

Metric:
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
419,38 tok/s TG
Prefill 11.768 · TTFT 2.673 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
2Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
418,31 tok/s TG
Prefill 11.855 · TTFT 2.651 ms
10×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0Details →
3Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
318,18 tok/s TG
Prefill 9.348 · TTFT 1.524 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
4Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
294,88 tok/s TG
Prefill 9.401 · TTFT 1.516 ms
5×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0Details →
5Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
139,33 tok/s TG
Prefill 4.729 · TTFT 517 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ4_K_MDetails →
6Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
110,40 tok/s TG
Prefill 5.361 · TTFT 457 ms
1×AMD Radeon RX 7900 XTXAMD Ryzen Threadripper PRO 3955WX 16-Cores · 63 GB RAMllama.cppgodclawQ8_0Details →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.