Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
NVIDIA

Nemotron-3.5-Lightning-30B-A3B30B

Benchmark profile and published results.

Hybrid Mamba-2 + MoE (3B aktiv)30B
Position in the field

Best values compared

Best metrics of this model against the minimum, average and maximum of all published systems.

AMD Radeon AI PRO R9700All GPUs
Generation290,3 tok/s
Min 0,0Ø 65,8Max 667,0
Ueber dem Durchschnitt · 855 Systeme im Feld
Prefill35.719 tok/s
Min 4Ø 1.970Max 35.719
Spitzenwert im Feld · 855 Systeme im Feld
Time to First Token155 ms
Min 31Ø 65.510Max 535.235
Ueber dem Durchschnitt · 853 Systeme im Feld
Performance profile

Throughput by hardware & engine

Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.

GPUby graphics card

3193052902762616.2226.4876.7517.016Prefill (tok/s)Generation (tok/s)AMD Radeon AI PRO R9700 - 290,3 tok/s Generation, 6.619 tok/s Prefill, TTFT 679 ms (27 Laufe)AMD Radeon AI PRO R97...
AMD Radeon AI PRO R9700 290,3 tok/s

CPUby processor

3193052902762616.2226.4876.7517.016Prefill (tok/s)Generation (tok/s)AMD Ryzen Threadripper PRO 7955WX 16-Cores - 290,3 tok/s Generation, 6.619 tok/s Prefill, TTFT 679 ms (27 Laufe)AMD Ryzen Threadrippe...
AMD Ryzen Threadripper PRO 7955WX 16-Cores 290,3 tok/s

ENGby engine

3203042882722562608.98617.71226.438Prefill (tok/s)Generation (tok/s)vLLM - 290,3 tok/s Generation, 22.003 tok/s Prefill, TTFT 674 ms (3 Laufe)vLLMllama.cpp - 286,2 tok/s Generation, 4.696 tok/s Prefill, TTFT 679 ms (24 Laufe)llama.cpp
vLLM 290,3 tok/sllama.cpp 286,2 tok/s
Throughput & latency

Performance benchmark

Metric:
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Nemotron-3.5-Lightning-30B-A3B30BBF16NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
290,27 tok/s TG
Prefill 35.719 · TTFT 1.015 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawDetails →
2Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
286,24 tok/s TG
Prefill 2.858 · TTFT 1.792 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
3Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
277,95 tok/s TG
Prefill 8.575 · TTFT 597 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
4Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
273,62 tok/s TG
Prefill 5.151 · TTFT 994 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
5Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
270,40 tok/s TG
Prefill 6.824 · TTFT 750 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
6Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
265,88 tok/s TG
Prefill 2.862 · TTFT 894 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
7Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
260,59 tok/s TG
Prefill 6.954 · TTFT 368 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
8Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
256,65 tok/s TG
Prefill 4.749 · TTFT 539 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
9Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
253,13 tok/s TG
Prefill 5.903 · TTFT 434 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
10Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
247,13 tok/s TG
Prefill 9.886 · TTFT 518 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
11Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
244,04 tok/s TG
Prefill 5.934 · TTFT 863 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
12Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
241,82 tok/s TG
Prefill 7.918 · TTFT 647 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
13Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
216,36 tok/s TG
Prefill 8.015 · TTFT 319 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
14Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
210,46 tok/s TG
Prefill 6.833 · TTFT 375 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
15Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
202,20 tok/s TG
Prefill 5.476 · TTFT 468 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
16Nemotron-3.5-Lightning-30B-A3B30BBF16NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
170,48 tok/s TG
Prefill 23.477 · TTFT 582 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawDetails →
17Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
127,39 tok/s TG
Prefill 1.511 · TTFT 3.388 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
18Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
121,30 tok/s TG
Prefill 2.872 · TTFT 178 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
19Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
113,57 tok/s TG
Prefill 2.783 · TTFT 184 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
20Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
108,12 tok/s TG
Prefill 2.777 · TTFT 184 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
21Nemotron-3.5-Lightning-30B-A3B30BQ4_K_MNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
105,94 tok/s TG
Prefill 2.835 · TTFT 181 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
22Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
103,17 tok/s TG
Prefill 1.484 · TTFT 1.725 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
23Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
93,53 tok/s TG
Prefill 2.671 · TTFT 192 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
24Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
91,48 tok/s TG
Prefill 3.221 · TTFT 159 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
25Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
90,81 tok/s TG
Prefill 3.309 · TTFT 155 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
26Nemotron-3.5-Lightning-30B-A3B30BBF16NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
53,38 tok/s TG
Prefill 6.812 · TTFT 427 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawDetails →
27Nemotron-3.5-Lightning-30B-A3B30BQ8_0NVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) i
50,67 tok/s TG
Prefill 1.300 · TTFT 394 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmDetails →
Agent & chat rating

Harness benchmark

Noch keine Harnessbenchmarks fuer dieses Modell.