Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Anmelden Registrieren
Beta
NVIDIA

Nemotron-3.5-Lightning-30B-A3B30B

Benchmarkprofil und veröffentlichte Ergebnisse.

Hybrid Mamba-2 + MoE (3B aktiv)30B
Einordnung im Feld

Bestwerte im Vergleich

Beste Messwerte dieses Modells gegen Minimum, Durchschnitt und Maximum aller veröffentlichten Systeme.

NVIDIA GeForce RTX 3090 TiAlle GPUs
Generation2.263,7 tok/s
Min 0,0Ø 265,2Max 2.263,7
Spitzenwert im Feld · 273 Systeme im Feld
Prefill9.016 tok/s
Min 5Ø 3.549Max 32.005
Ueber dem Durchschnitt · 273 Systeme im Feld
Time to First Token346 ms
Min 20Ø 78.661Max 459.880
Ueber dem Durchschnitt · 273 Systeme im Feld
Performanceprofil

Durchsatz nach Hardware & Engine

Jede Blase steht für eine GPU, CPU bzw. Engine – die Position zeigt Prompt-Verarbeitung (X) und Ausgabe-Geschwindigkeit (Y), die Blasengröße die Anzahl der Messläufe.

GPUnach Grafikkarte

2.8852.1641.4427210,05.4716.6207.7688.917Prefill (tok/s)Generation (tok/s)NVIDIA GeForce RTX 3090 Ti - 2.263,7 tok/s Generation, 8.103 tok/s Prefill, TTFT 10.659 ms (3 Laufe)NVIDIA GeForce RTX 30...AMD Radeon AI PRO R9700 - 290,3 tok/s Generation, 6.285 tok/s Prefill, TTFT 963 ms (30 Laufe)AMD Radeon AI PRO R97...
NVIDIA GeForce RTX 3090 Ti 2.263,7 tok/sAMD Radeon AI PRO R9700 290,3 tok/s

CPUnach Prozessor

2.8862.1651.4437220,01.9214.4346.9469.458Prefill (tok/s)Generation (tok/s)AMD Ryzen 5 5600X 6-Core Processor - 2.263,7 tok/s Generation, 8.103 tok/s Prefill, TTFT 10.659 ms (3 Laufe)AMD Ryzen 5 5600X 6-C...AMD Ryzen Threadripper PRO 7955WX 16-Cores - 290,3 tok/s Generation, 6.619 tok/s Prefill, TTFT 679 ms (27 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 3 3100 4-Core Processor - 283,5 tok/s Generation, 3.276 tok/s Prefill, TTFT 3.524 ms (3 Laufe)AMD Ryzen 3 3100 4-Co...
AMD Ryzen 5 5600X 6-Core Processor 2.263,7 tok/sAMD Ryzen Threadripper PRO 7955WX 16-Cores 290,3 tok/sAMD Ryzen 3 3100 4-Core Processor 283,5 tok/s

ENGnach Engine

2.8862.1641.4437210,01.7427.11112.48017.849Prefill (tok/s)Generation (tok/s)vLLM - 2.263,7 tok/s Generation, 15.053 tok/s Prefill, TTFT 5.666 ms (6 Laufe)vLLMllama.cpp - 286,2 tok/s Generation, 4.538 tok/s Prefill, TTFT 995 ms (27 Laufe)llama.cpp
vLLM 2.263,7 tok/sllama.cpp 286,2 tok/s
Durchsatz & Latenz

Performancebenchmark

Messwert:
GPUs
#Modell / HerstellerMesswerteParallelGPU / CPU / RAMRuntime
1Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
2.263,71 tok/s TG
Prefill 9.016 · TTFT 21.818 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllmNVFP4Details →
2Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
1.134,97 tok/s TG
Prefill 8.757 · TTFT 9.812 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllmNVFP4Details →
3Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
290,27 tok/s TG
Prefill 35.719 · TTFT 1.015 ms
10×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawBF16Details →
4Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
286,24 tok/s TG
Prefill 2.858 · TTFT 1.792 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
5Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
283,50 tok/s TG
Prefill 4.190 · TTFT 6.068 ms
10×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
6Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
277,95 tok/s TG
Prefill 8.575 · TTFT 597 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
7Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
273,62 tok/s TG
Prefill 5.151 · TTFT 994 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
8Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
270,40 tok/s TG
Prefill 6.824 · TTFT 750 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
9Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
265,88 tok/s TG
Prefill 2.862 · TTFT 894 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
10Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
260,59 tok/s TG
Prefill 6.954 · TTFT 368 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
11Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
256,65 tok/s TG
Prefill 4.749 · TTFT 539 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
12Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
253,13 tok/s TG
Prefill 5.903 · TTFT 434 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
13Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
247,13 tok/s TG
Prefill 9.886 · TTFT 518 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
14Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
244,04 tok/s TG
Prefill 5.934 · TTFT 863 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
15Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
241,82 tok/s TG
Prefill 7.918 · TTFT 647 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
16Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
227,41 tok/s TG
Prefill 6.536 · TTFT 346 ms
1×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllmNVFP4Details →
17Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
216,36 tok/s TG
Prefill 8.015 · TTFT 319 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
18Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
213,81 tok/s TG
Prefill 3.032 · TTFT 3.678 ms
5×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
19Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
210,46 tok/s TG
Prefill 6.833 · TTFT 375 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
20Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
202,20 tok/s TG
Prefill 5.476 · TTFT 468 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
21Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
170,48 tok/s TG
Prefill 23.477 · TTFT 582 ms
5×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawBF16Details →
22Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
127,39 tok/s TG
Prefill 1.511 · TTFT 3.388 ms
10×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
23Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
121,30 tok/s TG
Prefill 2.872 · TTFT 178 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
24Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
113,57 tok/s TG
Prefill 2.783 · TTFT 184 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
25Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
112,45 tok/s TG
Prefill 2.607 · TTFT 827 ms
1×AMD Radeon AI PRO R9700AMD Ryzen 3 3100 4-Core Processor · 31 GB RAMllama.cppopenclaw_cliQ4_K_MDetails →
26Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
108,12 tok/s TG
Prefill 2.777 · TTFT 184 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
27Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
105,94 tok/s TG
Prefill 2.835 · TTFT 181 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ4_K_MDetails →
28Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
103,17 tok/s TG
Prefill 1.484 · TTFT 1.725 ms
5×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
29Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
93,53 tok/s TG
Prefill 2.671 · TTFT 192 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
30Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
91,48 tok/s TG
Prefill 3.221 · TTFT 159 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
31Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
90,81 tok/s TG
Prefill 3.309 · TTFT 155 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
32Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
53,38 tok/s TG
Prefill 6.812 · TTFT 427 ms
1×4x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMvLLMgodclawBF16Details →
33Nemotron-3.5-Lightning-30B-A3B30BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
50,67 tok/s TG
Prefill 1.300 · TTFT 394 ms
1×4x AMD Radeon AI PRO R9700 iAMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAMllama.cppopenclaw_cli batched-bench ROCmQ8_0Details →
Agent- & Chat-Bewertung

Harnessbenchmark

Noch keine Harnessbenchmarks fuer dieses Modell.