ℹ

Public note: NVIDIA DGX Spark GB10 (Grace-Blackwell, 120 GB unified, ARM64) . Qwen3.8-Flash-Next . vLLM(NVFP4, NVMe/PLE-Offload) + llama.cpp(IQ3_XXS)

ℹ

For context: Diese Plattform nutzt Unified Memory – die "VRAM" ist gemeinsamer System-RAM (APU/Superchip); das Modell teilt sich den Speicher mit dem System.

Generation36,38tok/s

Prefill714,11tok/s

Time to First Token2.759,50ms

Total duration61,82s

Concurrency1parallel

Ranking in the field

This GPU All GPUs

26of 38 systems

Performance benchmark · Primary metric: Generation-Speed (tok/s) · 1× concurrent

This run is better than 32 % of all comparable systems.

Generation 36,4 tok/s

-30 % vs Ø 52,3

Prefill 714,1 tok/s

-72 % vs Ø 2.556,0

Time to First Token 2.760 ms

+115 % vs Ø 1.286

Distribution in the field11 – 101 tok/s

Ø 52 Median Dieser Lauf

Wie schlägt sich dieser Benchmark mit anderen Modellen?

Qwen3.6-35B-A3BNVIDIA GB10 (DGX Spark) · run-20260809-110710-d6b1e3

101,2 tok/s

Qwen3.6-35B-A3BNVIDIA GB10 (DGX Spark) · run-20260809-110710-a1889b

100,6 tok/s

Qwen3.6-35B-A3BNVIDIA GB10 (DGX Spark) · run-20260809-110710-c441e7

93,0 tok/s

Qwen3-Coder-30B-A3B-InstructNVIDIA GB10 (DGX Spark) · run-20260820-205402-6695a4

86,2 tok/s

Qwen3-30B-A3B-Instruct-2507NVIDIA GB10 (DGX Spark) · run-20260820-194812-0bbbbe

84,2 tok/s

gpt-oss-20bNVIDIA GB10 (DGX Spark) · run-20260820-194812-a1fa02

82,2 tok/s

Nemotron-Cascade-2-30B-A3BNVIDIA GB10 (DGX Spark) · run-20260820-124207-595ea8

77,0 tok/s

Nemotron-3-Nano-4BNVIDIA GB10 (DGX Spark) · run-20260730-084906-365733

76,9 tok/s

Qwen3.6-35B-A3BNVIDIA GB10 (DGX Spark) · run-20260809-110710-89be7e

74,2 tok/s

Nemotron-3-Nano-4BNVIDIA GB10 (DGX Spark) · run-20260730-035052-355f35

72,7 tok/s

Qwen3.6-35B-A3BNVIDIA GB10 (DGX Spark) · run-20260821-071254-0f1b4f

68,4 tok/s

gemma-4-E4B-itNVIDIA GB10 (DGX Spark) · run-20260730-084905-ef5ef1

63,5 tok/s

Nemotron-Cascade-2-30B-A3BNVIDIA GB10 (DGX Spark) · run-20260820-124207-7e0db9

62,0 tok/s

gemma-4-E4B-itNVIDIA GB10 (DGX Spark) · run-20260729-224611-11c64f

60,5 tok/s

Qwen3.8-Flash-Next this runNVIDIA GB10 (DGX Spark) · run-20261013-151009-529362

36,4 tok/s

How does this benchmark compare on other GPUs?

Same model on different hardware · 1× concurrent · Generation (tok/s)

NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20261012-182514-5f16dc

164,6 tok/s

NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20261012-175110-537c3d

158,1 tok/s

NVIDIA RTX A6000run-20261021-093815-fdc402

125,2 tok/s

NVIDIA GeForce RTX 3090run-20261013-092430-8e3577

116,4 tok/s

NVIDIA RTX A6000run-20261021-093816-efa091

115,5 tok/s

NVIDIA Tesla V100 PCIe 32GBrun-20261021-093816-154b4e

59,3 tok/s

NVIDIA Tesla V100 PCIe 32GBrun-20261019-134138-464b84

52,1 tok/s

NVIDIA GeForce RTX 3090run-20261013-165111-42c54a

50,5 tok/s

NVIDIA GB10run-20261013-151008-44a477

49,9 tok/s

NVIDIA Tesla V100 PCIe 32GBrun-20261019-100734-f18656

49,6 tok/s

NVIDIA GB10run-20261013-165111-ea315f

41,8 tok/s

2× NVIDIA RTX A6000run-20261014-073140-f006db

38,8 tok/s

2× NVIDIA RTX A6000run-20260930-183837-07a63e

37,0 tok/s

NVIDIA GB10 this runrun-20261013-151009-529362

36,4 tok/s

Hardware

GPU: NVIDIA GB10 · 128 GB VRAM
CPU: NVIDIA Grace
RAM: 120 GB
Mainboard: ASUSTeK COMPUTER INC. GX10

Setup

Runtime: llama.cpp
Quantization: IQ3_XXS
Driver: NVIDIA 590.48.01 / CUDA 13.1
Model: Qwen3.8-Flash-Next

Configuration

benchmark-konfiguration — run-20261013-151009-529362

# LLM-Benchmark Konfiguration # Modell : Qwen3.8-Flash-Next # Engine : llama.cpp # Run-ID : run-20261013-151009-529362 # GPU : NVIDIA GB10 # CPU : NVIDIA Grace # RAM : 120 GB bench@llm-benchmark:~$ llama-server \ -m Qwen3.8-Flash-Next-IQ3_XXS \ --alias Qwen3.8-Flash-Next \ -ngl 999 \ -c 32768 \ -np 10 \ -fa on '(DGX' Spark GB10, ARM+CUDA, frischer 'Build)'

Engine?Die Inferenz-Software, die das Modell ausliefert (z.B. vLLM oder llama.cpp). Sie bestimmt Geschwindigkeit, unterstuetzte Modellformate und welche Parameter ueberhaupt verfuegbar sind.	llamacpp
Modellalias?Der Name, unter dem das Modell ueber die API angesprochen wird. Genau dieser Wert muss im Request-Feld 'model' stehen.	Qwen3.8-Flash-Next
Kontextlaenge?Maximale Anzahl Tokens (Eingabe + erzeugte Ausgabe zusammen), die das Modell pro Anfrage verarbeiten kann.	32768
Modellpfad?Pfad zur GGUF-Modelldatei, die geladen und ausgeliefert wird.	Qwen3.8-Flash-Next-IQ3_XXS
GPU-Layer?Anzahl der auf die GPU ausgelagerten Modell-Layer. Hoeher = mehr VRAM und schneller; der Rest laeuft auf der CPU. 999 = alles auf GPU.	999
Kontext?Groesse des Kontextfensters in Token. 0 = der beim Training verwendete Kontext des Modells.	32768
np	10
fa	on
execution_type	local

All benchmarks of this model To leaderboard

Model comparison

Qwen3.8-Flash-Next on various hardware

All published performance runs of this model – each bubble a variant: position = prefill (X) × generation (Y), bubble size = number of runs. Closer to the top right = faster. ★ Marked gold = this benchmark.

GPUby graphics card

NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 1.564,6 tok/sNVIDIA RTX A6000 1.171,8 tok/sNVIDIA GeForce RTX 3090 1.132,7 tok/s★ NVIDIA GB10 (DGX Spark) 492,8 tok/s this runNVIDIA Tesla V100 PCIe 32GB 196,2 tok/sAMD Radeon RX 7900 XTX 18,7 tok/s

CPUby processor

AMD Ryzen 9 7945HX with Radeon Graphics 1.564,6 tok/sAMD Ryzen Threadripper PRO 3945WX 12-Cores 1.171,8 tok/sAMD Ryzen 5 5600X 6-Core Processor 1.132,7 tok/s★ NVIDIA Grace 492,8 tok/s this runAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics 196,2 tok/sAMD EPYC 7203P 8-Core Processor 79,2 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 63,2 tok/sAMD Ryzen Threadripper PRO 3955WX 16-Cores 18,7 tok/s

MBby mainboard

Shenzhen Meigao Electronic Equipment Co.,Ltd DRFXI (MotherBoard Series) 1.564,6 tok/sunbekannt 1.171,8 tok/sASUSTeK COMPUTER INC. ROG STRIX B550-A GAMING 1.132,7 tok/s★ ASUSTeK COMPUTER INC. GX10 492,8 tok/s this runASUSTeK COMPUTER INC. ROG STRIX X570-F GAMING 196,2 tok/sASRockRack ROMED8-2T/BCM 79,2 tok/sDell Inc. PowerEdge R820 63,2 tok/sASUSTeK COMPUTER INC. Pro WS WRX80E-SAGE SE WIFI 35,4 tok/s

ENGby engine

strata 1.564,6 tok/s★ llama.cpp 1.555,3 tok/s this rununbekannt 1.171,8 tok/svLLM 122,4 tok/s

DRVby driver

unbekannt 1.564,6 tok/s★ NVIDIA 590.48.01 / CUDA 13.1 492,8 tok/s this run

💰 Economics

Economics of this run

Operating cost, TCO and comparison with the next-best runs of the same model at identical concurrency (1× concurrent). Methodology →

⚙️ ConfigurationAll metrics and charts below follow these settings – based on a 24-month runtime.Save to URL Reset

⚡ Electricity price EUR/kWh

⚙️ System utilization 100 %

🖥️ Acquisition EUR

🔌 Idle 15 W

⚡ TDP 140 W

☁️ External LLM (API) show

Input €/1M Output €/1M Input per output token

Electricity0.30 EUR/kWh

Avg power (incl. idle)140 W estimated (TDP)GPU 140 W full load

Avg cost / hourEUR 0.042

Electricity / 1M tokensEUR 0.32

Token / kWh935.49K

Acquisition (system)EUR 5,800 partial priceGPU EUR 4,000 · RAM EUR 1,680 · PSU EUR 120

Electricity (2 years)–

TCO (2 years)EUR 6,536

Output tokens (2 years)2.29B

☁️ External LLM (API) – comparison

External LLM cost (2 years)–

Savings vs. external (2 years)–

All values above and the charts below take the configured system utilization into account: at X% the system generates only X% of the time, the rest it idles (15 W). Cost per hour drops (more idle), cost per token rises.

Cost over 2 years – electricity only

Cost over 2 years – incl. acquisition (TCO)

Speed vs. tokens per euro

Euro per 1M tokens

Comparison vs. API – economics per benchmark

	Qwen3.8-Flash-NextNVIDIA GB10 (DGX Spark)	Qwen3.8-Flash-NextNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition	Qwen3.8-Flash-NextNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition	Qwen3.8-Flash-NextNVIDIA RTX A6000
Electricity cost (24 mo.)	–	–	–	–
Acquisition cost	–	–	–	–
Total cost (TCO)	–	–	–	–
Generated tokens (24 mo.)	–	–	–	–
Token price via API	–	–	–	–
Break-even point (days)	–	–	–	–
Result (savings / extra cost)	–	–	–	–

NVIDIA GB10 (DGX Spark) · this run 36 tok/s NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 165 tok/s NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 158 tok/s NVIDIA RTX A6000 125 tok/s

Comparison with up to 3 next-best runs of this model at the same concurrency (at least one on different hardware). Power = GPU TDP + CPU (idle + 15 %) + board (estimated), acquisition = full system (GPU + CPU + board + RAM + PSU), prices = stored market prices.

Contributed by

Mario Alka Administrator

@marioalka

Ich bin Unternehmer, Softwareentwickler und KI-Enthusiast. Seit vielen Jahren entwickle ich Unternehmenssoftware und beschäftige mich inzwischen fast täglich mit lokalen LLMs, KI-Agenten und leistungsfähiger KI-Hardware.

Mit LLM-Benchmark.de möchte ich eine Plattform schaffen, auf der Modelle, GPUs und Agenten objektiv und reproduzierbar miteinander verglichen werden.