ℹ

Public note: Dell PowerEdge R820 . 2x AMD Radeon RX 7900 XTX (48 GB VRAM gesamt, ROCm/HIP) . RAM: 32x 16 GB SK Hynix DDR3-1600 ECC RDIMM (HMT42GR7AFR4A-PB) = 512 GB @ 1600 MT/s

Generation8,79tok/s

Prefill91,23tok/s

Time to First Token119.748,50ms

Total duration1.200,00s

Concurrency5parallel

Ranking in the field

This GPU All GPUs

114of 120 systems

Performance benchmark · Primary metric: Generation-Speed (tok/s) · 5× concurrent

This run is better than 5 % of all comparable systems.

Generation 8,8 tok/s

-95 % vs Ø 165,1

Prefill 91,2 tok/s

-99 % vs Ø 6.346,8

Time to First Token 119.749 ms

+597 % vs Ø 17.171

Distribution in the field2 – 496 tok/s

Ø 165 Median Dieser Lauf

Wie schlägt sich dieser Benchmark mit anderen Modellen?

Qwen3-0.6BAMD Radeon RX 7900 XTX · run-20260922-095735-570c64

495,5 tok/s

Llama-3.2-1B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20260927-015653-2a7127

479,2 tok/s

Llama-3.2-1B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20260927-001710-b1912f

443,7 tok/s

Llama-3.2-1B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20261008-105916-852cbe

376,5 tok/s

SmolLM2-1.7B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20260927-023029-88b3c3

338,7 tok/s

gemma-4-E2B-itAMD Radeon RX 7900 XTX · run-20260922-115730-99b6cb

331,5 tok/s

Qwen2.5-7B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20260926-171705-b0ed03

326,7 tok/s

Llama-3.2-3B-Instruct2× 2x AMD Radeon RX 7900 XTX · run-20260927-015653-1ca7fc

318,5 tok/s

Llama-3.1-Nemotron-Nano-4B-v1.1AMD Radeon RX 7900 XTX · run-20260922-163818-387a23

318,2 tok/s

gemma-4-E2B-itAMD Radeon RX 7900 XTX · run-20260922-115730-c821a3

316,4 tok/s

gpt-oss-20bAMD Radeon RX 7900 XTX · run-20260922-115731-1da105

314,8 tok/s

Qwen3-1.7B2× 2x AMD Radeon RX 7900 XTX · run-20260926-214312-100b47

314,1 tok/s

gpt-oss-20bAMD Radeon RX 7900 XTX · run-20260922-115731-94445e

310,6 tok/s

Llama-3.1-Nemotron-Nano-4B-v1.1AMD Radeon RX 7900 XTX · run-20260922-163818-e33391

294,9 tok/s

Qwen3.5-122B-A10B this run2× 2x AMD Radeon RX 7900 XTX · run-20261009-081232-77ecca

8,8 tok/s

How does this benchmark compare on other GPUs?

Same model on different hardware · 5× concurrent · Generation (tok/s)

3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20260803-043843-a6bd9a

309,0 tok/s

3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20260803-043845-f65556

308,3 tok/s

3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20260803-043843-3018bc

308,0 tok/s

3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Editionrun-20260803-043845-3050b4

307,7 tok/s

4× AMD Radeon AI PRO R9700run-20260818-011053-b9d7e6

100,6 tok/s

2× AMD Radeon PRO W7900 Dual Slotrun-20260818-151858-d04260

94,1 tok/s

AMD Radeon PRO W7900 Dual Slotrun-20260817-171106-2fa0c1

33,9 tok/s

4× AMD Radeon AI PRO R9700run-20260810-174913-aa5c24

23,9 tok/s

2× AMD Radeon PRO W7900 Dual Slotrun-20260818-155046-605520

23,4 tok/s

NVIDIA GeForce RTX 5070 Tirun-20260803-043846-78ade7

23,2 tok/s

NVIDIA GeForce RTX 5070 Tirun-20260803-043846-da353b

22,6 tok/s

NVIDIA GeForce RTX 5070 Tirun-20260803-043843-84b83c

20,1 tok/s

NVIDIA GeForce RTX 3090 Tirun-20260803-043843-203429

18,8 tok/s

4× AMD Radeon AI PRO R9700run-20260812-011250-288e21

17,5 tok/s

Hardware

GPU: 2x AMD Radeon RX 7900 XTX · 24 GB VRAM
CPU: 4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz
RAM: 504 GB
Mainboard: Dell Inc. PowerEdge R820

Setup

Runtime: llama.cpp
Quantization: Q4_K_M
Model: Qwen3.5-122B-A10B

Configuration

benchmark-konfiguration — run-20261009-081232-77ecca

# LLM-Benchmark Konfiguration # Modell : Qwen3.5-122B-A10B # Engine : llama.cpp # Run-ID : run-20261009-081232-77ecca # GPU : 2x AMD Radeon RX 7900 XTX # CPU : 4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz # RAM : 504 GB bench@llm-benchmark:~$ llama-server \ -m Qwen_Qwen3.5-122B-A10B-Q4_K_M/Qwen_Qwen3.5-122B-A10B-Q4_K_M-00001-of-00002.gguf \ --alias Qwen3.5-122B-A10B \ -ngl 999 \ -sm layer \ -fa on \ -c 40960 \ -np 10 \ --n-cpu-moe 99 '(2x' RX 7900 XTX ROCm/HIP, 'moe-offload)'

Engine?Die Inferenz-Software, die das Modell ausliefert (z.B. vLLM oder llama.cpp). Sie bestimmt Geschwindigkeit, unterstuetzte Modellformate und welche Parameter ueberhaupt verfuegbar sind.	llamacpp
Modellalias?Der Name, unter dem das Modell ueber die API angesprochen wird. Genau dieser Wert muss im Request-Feld 'model' stehen.	Qwen3.5-122B-A10B
Kontextlaenge?Maximale Anzahl Tokens (Eingabe + erzeugte Ausgabe zusammen), die das Modell pro Anfrage verarbeiten kann.	40960
Modellpfad?Pfad zur GGUF-Modelldatei, die geladen und ausgeliefert wird.	Qwen_Qwen3.5-122B-A10B-Q4_K_M/Qwen_Qwen3.5-122B-A10B-Q4_K_M-00001-of-00002.gguf
GPU-Layer?Anzahl der auf die GPU ausgelagerten Modell-Layer. Hoeher = mehr VRAM und schneller; der Rest laeuft auf der CPU. 999 = alles auf GPU.	999
Split-Mode?Verteilung ueber mehrere GPUs: none (nur eine GPU), layer (Layer aufteilen) oder row (Tensoren zeilenweise).	layer
fa	on
Kontext?Groesse des Kontextfensters in Token. 0 = der beim Training verwendete Kontext des Modells.	40960
np	10
n-cpu-moe	99
execution_type	local

All benchmarks of this model To leaderboard

Model comparison

Qwen3.5-122B-A10B on various hardware

All published performance runs of this model – each bubble a variant: position = prefill (X) × generation (Y), bubble size = number of runs. Closer to the top right = faster. ★ Marked gold = this benchmark.

GPUby graphics card

NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 564,6 tok/sAMD Radeon AI PRO R9700 168,6 tok/sAMD Radeon PRO W7900 Dual Slot 96,5 tok/sNVIDIA GeForce RTX 5070 Ti 31,1 tok/sNVIDIA GeForce RTX 3090 Ti 24,9 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 12,1 tok/s★ AMD Radeon RX 7900 XTX 11,8 tok/s this runNVIDIA GeForce RTX 5090 8,3 tok/sNVIDIA GeForce RTX 2060 7,1 tok/s

CPUby processor

AMD Ryzen Threadripper PRO 9965WX 24-Cores 564,6 tok/sAMD Ryzen Threadripper PRO 7955WX 16-Cores 168,6 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 96,5 tok/sAMD Ryzen 9 8945HX with Radeon Graphics 24,9 tok/sAMD Ryzen 9 9950X 16-Core Processor 12,1 tok/s★ Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 11,8 tok/s this runAMD Ryzen 7 5800X3D 8-Core Processor 8,3 tok/s

MBby mainboard

ASUSTeK COMPUTER INC. Pro WS WRX90E-SAGE SE 564,6 tok/sASUSTeK COMPUTER INC. Pro WS WRX80E-SAGE SE WIFI 96,5 tok/sMeigao Innovation Technology (Shen Zhen) Co., Ltd DRFXL (MotherBoard Series) 24,9 tok/sASUSTeK COMPUTER INC. ProArt X870E-CREATOR WIFI 12,1 tok/s★ Dell Inc. PowerEdge R820 11,8 tok/s this runASUSTeK COMPUTER INC. ROG STRIX B550-A GAMING 8,3 tok/s

ENGby engine

★ llama.cpp 564,6 tok/s this runvLLM 168,6 tok/s

DRVby driver

unbekannt 564,6 tok/s

💰 Economics

Economics of this run

Operating cost, TCO and comparison with the next-best runs of the same model at identical concurrency (5× concurrent). Methodology →

⚙️ ConfigurationAll metrics and charts below follow these settings – based on a 24-month runtime.Save to URL Reset

⚡ Electricity price EUR/kWh

⚙️ System utilization 100 %

🖥️ Acquisition EUR

🔌 Idle 35 W

⚡ TDP 749 W

☁️ External LLM (API) show

Input €/1M Output €/1M Input per output token

Electricity0.30 EUR/kWh

Avg power (incl. idle)749 W estimated (TDP)GPU 710 + CPU 29 + Board 10 W full load

Avg cost / hourEUR 0.22

Electricity / 1M tokensEUR 7.10

Token / kWh42.26K

Acquisition (system)EUR 4,353 partial priceGPU EUR 2,098 · CPU EUR 59 · RAM EUR 2,016 · PSU EUR 180

Electricity (2 years)–

TCO (2 years)EUR 8,288

Output tokens (2 years)554.40M

☁️ External LLM (API) – comparison

External LLM cost (2 years)–

Savings vs. external (2 years)–

All values above and the charts below take the configured system utilization into account: at X% the system generates only X% of the time, the rest it idles (35 W). Cost per hour drops (more idle), cost per token rises.

Cost over 2 years – electricity only

Cost over 2 years – incl. acquisition (TCO)

Speed vs. tokens per euro

Euro per 1M tokens

Comparison vs. API – economics per benchmark

	Qwen3.5-122B-A10B2x AMD Radeon RX 7900 XTX	Qwen3.5-122B-A10B3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition	Qwen3.5-122B-A10B3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition	Qwen3.5-122B-A10B3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
Electricity cost (24 mo.)	–	–	–	–
Acquisition cost	–	–	–	–
Total cost (TCO)	–	–	–	–
Generated tokens (24 mo.)	–	–	–	–
Token price via API	–	–	–	–
Break-even point (days)	–	–	–	–
Result (savings / extra cost)	–	–	–	–

2× AMD Radeon RX 7900 XTX · this run 9 tok/s 3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 309 tok/s 3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 308 tok/s 3× NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 308 tok/s

Comparison with up to 3 next-best runs of this model at the same concurrency (at least one on different hardware). Power = GPU TDP + CPU (idle + 15 %) + board (estimated), acquisition = full system (GPU + CPU + board + RAM + PSU), prices = stored market prices.

Contributed by

Mario Alka Administrator

@marioalka

Ich bin Unternehmer, Softwareentwickler und KI-Enthusiast. Seit vielen Jahren entwickle ich Unternehmenssoftware und beschäftige mich inzwischen fast täglich mit lokalen LLMs, KI-Agenten und leistungsfähiger KI-Hardware.

Mit LLM-Benchmark.de möchte ich eine Plattform schaffen, auf der Modelle, GPUs und Agenten objektiv und reproduzierbar miteinander verglichen werden.