Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
Contributed byMario AlkaQwen (Alibaba)

Qwen3.8-Flash-Next

Harness benchmark · measured on 08.10.2026 07:15

Benchmark-IDrun-20261023-180238-6829a9
Kolibri-Qwen Diverse Deterministic 210 V1.0MoE180BRuntime: deterministic210Quantisierung: NVFP4
ℹ
Public note: Variante: Uncensored (mazinb).
⚠
Configuration note: Es sind 2 Grafikkarten installiert, aber nur 1 wurde fuer diesen Lauf genutzt. Mit allen 2 GPUs koennte der Durchsatz deutlich hoeher liegen.
Points145,00von 210,00
Points rate69,05%
Tasks passed145von 210
Total duration145,36s
Ranking in the field
17of 32 systems

Harness benchmark · Primary metric: Score rate (%)

This run is better than 48 % of all comparable systems.
Score rate 69 %
-6 % vs Ø 73,3
Tasks passed 145
+17 % vs Ø 124,4
Total duration 145,4 s
-87 % vs Ø 1.133,5
Distribution in the field37 – 100 %
Ø 73 Median Dieser Lauf

Wie schlägt sich dieser Benchmark mit anderen Modellen?

Qwen3.8-Flash-Next same model2× 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261023-180243-30647c
100 %
Kolibri-1NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261008-165122-c5dea7
100 %
Qwen3.6-35B-A3B-AWQNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260929-153935-691df9
100 %
Qwen3.6-35B-A3B-AWQNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260929-091748-cbe456
100 %
Qwen3.8-27B-Uncensored2× 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260929-091747-e13da0
100 %
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260929-091747-5d8477
100 %
Kolibri-1NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261008-125923-3f1672
99 %
Qwen3.8-Flash-Next same model2× 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261023-180239-3519b5
98 %
Qwen3.6-35B-A3B-AWQNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260929-112249-dba0f2
97 %
Qwen3.8-Flash-Next same model2× 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261023-180239-2b7b40
96 %
Kolibri-1NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261008-125923-43651d
93 %
Kolibri-1NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261008-125923-5b5304
91 %
Gemma-4-26B-A4B-itNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260722-161611-05d866
91 %
MiniMax-M2.53× 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20260722-161611-f6cfe1
81 %
Qwen3.8-Flash-Next this run2× 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · run-20261023-180238-6829a9
69 %

Hardware

GPU: 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · 96 GB VRAM
CPU: AMD Ryzen Threadripper PRO 9965WX 24-Cores
RAM: 125 GB
Mainboard: ASUSTeK COMPUTER INC. Pro WS WRX90E-SAGE SE

Setup

Runtime: deterministic210
Quantization: NVFP4
Model: Qwen3.8-Flash-Next

Aufgaben (210)

TaskStatusPointsDuration
arithmetic-01passed1 / 10,47 s
arithmetic-02failed0 / 10,47 s
arithmetic-03passed1 / 10,49 s
arithmetic-04failed0 / 10,48 s
arithmetic-05passed1 / 10,41 s
arithmetic-06passed1 / 10,53 s
arithmetic-07failed0 / 10,52 s
arithmetic-08passed1 / 10,52 s
arithmetic-09failed0 / 10,44 s
arithmetic-10failed0 / 10,44 s
arithmetic-11passed1 / 10,78 s
arithmetic-12failed0 / 10,78 s
arithmetic-13passed1 / 10,84 s
arithmetic-14failed0 / 10,69 s
arithmetic-15failed0 / 10,78 s
ordering_logic-01passed1 / 10,66 s
ordering_logic-02passed1 / 11,06 s
ordering_logic-03passed1 / 11,06 s
ordering_logic-04passed1 / 11,02 s
ordering_logic-05passed1 / 10,62 s
ordering_logic-06failed0 / 10,58 s
ordering_logic-07passed1 / 10,77 s
ordering_logic-08passed1 / 11,12 s
ordering_logic-09passed1 / 11,12 s
ordering_logic-10failed0 / 10,71 s
ordering_logic-11passed1 / 11,02 s
ordering_logic-12failed0 / 10,68 s
ordering_logic-13passed1 / 10,61 s
ordering_logic-14passed1 / 11,06 s
ordering_logic-15passed1 / 11,08 s
extraction-01failed0 / 10,94 s
extraction-02passed1 / 10,96 s
extraction-03passed1 / 10,91 s
extraction-04failed0 / 10,95 s
extraction-05failed0 / 10,92 s
extraction-06failed0 / 11,36 s
extraction-07failed0 / 10,97 s
extraction-08failed0 / 10,86 s
extraction-09failed0 / 10,98 s
extraction-10failed0 / 11,20 s
extraction-11passed1 / 11,05 s
extraction-12passed1 / 11,04 s
extraction-13passed1 / 11,03 s
extraction-14passed1 / 10,89 s
extraction-15passed1 / 10,91 s
code_comprehension-01passed1 / 10,36 s
code_comprehension-02passed1 / 10,42 s
code_comprehension-03passed1 / 10,58 s
code_comprehension-04passed1 / 10,57 s
code_comprehension-05failed0 / 10,48 s
code_comprehension-06passed1 / 10,48 s
code_comprehension-07passed1 / 10,66 s
code_comprehension-08passed1 / 10,56 s
code_comprehension-09passed1 / 10,75 s
code_comprehension-10failed0 / 10,55 s
code_comprehension-11passed1 / 10,41 s
code_comprehension-12failed0 / 10,80 s
code_comprehension-13passed1 / 10,63 s
code_comprehension-14passed1 / 10,79 s
code_comprehension-15failed0 / 10,58 s
grounded_qa-01passed1 / 10,49 s
grounded_qa-02passed1 / 10,50 s
grounded_qa-03passed1 / 10,45 s
grounded_qa-04passed1 / 10,49 s
grounded_qa-05passed1 / 10,40 s
grounded_qa-06passed1 / 10,35 s
grounded_qa-07passed1 / 10,37 s
grounded_qa-08passed1 / 10,39 s
grounded_qa-09passed1 / 10,33 s
grounded_qa-10passed1 / 10,37 s
grounded_qa-11passed1 / 10,36 s
grounded_qa-12passed1 / 10,38 s
grounded_qa-13passed1 / 10,43 s
grounded_qa-14passed1 / 10,45 s
grounded_qa-15passed1 / 10,38 s
classification-01passed1 / 10,63 s
classification-02passed1 / 10,66 s
classification-03passed1 / 10,65 s
classification-04failed0 / 10,59 s
classification-05passed1 / 10,62 s
classification-06passed1 / 10,54 s
classification-07passed1 / 10,72 s
classification-08passed1 / 10,73 s
classification-09passed1 / 10,70 s
classification-10passed1 / 11,07 s
classification-11failed0 / 10,54 s
classification-12passed1 / 10,58 s
classification-13passed1 / 10,51 s
classification-14passed1 / 10,61 s
classification-15passed1 / 10,61 s
path_planning-01failed0 / 10,77 s
path_planning-02failed0 / 10,72 s
path_planning-03passed1 / 10,72 s
path_planning-04failed0 / 10,69 s
path_planning-05passed1 / 10,69 s
path_planning-06passed1 / 11,26 s
path_planning-07passed1 / 10,86 s
path_planning-08passed1 / 10,82 s
path_planning-09failed0 / 10,99 s
path_planning-10failed0 / 11,13 s
path_planning-11passed1 / 10,95 s
path_planning-12passed1 / 10,79 s
path_planning-13failed0 / 10,74 s
path_planning-14failed0 / 10,83 s
path_planning-15failed0 / 10,69 s
relational_query-01passed1 / 10,54 s
relational_query-02passed1 / 10,45 s
relational_query-03passed1 / 10,44 s
relational_query-04passed1 / 10,45 s
relational_query-05passed1 / 10,45 s
relational_query-06passed1 / 10,50 s
relational_query-07passed1 / 10,52 s
relational_query-08passed1 / 10,52 s
relational_query-09passed1 / 10,50 s
relational_query-10passed1 / 10,46 s
relational_query-11passed1 / 10,57 s
relational_query-12passed1 / 10,54 s
relational_query-13passed1 / 10,52 s
relational_query-14passed1 / 10,48 s
relational_query-15passed1 / 10,44 s
date_time-01passed1 / 10,64 s
date_time-02passed1 / 10,84 s
date_time-03failed0 / 10,73 s
date_time-04passed1 / 10,63 s
date_time-05passed1 / 10,83 s
date_time-06failed0 / 10,71 s
date_time-07passed1 / 10,63 s
date_time-08failed0 / 10,69 s
date_time-09failed0 / 10,60 s
date_time-10passed1 / 10,54 s
date_time-11failed0 / 11,24 s
date_time-12failed0 / 11,14 s
date_time-13passed1 / 11,08 s
date_time-14failed0 / 11,05 s
date_time-15failed0 / 11,03 s
units-01passed1 / 10,57 s
units-02passed1 / 10,58 s
units-03passed1 / 10,51 s
units-04failed0 / 10,58 s
units-05passed1 / 10,52 s
units-06passed1 / 10,54 s
units-07passed1 / 10,64 s
units-08passed1 / 10,46 s
units-09passed1 / 10,44 s
units-10passed1 / 10,49 s
units-11passed1 / 10,41 s
units-12passed1 / 10,60 s
units-13passed1 / 10,50 s
units-14passed1 / 10,57 s
units-15passed1 / 10,55 s
string_processing-01passed1 / 10,46 s
string_processing-02passed1 / 10,71 s
string_processing-03passed1 / 10,72 s
string_processing-04failed0 / 10,59 s
string_processing-05passed1 / 10,61 s
string_processing-06passed1 / 10,53 s
string_processing-07failed0 / 10,62 s
string_processing-08passed1 / 10,77 s
string_processing-09passed1 / 10,70 s
string_processing-10failed0 / 10,59 s
string_processing-11passed1 / 10,60 s
string_processing-12passed1 / 10,73 s
string_processing-13passed1 / 10,59 s
string_processing-14passed1 / 10,61 s
string_processing-15passed1 / 10,68 s
sets-01failed0 / 11,14 s
sets-02failed0 / 10,88 s
sets-03failed0 / 11,57 s
sets-04failed0 / 10,80 s
sets-05failed0 / 10,57 s
sets-06failed0 / 11,68 s
sets-07failed0 / 10,86 s
sets-08failed0 / 10,95 s
sets-09failed0 / 11,49 s
sets-10failed0 / 10,66 s
sets-11failed0 / 10,74 s
sets-12failed0 / 11,97 s
sets-13failed0 / 11,09 s
sets-14failed0 / 11,01 s
sets-15failed0 / 11,71 s
constraint_selection-01passed1 / 10,48 s
constraint_selection-02failed0 / 10,31 s
constraint_selection-03passed1 / 10,77 s
constraint_selection-04passed1 / 10,52 s
constraint_selection-05passed1 / 10,78 s
constraint_selection-06passed1 / 10,64 s
constraint_selection-07passed1 / 10,90 s
constraint_selection-08passed1 / 10,86 s
constraint_selection-09passed1 / 10,54 s
constraint_selection-10passed1 / 10,75 s
constraint_selection-11passed1 / 10,37 s
constraint_selection-12passed1 / 10,61 s
constraint_selection-13passed1 / 10,60 s
constraint_selection-14passed1 / 10,90 s
constraint_selection-15failed0 / 10,53 s
probability-01passed1 / 10,50 s
probability-02passed1 / 10,58 s
probability-03passed1 / 10,54 s
probability-04passed1 / 10,68 s
probability-05passed1 / 10,51 s
probability-06passed1 / 10,54 s
probability-07failed0 / 10,60 s
probability-08passed1 / 10,45 s
probability-09passed1 / 10,43 s
probability-10failed0 / 10,54 s
probability-11passed1 / 10,46 s
probability-12passed1 / 10,43 s
probability-13failed0 / 10,42 s
probability-14passed1 / 10,44 s
probability-15passed1 / 10,45 s

Configuration

benchmark-konfiguration — run-20261023-180238-6829a9
# LLM-Benchmark Konfiguration # Modell : Qwen3.8-Flash-Next # Engine : vLLM # Run-ID : run-20261023-180238-6829a9 # GPU : 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition # CPU : AMD Ryzen Threadripper PRO 9965WX 24-Cores # RAM : 125 GB bench@llm-benchmark:~$ /usr/bin/python3 /usr/local/bin/vllm serve mazinb/Qwen3.8-Flash-Next-Uncensored-NVFP4 \ --served-model-name Qwen3.8-Flash-Next-Uncensored \ --host 0.0.0.0 \ --port 8000 \ --trust-remote-code \ --distributed-executor-backend mp \ --gpu-memory-utilization 0.90 \ --max-model-len 131072 \ --max-num-seqs 10 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3
Engine?Die Inferenz-Software, die das Modell ausliefert (z.B. vLLM oder llama.cpp). Sie bestimmt Geschwindigkeit, unterstuetzte Modellformate und welche Parameter ueberhaupt verfuegbar sind.vllm
Modellalias?Der Name, unter dem das Modell ueber die API angesprochen wird. Genau dieser Wert muss im Request-Feld 'model' stehen.Qwen3.8-Flash-Next-Uncensored
Kontextlaenge?Maximale Anzahl Tokens (Eingabe + erzeugte Ausgabe zusammen), die das Modell pro Anfrage verarbeiten kann.131072
Alias?Anzeigename des Modells nach aussen (served model name), unabhaengig vom Dateinamen.Qwen3.8-Flash-Next-Uncensored
Trust-Remote-Code?Erlaubt vLLM, mit dem Modell mitgelieferten Python-Code auszufuehren. Noetig fuer Architekturen, die vLLM nicht von Haus aus kennt. Nur bei vertrauenswuerdigen Modellen aktivieren.aktiv
Executor-Backend?Verfahren fuer Multi-GPU/Multi-Node: mp = Prozesse auf einer Maschine, ray = ueber Ray (auch ueber mehrere Knoten).mp
GPU-Speicher?Anteil des GPU-Speichers (0 bis 1), den vLLM belegen darf. 0.92 = 92 %. Hoeher = mehr Platz fuer den KV-Cache (mehr/laengere parallele Anfragen), aber groesseres Risiko fuer 'Out of Memory'.0.90
Kontext?Groesse des Kontextfensters in Token. 0 = der beim Training verwendete Kontext des Modells.131072
Max-Sequenzen?Maximale Anzahl gleichzeitig bearbeiteter Anfragen (Batch-Groesse). Hoeher = mehr Durchsatz, braucht aber mehr KV-Cache-Speicher.10
Auto-Tool-Choice?Erlaubt dem Modell, Tools (Function-Calling) selbststaendig auszuwaehlen. MUSS aktiv sein, damit 'tools' im Request akzeptiert werden - sonst lehnt vLLM sie mit HTTP 400 ab.aktiv
Tool-Parser?Bestimmt, wie vLLM Tool-Aufrufe aus der Modellantwort ausliest (z.B. hermes, mistral, llama3_json). Muss zum Modell passen.qwen3_coder
Reasoning-Parser?Trennt den Denk-/Reasoning-Teil der Antwort vom eigentlichen Inhalt (z.B. bei Modellen mit <think>-Bloecken).qwen3
execution_typelocal

All benchmarks of this model To leaderboard

Anzeige
Model comparison

Qwen3.8-Flash-Next on various hardware

All published performance runs of this model – each bubble a variant: position = prefill (X) × generation (Y), bubble size = number of runs. Closer to the top right = faster. ★ Marked gold = this benchmark.

GPUby graphics card

2.0301.5231.0155080,005.77211.54417.316Prefill (tok/s)Generation (tok/s)NVIDIA RTX A6000 - 1.171,8 tok/s Generation, 963 tok/s Prefill, TTFT 32.556 ms (21 Laufe)NVIDIA RTX A6000NVIDIA GeForce RTX 3090 - 1.132,7 tok/s Generation, 968 tok/s Prefill, TTFT 51.682 ms (6 Laufe)NVIDIA GeForce RTX 30...NVIDIA RTX PRO 6000 Blackwell Workstation Edition - 584,1 tok/s Generation, 13.991 tok/s Prefill, TTFT 512 ms (5 Laufe)NVIDIA RTX PRO 6000 B...NVIDIA GB10 (DGX Spark) - 492,8 tok/s Generation, 1.262 tok/s Prefill, TTFT 33.373 ms (12 Laufe)NVIDIA GB10 (DGX Spar...NVIDIA Tesla V100 PCIe 32GB - 196,2 tok/s Generation, 775 tok/s Prefill, TTFT 30.834 ms (8 Laufe)NVIDIA Tesla V100 PCI...AMD Radeon RX 7900 XTX - 18,7 tok/s Generation, 180 tok/s Prefill, TTFT 55.956 ms (3 Laufe)AMD Radeon RX 7900 XTXNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition - 1.564,6 tok/s Generation, 6.065 tok/s Prefill, TTFT 7.635 ms (15 Laufe) | DIESER LAUF★ NVIDIA RTX PRO 6000 B...
★ NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 1.564,6 tok/s this runNVIDIA RTX A6000 1.171,8 tok/sNVIDIA GeForce RTX 3090 1.132,7 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 584,1 tok/sNVIDIA GB10 (DGX Spark) 492,8 tok/sNVIDIA Tesla V100 PCIe 32GB 196,2 tok/sAMD Radeon RX 7900 XTX 18,7 tok/s

CPUby processor

2.0301.5231.0155080,005.78311.56617.348Prefill (tok/s)Generation (tok/s)AMD Ryzen 9 7945HX with Radeon Graphics - 1.564,6 tok/s Generation, 2.312 tok/s Prefill, TTFT 17.248 ms (6 Laufe)AMD Ryzen 9 7945HX wi...AMD Ryzen Threadripper PRO 3945WX 12-Cores - 1.171,8 tok/s Generation, 2.569 tok/s Prefill, TTFT 15.123 ms (6 Laufe)AMD Ryzen Threadrippe...AMD Ryzen 5 5600X 6-Core Processor - 1.132,7 tok/s Generation, 5.169 tok/s Prefill, TTFT 34.723 ms (9 Laufe)AMD Ryzen 5 5600X 6-C...AMD Ryzen 9 9950X 16-Core Processor - 584,1 tok/s Generation, 13.991 tok/s Prefill, TTFT 512 ms (5 Laufe)AMD Ryzen 9 9950X 16-...NVIDIA Grace - 492,8 tok/s Generation, 1.262 tok/s Prefill, TTFT 33.373 ms (12 Laufe)NVIDIA GraceAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics - 196,2 tok/s Generation, 757 tok/s Prefill, TTFT 36.737 ms (5 Laufe)AMD Ryzen 3 PRO 3200G...AMD EPYC 7203P 8-Core Processor - 79,2 tok/s Generation, 320 tok/s Prefill, TTFT 39.530 ms (15 Laufe)AMD EPYC 7203P 8-Core...Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz - 63,2 tok/s Generation, 803 tok/s Prefill, TTFT 20.996 ms (3 Laufe)Intel(R) Xeon(R) CPU ...AMD Ryzen Threadripper PRO 3955WX 16-Cores - 18,7 tok/s Generation, 180 tok/s Prefill, TTFT 55.956 ms (3 Laufe)AMD Ryzen Threadrippe...AMD Ryzen Threadripper PRO 9965WX 24-Cores - 424,2 tok/s Generation, 0 tok/s Prefill, TTFT 1.438 ms (6 Laufe) | DIESER LAUF★ AMD Ryzen Threadrippe...
AMD Ryzen 9 7945HX with Radeon Graphics 1.564,6 tok/sAMD Ryzen Threadripper PRO 3945WX 12-Cores 1.171,8 tok/sAMD Ryzen 5 5600X 6-Core Processor 1.132,7 tok/sAMD Ryzen 9 9950X 16-Core Processor 584,1 tok/sNVIDIA Grace 492,8 tok/s★ AMD Ryzen Threadripper PRO 9965WX 24-Cores 424,2 tok/s this runAMD Ryzen 3 PRO 3200GE w/ Radeon Vega Graphics 196,2 tok/sAMD EPYC 7203P 8-Core Processor 79,2 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 63,2 tok/sAMD Ryzen Threadripper PRO 3955WX 16-Cores 18,7 tok/s

MBby mainboard

2.0271.5201.0135070,005.78311.56617.348Prefill (tok/s)Generation (tok/s)Shenzhen Meigao Electronic Equipment Co.,Ltd DRFXI (MotherBoard Series) - 1.564,6 tok/s Generation, 2.312 tok/s Prefill, TTFT 17.248 ms (6 Laufe)Shenzhen Meigao Elect...unbekannt - 1.171,8 tok/s Generation, 2.569 tok/s Prefill, TTFT 15.123 ms (6 Laufe)unbekanntASUSTeK COMPUTER INC. ROG STRIX B550-A GAMING - 1.132,7 tok/s Generation, 5.169 tok/s Prefill, TTFT 34.723 ms (9 Laufe)ASUSTeK COMPUTER INC....ASUSTeK COMPUTER INC. ProArt X870E-CREATOR WIFI - 584,1 tok/s Generation, 13.991 tok/s Prefill, TTFT 512 ms (5 Laufe)ASUSTeK COMPUTER INC....ASUSTeK COMPUTER INC. GX10 - 492,8 tok/s Generation, 1.262 tok/s Prefill, TTFT 33.373 ms (12 Laufe)ASUSTeK COMPUTER INC....ASUSTeK COMPUTER INC. ROG STRIX X570-F GAMING - 196,2 tok/s Generation, 757 tok/s Prefill, TTFT 36.737 ms (5 Laufe)ASUSTeK COMPUTER INC....ASRockRack ROMED8-2T/BCM - 79,2 tok/s Generation, 423 tok/s Prefill, TTFT 24.603 ms (9 Laufe)ASRockRack ROMED8-2T/...Dell Inc. PowerEdge R820 - 63,2 tok/s Generation, 803 tok/s Prefill, TTFT 20.996 ms (3 Laufe)Dell Inc. PowerEdge R...ASUSTeK COMPUTER INC. Pro WS WRX80E-SAGE SE WIFI - 35,4 tok/s Generation, 170 tok/s Prefill, TTFT 59.933 ms (9 Laufe)ASUSTeK COMPUTER INC....ASUSTeK COMPUTER INC. Pro WS WRX90E-SAGE SE - 424,2 tok/s Generation, 0 tok/s Prefill, TTFT 1.438 ms (6 Laufe) | DIESER LAUF★ ASUSTeK COMPUTER INC....
Shenzhen Meigao Electronic Equipment Co.,Ltd DRFXI (MotherBoard Series) 1.564,6 tok/sunbekannt 1.171,8 tok/sASUSTeK COMPUTER INC. ROG STRIX B550-A GAMING 1.132,7 tok/sASUSTeK COMPUTER INC. ProArt X870E-CREATOR WIFI 584,1 tok/sASUSTeK COMPUTER INC. GX10 492,8 tok/s★ ASUSTeK COMPUTER INC. Pro WS WRX90E-SAGE SE 424,2 tok/s this runASUSTeK COMPUTER INC. ROG STRIX X570-F GAMING 196,2 tok/sASRockRack ROMED8-2T/BCM 79,2 tok/sDell Inc. PowerEdge R820 63,2 tok/sASUSTeK COMPUTER INC. Pro WS WRX80E-SAGE SE WIFI 35,4 tok/s

ENGby engine

1.9171.4961.07465323104.2738.54612.819Prefill (tok/s)Generation (tok/s)strata - 1.564,6 tok/s Generation, 1.064 tok/s Prefill, TTFT 44.996 ms (20 Laufe)stratallama.cpp - 1.555,3 tok/s Generation, 627 tok/s Prefill, TTFT 35.323 ms (24 Laufe)llama.cppunbekannt - 1.171,8 tok/s Generation, 1.981 tok/s Prefill, TTFT 17.080 ms (9 Laufe)unbekanntvLLM - 584,1 tok/s Generation, 10.429 tok/s Prefill, TTFT 1.500 ms (17 Laufe) | DIESER LAUF★ vLLM
strata 1.564,6 tok/sllama.cpp 1.555,3 tok/sunbekannt 1.171,8 tok/s★ vLLM 584,1 tok/s this run

DRVby driver

1.9351.4821.0295751228151.6292.4433.257Prefill (tok/s)Generation (tok/s)unbekannt - 1.564,6 tok/s Generation, 2.810 tok/s Prefill, TTFT 26.753 ms (58 Laufe)unbekanntNVIDIA 590.48.01 / CUDA 13.1 - 492,8 tok/s Generation, 1.262 tok/s Prefill, TTFT 33.373 ms (12 Laufe)NVIDIA 590.48.01 / CU...
unbekannt 1.564,6 tok/sNVIDIA 590.48.01 / CUDA 13.1 492,8 tok/s
Contributed by

Mario Alka Administrator

@marioalka

Ich bin Unternehmer, Softwareentwickler und KI-Enthusiast. Seit vielen Jahren entwickle ich Unternehmenssoftware und beschäftige mich inzwischen fast täglich mit lokalen LLMs, KI-Agenten und leistungsfähiger KI-Hardware.

Mit LLM-Benchmark.de möchte ich eine Plattform schaffen, auf der Modelle, GPUs und Agenten objektiv und reproduzierbar miteinander verglichen werden.