Contributed byMario AlkaMistral AI Mistral-Small-3.1-24B-Instruct-2503
Performance benchmark · measured on 12.11.2026 11:10
Benchmark-ID
run-20261112-104057-6d7c5dTimebench 3 - Kombi (Prefill + Generation)Dense24BRuntime: OllamaQuantisierung: Q4_K_M
Generation8,73tok/s
Prefill245,16tok/s
Time to First Token10.365,80ms
Total duration414,57s
Concurrency1parallel
Ranking in the field
32of 65 systems
Performance benchmark · Primary metric: Generation-Speed (tok/s) · 1× concurrent
This run is better than 52 % of all comparable systems.
Generation
8,7 tok/s
-70 % vs Ø 28,8
Prefill
245,2 tok/s
-74 % vs Ø 928,1
Time to First Token
10.366 ms
-80 % vs Ø 51.273
Distribution in the field1 – 256 tok/s
Wie schlägt sich dieser Benchmark mit anderen Modellen?
Qwen2.5-0.5B-InstructKeine GPU (CPU-only) · run-20261112-085343-7741e3
256,3 tok/s
Qwen3-0.6BKeine GPU (CPU-only) · run-20261112-085343-db3850
164,8 tok/s
gemma-4-E2B-itKeine GPU (CPU-only) · run-20261112-092948-fd3c9e
119,9 tok/s
Qwen2.5-1.5B-InstructKeine GPU (CPU-only) · run-20261112-092948-3b4600
113,3 tok/s
Llama-3.2-1B-InstructKeine GPU (CPU-only) · run-20261112-092948-3f1556
97,1 tok/s
Qwen3-1.7BKeine GPU (CPU-only) · run-20261112-092948-c7c09b
85,5 tok/s
gemma-4-E4B-itKeine GPU (CPU-only) · run-20261112-092948-bf2fa6
85,2 tok/s
gemma-2-2b-itKeine GPU (CPU-only) · run-20261112-092948-1935ac
68,1 tok/s
gemma-4-E4B-itKeine GPU (CPU-only) · run-20261112-104057-d67a02
64,7 tok/s
Qwen2.5-3B-InstructKeine GPU (CPU-only) · run-20261112-092948-4df138
57,5 tok/s
SmolLM2-1.7B-InstructKeine GPU (CPU-only) · run-20261112-092948-4e28a7
55,0 tok/s
Llama-3.2-3B-InstructKeine GPU (CPU-only) · run-20261112-092948-4825d5
52,9 tok/s
NVIDIA-Nemotron-3-Nano-4BKeine GPU (CPU-only) · run-20261112-104057-a8415d
42,3 tok/s
Qwen3-4BKeine GPU (CPU-only) · run-20261112-092948-c8bd7f
41,8 tok/s
Mistral-Small-3.1-24B-Instruct-2503 this runKeine GPU (CPU-only) · run-20261112-104057-6d7c5d
8,7 tok/s
How does this benchmark compare on other GPUs?
Same model on different hardware · 1× concurrent · Generation (tok/s)
AMD Radeon PRO W7900 Dual Slotrun-20260818-003841-e4b3b8
49,1 tok/s
AMD Radeon RX 7900 XTXrun-20260922-123036-d323a5
46,7 tok/s
2× AMD Radeon RX 7900 XTXrun-20261009-081229-2c3d29
45,8 tok/s
2× NVIDIA RTX A6000run-20260830-011927-6159d8
44,4 tok/s
2× NVIDIA RTX A6000run-20260827-185904-407b57
44,4 tok/s
2× AMD Radeon PRO W7900 Dual Slotrun-20260818-115056-d40760
39,0 tok/s
NVIDIA RTX A6000run-20260826-065247-76b2d6
38,4 tok/s
AMD Radeon AI PRO R9700run-20260922-063731-a56df0
33,5 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165907-01396d
33,1 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165909-428031
33,1 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165905-e839b4
33,0 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165906-4b8178
33,0 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165907-63e728
31,3 tok/s
3× AMD Radeon AI PRO R9700run-20260722-165909-c33041
31,3 tok/s
Hardware
GPU: Keine GPU (CPU-only)
CPU: Apple M6 (unified CPU/GPU SoC)
RAM: 24 GB
Mainboard: Apple Mac mini
Setup
Runtime: Ollama
Quantization: Q4_K_M
Model: Mistral-Small-3.1-24B-Instruct-2503
Configuration
# LLM-Benchmark Konfiguration
# Modell : Mistral-Small-3.1-24B-Instruct-2503
# Engine : Ollama
# Run-ID : run-20261112-104057-6d7c5d
# GPU : Keine GPU (CPU-only)
# CPU : Apple M6 (unified CPU/GPU SoC)
# RAM : 24 GB
bench@llm-benchmark:~$ ollama serve \
--model mistral-small3.1:24b
| Engine?Die Inferenz-Software, die das Modell ausliefert (z.B. vLLM oder llama.cpp). Sie bestimmt Geschwindigkeit, unterstuetzte Modellformate und welche Parameter ueberhaupt verfuegbar sind. | ollama |
| Kontextlaenge?Maximale Anzahl Tokens (Eingabe + erzeugte Ausgabe zusammen), die das Modell pro Anfrage verarbeiten kann. | 8192 |
| execution_type | local |
Anzeige
Model comparison
Mistral-Small-3.1-24B-Instruct-2503 on various hardware
All published performance runs of this model – each bubble a variant: position = prefill (X) × generation (Y), bubble size = number of runs. Closer to the top right = faster. ★ Marked gold = this benchmark.
GPUby graphics card
NVIDIA RTX A6000 214,9 tok/sAMD Radeon RX 7900 XTX 170,1 tok/sAMD Radeon PRO W7900 Dual Slot 150,4 tok/sAMD Radeon PRO W7800 48GB 141,9 tok/sAMD Radeon AI PRO R9700 133,1 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 88,3 tok/sAMD Radeon 8060S Graphics 31,3 tok/sNVIDIA Tesla P100 PCIe 16GB 23,8 tok/s★ CPU-only 8,7 tok/s this run
CPUby processor
AMD Ryzen Threadripper PRO 7955WX 16-Cores 214,9 tok/sIntel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz 170,1 tok/sAMD Ryzen Threadripper PRO 3955WX 16-Cores 151,5 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 150,4 tok/sAMD Ryzen 3 3100 4-Core Processor 133,1 tok/sAMD Ryzen 9 9950X 16-Core Processor 88,3 tok/sAMD RYZEN AI MAX+ 395 w/ Radeon 8060S 31,3 tok/sAMD Ryzen 9 7945HX with Radeon Graphics 23,8 tok/s★ Apple M6 (unified CPU/GPU SoC) 8,7 tok/s this run
MBby mainboard
ASUSTeK COMPUTER INC. Pro WS WRX90E-SAGE SE 214,9 tok/sDell Inc. PowerEdge R820 170,1 tok/sASUSTeK COMPUTER INC. Pro WS WRX80E-SAGE SE WIFI 151,5 tok/sASUSTeK COMPUTER INC. ROG STRIX X570-F GAMING 133,1 tok/sASUSTeK COMPUTER INC. ProArt X870E-CREATOR WIFI 88,3 tok/sMeigao Innovation Technology (Shen Zhen) Co., Ltd SHWSA (MS-S1 MAX) 31,3 tok/sShenzhen Meigao Electronic Equipment Co.,Ltd F1FXM (DeskMini Series) 23,8 tok/s★ Apple Mac mini 8,7 tok/s this run
DRVby driver
unbekannt 214,9 tok/sAMD 7.0.0-27-generic 33,1 tok/s
💰 Economics
Economics of this run
Operating cost, TCO and comparison with the next-best runs of the same model at identical concurrency (1× concurrent). Methodology →
⚙️ ConfigurationAll metrics and charts below follow these settings – based on a 24-month runtime.Save to URLReset
⚡ Electricity price
EUR/kWh
⚙️ System utilization
100 %
🔌 Idle
0 W
⚡ TDP
10 W
☁️ External LLM (API)
Electricity0.30 EUR/kWh
Avg power (incl. idle)0 W missing
Avg cost / hour–
Electricity / 1M tokens–
Token / kWh–
Acquisition (system)– partial price
Electricity (2 years)–
TCO (2 years)–
Output tokens (2 years)550.62M
☁️ External LLM (API) – comparison
External LLM cost (2 years)–
Savings vs. external (2 years)–
No power draw measured – values estimated from GPU TDP + CPU (idle + 15 %) + board.
All values above and the charts below take the configured system utilization into account: at X% the system generates only X% of the time, the rest it idles (0 W). Cost per hour drops (more idle), cost per token rises.
Cost over 2 years – electricity only
Cost over 2 years – incl. acquisition (TCO)
Speed vs. tokens per euro
Euro per 1M tokens
Comparison vs. API – economics per benchmark
| Mistral-Small-3.1-24B-Instruct-2503Keine GPU (CPU-only) | Mistral-Small-3.1-24B-Instruct-2503AMD Radeon PRO W7900 Dual Slot | Mistral-Small-3.1-24B-Instruct-2503AMD Radeon RX 7900 XTX | Mistral-Small-3.1-24B-Instruct-25032x AMD Radeon RX 7900 XTX | |
|---|---|---|---|---|
| Electricity cost (24 mo.) | – | – | – | – |
| Acquisition cost | – | – | – | – |
| Total cost (TCO) | – | – | – | – |
| Generated tokens (24 mo.) | – | – | – | – |
| Token price via API | – | – | – | – |
| Break-even point (days) | – | – | – | – |
| Result (savings / extra cost) | – | – | – | – |
Keine GPU (CPU-only) · this run 9 tok/sAMD Radeon PRO W7900 Dual Slot 49 tok/sAMD Radeon RX 7900 XTX 47 tok/s2× AMD Radeon RX 7900 XTX 46 tok/s
Comparison with up to 3 next-best runs of this model at the same concurrency (at least one on different hardware). Power = GPU TDP + CPU (idle + 15 %) + board (estimated), acquisition = full system (GPU + CPU + board + RAM + PSU), prices = stored market prices.
Contributed by
Mario Alka Administrator
@marioalka
Ich bin Unternehmer, Softwareentwickler und KI-Enthusiast. Seit vielen Jahren entwickle ich Unternehmenssoftware und beschäftige mich inzwischen fast täglich mit lokalen LLMs, KI-Agenten und leistungsfähiger KI-Hardware.
Mit LLM-Benchmark.de möchte ich eine Plattform schaffen, auf der Modelle, GPUs und Agenten objektiv und reproduzierbar miteinander verglichen werden.