Created bymario-alka.dePowered bynoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
lm
Modellserver / Inferenz-Engine

lmstudio-mlx

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

30Benchmarks
30Models
Best generation
298,1 tok/s
Qwen2.5-0.5B-Instruct
∅ Generation
56,8 tok/s
Schnitt aus 30 Läufen
Best prefill
14.858 tok/s
Prompt processing
Best TTFT
175 ms
Time to First Token

Top-Modelle auf lmstudio-mlx

Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.

Qwen2.5-0.5B-Instruct0.5B
298,1 tok/s
Qwen3-0.6B0.6B
189,5 tok/s
Llama-3.2-1B-Instruct1B
133,2 tok/s
Qwen2.5-1.5B-Instruct1.5B
126,7 tok/s
Qwen3-1.7B1.7B
100,5 tok/s
gemma-2-2b-it2B
75,1 tok/s
Qwen2.5-3B-Instruct3B
66,0 tok/s
Llama-3.2-3B-Instruct3B
62,9 tok/s
NVIDIA-Nemotron-3-Nano-4B4B
56,2 tok/s
gpt-oss-20b20B
53,1 tok/s
Performance profile

Durchsatz mit lmstudio-mlx

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

3283132982832682.3582.4592.5592.660Prefill (tok/s)Generation (tok/s)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) - 298,1 tok/s Generation, 2.509 tok/s Prefill, TTFT 3.036 ms (30 Laufe)Apple M6 (unified CPU...
Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) 298,1 tok/s

GPUnach Grafikkarte

3283132982832682.3582.4592.5592.660Prefill (tok/s)Generation (tok/s)CPU-only - 298,1 tok/s Generation, 2.509 tok/s Prefill, TTFT 3.036 ms (30 Laufe)CPU-only
CPU-only 298,1 tok/s

MODnach Modell

38628919396,40,006.12612.25118.377Prefill (tok/s)Generation (tok/s)Qwen2.5-0.5B-Instruct - 298,1 tok/s Generation, 14.858 tok/s Prefill, TTFT 175 ms (1 Lauf)Qwen2.5-0.5B-InstructQwen3-0.6B - 189,5 tok/s Generation, 11.438 tok/s Prefill, TTFT 251 ms (1 Lauf)Qwen3-0.6BLlama-3.2-1B-Instruct - 133,2 tok/s Generation, 7.177 tok/s Prefill, TTFT 377 ms (1 Lauf)Llama-3.2-1B-InstructQwen2.5-1.5B-Instruct - 126,7 tok/s Generation, 5.137 tok/s Prefill, TTFT 507 ms (1 Lauf)Qwen2.5-1.5B-InstructQwen3-1.7B - 100,5 tok/s Generation, 4.497 tok/s Prefill, TTFT 604 ms (1 Lauf)Qwen3-1.7Bgemma-2-2b-it - 75,1 tok/s Generation, 2.212 tok/s Prefill, TTFT 876 ms (1 Lauf)gemma-2-2b-itQwen2.5-3B-Instruct - 66,0 tok/s Generation, 2.669 tok/s Prefill, TTFT 988 ms (1 Lauf)Qwen2.5-3B-InstructLlama-3.2-3B-Instruct - 62,9 tok/s Generation, 2.562 tok/s Prefill, TTFT 1.026 ms (1 Lauf)Llama-3.2-3B-InstructNVIDIA-Nemotron-3-Nano-4B - 56,2 tok/s Generation, 1.002 tok/s Prefill, TTFT 2.226 ms (1 Lauf)NVIDIA-Nemotron-3-Nan...gpt-oss-20b - 53,1 tok/s Generation, 1.769 tok/s Prefill, TTFT 1.368 ms (1 Lauf)gpt-oss-20bLlama-3.1-Nemotron-Nano-4B-v1.1 - 49,2 tok/s Generation, 1.978 tok/s Prefill, TTFT 1.328 ms (1 Lauf)Llama-3.1-Nemotron-Na...Qwen3-4B - 48,3 tok/s Generation, 1.927 tok/s Prefill, TTFT 1.388 ms (1 Lauf)Qwen3-4BGemma-4-26B-A4B-it - 47,4 tok/s Generation, 1.491 tok/s Prefill, TTFT 1.402 ms (1 Lauf)Gemma-4-26B-A4B-itgemma-3n-E4B-it - 46,8 tok/s Generation, 1.560 tok/s Prefill, TTFT 1.426 ms (1 Lauf)gemma-3n-E4B-itPhi-3.5-mini-instruct - 41,6 tok/s Generation, 1.815 tok/s Prefill, TTFT 1.555 ms (1 Lauf)Phi-3.5-mini-instructSmolLM2-1.7B-Instruct - 34,5 tok/s Generation, 4.031 tok/s Prefill, TTFT 886 ms (1 Lauf)SmolLM2-1.7B-InstructQwen2.5-7B-Instruct - 31,5 tok/s Generation, 1.123 tok/s Prefill, TTFT 2.322 ms (1 Lauf)Qwen2.5-7B-InstructQwen2.5-Coder-7B-Instruct - 31,4 tok/s Generation, 1.117 tok/s Prefill, TTFT 2.334 ms (1 Lauf)Qwen2.5-Coder-7B-Inst...Mamba-Codestral-7B-v0.1 - 29,3 tok/s Generation, 327 tok/s Prefill, TTFT 8.424 ms (1 Lauf)Mamba-Codestral-7B-v0...Meta-Llama-3.1-8B-Instruct - 29,2 tok/s Generation, 1.076 tok/s Prefill, TTFT 2.447 ms (1 Lauf)Meta-Llama-3.1-8B-Ins...Qwen3-8B - 28,4 tok/s Generation, 1.096 tok/s Prefill, TTFT 2.410 ms (1 Lauf)Qwen3-8Bgemma-2-9b-it - 22,0 tok/s Generation, 646 tok/s Prefill, TTFT 3.068 ms (1 Lauf)gemma-2-9b-itOrnith-1.5-9B - 21,9 tok/s Generation, 1.047 tok/s Prefill, TTFT 2.009 ms (1 Lauf)Ornith-1.5-9BMinistral-3-14B-Reasoning-2512 - 17,3 tok/s Generation, 621 tok/s Prefill, TTFT 3.879 ms (1 Lauf)Ministral-3-14B-Reaso...Qwen3-14B - 15,5 tok/s Generation, 525 tok/s Prefill, TTFT 4.974 ms (1 Lauf)Qwen3-14BMistral-Small-3.1-24B-Instruct-2503 - 9,9 tok/s Generation, 290 tok/s Prefill, TTFT 8.335 ms (1 Lauf)Mistral-Small-3.1-24B...Mistral-Small-24B-Instruct-2501 - 9,9 tok/s Generation, 266 tok/s Prefill, TTFT 8.275 ms (1 Lauf)Mistral-Small-24B-Ins...Devstral-Small-2507 - 9,7 tok/s Generation, 427 tok/s Prefill, TTFT 8.728 ms (1 Lauf)Devstral-Small-2507Codestral-22B-v0.1 - 9,7 tok/s Generation, 325 tok/s Prefill, TTFT 8.859 ms (1 Lauf)Codestral-22B-v0.1Devstral-Small-2-24B-Instruct-2512 - 9,2 tok/s Generation, 261 tok/s Prefill, TTFT 8.641 ms (1 Lauf)Devstral-Small-2-24B-...
Qwen2.5-0.5B-Instruct 298,1 tok/sQwen3-0.6B 189,5 tok/sLlama-3.2-1B-Instruct 133,2 tok/sQwen2.5-1.5B-Instruct 126,7 tok/sQwen3-1.7B 100,5 tok/sgemma-2-2b-it 75,1 tok/sQwen2.5-3B-Instruct 66,0 tok/sLlama-3.2-3B-Instruct 62,9 tok/sNVIDIA-Nemotron-3-Nano-4B 56,2 tok/sgpt-oss-20b 53,1 tok/sLlama-3.1-Nemotron-Nano-4B-v1.1 49,2 tok/sQwen3-4B 48,3 tok/sGemma-4-26B-A4B-it 47,4 tok/sgemma-3n-E4B-it 46,8 tok/sPhi-3.5-mini-instruct 41,6 tok/sSmolLM2-1.7B-Instruct 34,5 tok/sQwen2.5-7B-Instruct 31,5 tok/sQwen2.5-Coder-7B-Instruct 31,4 tok/sMamba-Codestral-7B-v0.1 29,3 tok/sMeta-Llama-3.1-8B-Instruct 29,2 tok/sQwen3-8B 28,4 tok/sgemma-2-9b-it 22,0 tok/sOrnith-1.5-9B 21,9 tok/sMinistral-3-14B-Reasoning-2512 17,3 tok/sQwen3-14B 15,5 tok/sMistral-Small-3.1-24B-Instruct-2503 9,9 tok/sMistral-Small-24B-Instruct-2501 9,9 tok/sDevstral-Small-2507 9,7 tok/sCodestral-22B-v0.1 9,7 tok/sDevstral-Small-2-24B-Instruct-2512 9,2 tok/s
All measurements

30 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit lmstudio-mlx.

← Alle Engines
Metric:
Size
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1Qwen2.5-0.5B-Instruct0.5BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
298,13 tok/s TG
Prefill 14.858 · TTFT 175 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
2Qwen3-0.6B0.6BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
189,50 tok/s TG
Prefill 11.438 · TTFT 251 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
3Llama-3.2-1B-Instruct1BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
133,16 tok/s TG
Prefill 7.177 · TTFT 377 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
4Qwen2.5-1.5B-Instruct1.5BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
126,71 tok/s TG
Prefill 5.137 · TTFT 507 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
5Qwen3-1.7B1.7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
100,53 tok/s TG
Prefill 4.497 · TTFT 604 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
6gemma-2-2b-it2BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
75,08 tok/s TG
Prefill 2.212 · TTFT 876 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
7Qwen2.5-3B-Instruct3BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
65,98 tok/s TG
Prefill 2.669 · TTFT 988 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
8Llama-3.2-3B-Instruct3BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
62,86 tok/s TG
Prefill 2.562 · TTFT 1.026 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
9NVIDIA-Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
56,16 tok/s TG
Prefill 1.002 · TTFT 2.226 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
10gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
53,05 tok/s TG
Prefill 1.769 · TTFT 1.368 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX MXFP4Details →
11Llama-3.1-Nemotron-Nano-4B-v1.14.5BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
49,15 tok/s TG
Prefill 1.978 · TTFT 1.328 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
12Qwen3-4B4BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
48,34 tok/s TG
Prefill 1.927 · TTFT 1.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
13Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
47,35 tok/s TG
Prefill 1.491 · TTFT 1.402 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
14gemma-3n-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
46,82 tok/s TG
Prefill 1.560 · TTFT 1.426 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
15Phi-3.5-mini-instruct3.8Bmicrosoft Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
41,56 tok/s TG
Prefill 1.815 · TTFT 1.555 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
16SmolLM2-1.7B-Instruct1.7BHuggingFaceTB Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
34,49 tok/s TG
Prefill 4.031 · TTFT 886 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX bf16Details →
17Qwen2.5-7B-Instruct7BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,46 tok/s TG
Prefill 1.123 · TTFT 2.322 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
18Qwen2.5-Coder-7B-Instruct7.6BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
31,36 tok/s TG
Prefill 1.117 · TTFT 2.334 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
19Mamba-Codestral-7B-v0.17BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,25 tok/s TG
Prefill 327 · TTFT 8.424 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
20Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
29,20 tok/s TG
Prefill 1.076 · TTFT 2.447 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
21Qwen3-8B8.2BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
28,42 tok/s TG
Prefill 1.096 · TTFT 2.410 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
22gemma-2-9b-it9BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,98 tok/s TG
Prefill 646 · TTFT 3.068 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
23Ornith-1.5-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
21,94 tok/s TG
Prefill 1.047 · TTFT 2.009 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bit (OptiQ)Details →
24Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
17,33 tok/s TG
Prefill 621 · TTFT 3.879 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
25Qwen3-14B14BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
15,53 tok/s TG
Prefill 525 · TTFT 4.974 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
26Mistral-Small-3.1-24B-Instruct-250324BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,93 tok/s TG
Prefill 290 · TTFT 8.335 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
27Mistral-Small-24B-Instruct-250123.6BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,90 tok/s TG
Prefill 266 · TTFT 8.275 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
28Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,69 tok/s TG
Prefill 427 · TTFT 8.728 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
29Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,65 tok/s TG
Prefill 325 · TTFT 8.859 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →
30Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
9,20 tok/s TG
Prefill 261 · TTFT 8.641 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC, ARM64, AdvSIMD) · 24 GB RAMlmstudio-mlxlmstudioMLX 4bitDetails →