Created bymario-alka.dePowered bynoob2claw.detricoma.de
DEEN
Anmelden Registrieren
Beta
Ol
Modellserver / Inferenz-Engine

Ollama

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

29Benchmarks
26Modelle
Beste Generation
256,3 tok/s
Qwen2.5-0.5B-Instruct
∅ Generation
57,0 tok/s
Schnitt aus 29 Läufen
Bester Prefill
10.267 tok/s
Prompt-Verarbeitung
Bester TTFT
249 ms
Time to First Token

Top-Modelle auf Ollama

Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.

Qwen2.5-0.5B-Instruct0.5B
256,3 tok/s
Qwen3-0.6B0.6B
164,8 tok/s
gemma-4-E2B-it5B
119,9 tok/s
Qwen2.5-1.5B-Instruct1.5B
113,3 tok/s
Llama-3.2-1B-Instruct1B
97,1 tok/s
Qwen3-1.7B1.7B
85,5 tok/s
gemma-4-E4B-it8B
85,2 tok/s
gemma-2-2b-it2B
68,1 tok/s
Qwen2.5-3B-Instruct3B
57,5 tok/s
SmolLM2-1.7B-Instruct1.7B
55,0 tok/s
Performanceprofil

Durchsatz mit Ollama

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

2822692562432311.8551.9342.0132.092Prefill (tok/s)Generation (tok/s)Apple M6 (unified CPU/GPU SoC) - 256,3 tok/s Generation, 1.973 tok/s Prefill, TTFT 3.398 ms (29 Laufe)Apple M6 (unified CPU...
Apple M6 (unified CPU/GPU SoC) 256,3 tok/s

GPUnach Grafikkarte

2822692562432311.8551.9342.0132.092Prefill (tok/s)Generation (tok/s)CPU-only - 256,3 tok/s Generation, 1.973 tok/s Prefill, TTFT 3.398 ms (29 Laufe)CPU-only
CPU-only 256,3 tok/s

MODnach Modell

33124916682,90,004.2308.46012.690Prefill (tok/s)Generation (tok/s)Qwen2.5-0.5B-Instruct - 256,3 tok/s Generation, 10.267 tok/s Prefill, TTFT 249 ms (1 Lauf)Qwen2.5-0.5B-InstructQwen3-0.6B - 164,8 tok/s Generation, 7.014 tok/s Prefill, TTFT 390 ms (1 Lauf)Qwen3-0.6Bgemma-4-E2B-it - 119,9 tok/s Generation, 4.370 tok/s Prefill, TTFT 507 ms (1 Lauf)gemma-4-E2B-itQwen2.5-1.5B-Instruct - 113,3 tok/s Generation, 3.554 tok/s Prefill, TTFT 706 ms (1 Lauf)Qwen2.5-1.5B-InstructLlama-3.2-1B-Instruct - 97,1 tok/s Generation, 5.083 tok/s Prefill, TTFT 500 ms (1 Lauf)Llama-3.2-1B-InstructQwen3-1.7B - 85,5 tok/s Generation, 3.232 tok/s Prefill, TTFT 815 ms (1 Lauf)Qwen3-1.7Bgemma-4-E4B-it - 85,2 tok/s Generation, 1.963 tok/s Prefill, TTFT 1.056 ms (2 Laufe)gemma-4-E4B-itgemma-2-2b-it - 68,1 tok/s Generation, 2.491 tok/s Prefill, TTFT 768 ms (1 Lauf)gemma-2-2b-itQwen2.5-3B-Instruct - 57,5 tok/s Generation, 1.718 tok/s Prefill, TTFT 1.461 ms (1 Lauf)Qwen2.5-3B-InstructSmolLM2-1.7B-Instruct - 55,0 tok/s Generation, 2.966 tok/s Prefill, TTFT 1.194 ms (1 Lauf)SmolLM2-1.7B-InstructLlama-3.2-3B-Instruct - 52,9 tok/s Generation, 1.698 tok/s Prefill, TTFT 1.508 ms (1 Lauf)Llama-3.2-3B-InstructNVIDIA-Nemotron-3-Nano-4B - 42,3 tok/s Generation, 1.132 tok/s Prefill, TTFT 2.150 ms (2 Laufe)NVIDIA-Nemotron-3-Nan...Qwen3-4B - 41,8 tok/s Generation, 1.253 tok/s Prefill, TTFT 2.021 ms (1 Lauf)Qwen3-4Bgemma-3n-E4B-it - 39,4 tok/s Generation, 838 tok/s Prefill, TTFT 2.388 ms (1 Lauf)gemma-3n-E4B-itgpt-oss-20b - 34,8 tok/s Generation, 1.030 tok/s Prefill, TTFT 2.057 ms (1 Lauf)gpt-oss-20bgemma-4-12B-it - 31,7 tok/s Generation, 490 tok/s Prefill, TTFT 4.423 ms (2 Laufe)gemma-4-12B-itQwen2.5-7B-Instruct - 28,3 tok/s Generation, 637 tok/s Prefill, TTFT 3.902 ms (1 Lauf)Qwen2.5-7B-InstructQwen2.5-Coder-7B-Instruct - 28,2 tok/s Generation, 722 tok/s Prefill, TTFT 3.452 ms (1 Lauf)Qwen2.5-Coder-7B-Inst...Meta-Llama-3.1-8B-Instruct - 26,1 tok/s Generation, 582 tok/s Prefill, TTFT 4.250 ms (1 Lauf)Meta-Llama-3.1-8B-Ins...Qwen3-8B - 25,0 tok/s Generation, 745 tok/s Prefill, TTFT 3.467 ms (1 Lauf)Qwen3-8Bgemma-2-9b-it - 20,3 tok/s Generation, 550 tok/s Prefill, TTFT 3.503 ms (1 Lauf)gemma-2-9b-itQwen3-14B - 14,0 tok/s Generation, 321 tok/s Prefill, TTFT 8.097 ms (1 Lauf)Qwen3-14BCodestral-22B-v0.1 - 8,9 tok/s Generation, 241 tok/s Prefill, TTFT 11.388 ms (1 Lauf)Codestral-22B-v0.1Devstral-Small-2-24B-Instruct-2512 - 8,9 tok/s Generation, 276 tok/s Prefill, TTFT 9.964 ms (1 Lauf)Devstral-Small-2-24B-...Mistral-Small-3.1-24B-Instruct-2503 - 8,7 tok/s Generation, 245 tok/s Prefill, TTFT 10.366 ms (1 Lauf)Mistral-Small-3.1-24B...Mistral-Small-24B-Instruct-2501 - 8,5 tok/s Generation, 226 tok/s Prefill, TTFT 10.329 ms (1 Lauf)Mistral-Small-24B-Ins...
Qwen2.5-0.5B-Instruct 256,3 tok/sQwen3-0.6B 164,8 tok/sgemma-4-E2B-it 119,9 tok/sQwen2.5-1.5B-Instruct 113,3 tok/sLlama-3.2-1B-Instruct 97,1 tok/sQwen3-1.7B 85,5 tok/sgemma-4-E4B-it 85,2 tok/sgemma-2-2b-it 68,1 tok/sQwen2.5-3B-Instruct 57,5 tok/sSmolLM2-1.7B-Instruct 55,0 tok/sLlama-3.2-3B-Instruct 52,9 tok/sNVIDIA-Nemotron-3-Nano-4B 42,3 tok/sQwen3-4B 41,8 tok/sgemma-3n-E4B-it 39,4 tok/sgpt-oss-20b 34,8 tok/sgemma-4-12B-it 31,7 tok/sQwen2.5-7B-Instruct 28,3 tok/sQwen2.5-Coder-7B-Instruct 28,2 tok/sMeta-Llama-3.1-8B-Instruct 26,1 tok/sQwen3-8B 25,0 tok/sgemma-2-9b-it 20,3 tok/sQwen3-14B 14,0 tok/sCodestral-22B-v0.1 8,9 tok/sDevstral-Small-2-24B-Instruct-2512 8,9 tok/sMistral-Small-3.1-24B-Instruct-2503 8,7 tok/sMistral-Small-24B-Instruct-2501 8,5 tok/s
Alle Messungen

29 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit Ollama.

← Alle Engines
Messwert:
Größe
#Modell / HerstellerMesswerteParallelGPU / CPU / RAMRuntime
1Qwen2.5-0.5B-Instruct0.5BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
256,25 tok/s TG
Prefill 10.267 · TTFT 249 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
2Qwen3-0.6B0.6BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
164,83 tok/s TG
Prefill 7.014 · TTFT 390 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
3gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
119,93 tok/s TG
Prefill 4.370 · TTFT 507 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
4Qwen2.5-1.5B-Instruct1.5BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
113,29 tok/s TG
Prefill 3.554 · TTFT 706 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
5Llama-3.2-1B-Instruct1BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
97,11 tok/s TG
Prefill 5.083 · TTFT 500 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ8_0Details →
6Qwen3-1.7B1.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
85,48 tok/s TG
Prefill 3.232 · TTFT 815 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
7gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
85,23 tok/s TG
Prefill 2.052 · TTFT 990 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
8gemma-2-2b-it2BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
68,10 tok/s TG
Prefill 2.491 · TTFT 768 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →
9gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
64,65 tok/s TG
Prefill 1.873 · TTFT 1.123 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
10Qwen2.5-3B-Instruct3BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
57,50 tok/s TG
Prefill 1.718 · TTFT 1.461 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
11SmolLM2-1.7B-Instruct1.7BHuggingFaceTB PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
54,97 tok/s TG
Prefill 2.966 · TTFT 1.194 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ8_0Details →
12Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
52,92 tok/s TG
Prefill 1.698 · TTFT 1.508 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
13NVIDIA-Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
42,29 tok/s TG
Prefill 1.495 · TTFT 1.438 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
14Qwen3-4B4BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
41,82 tok/s TG
Prefill 1.253 · TTFT 2.021 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
15gemma-3n-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
39,36 tok/s TG
Prefill 838 · TTFT 2.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
16NVIDIA-Nemotron-3-Nano-4B4BNVIDIA PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
35,84 tok/s TG
Prefill 769 · TTFT 2.861 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
17gpt-oss-20b20BOpenAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
34,84 tok/s TG
Prefill 1.030 · TTFT 2.057 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliMXFP4Details →
18gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
31,69 tok/s TG
Prefill 616 · TTFT 3.219 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
19gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,63 tok/s TG
Prefill 364 · TTFT 5.626 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
20Qwen2.5-7B-Instruct7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,30 tok/s TG
Prefill 637 · TTFT 3.902 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
21Qwen2.5-Coder-7B-Instruct7.6BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,21 tok/s TG
Prefill 722 · TTFT 3.452 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
22Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
26,14 tok/s TG
Prefill 582 · TTFT 4.250 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
23Qwen3-8B8.2BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
24,96 tok/s TG
Prefill 745 · TTFT 3.467 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
24gemma-2-9b-it9BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
20,31 tok/s TG
Prefill 550 · TTFT 3.503 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →
25Qwen3-14B14BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
14,03 tok/s TG
Prefill 321 · TTFT 8.097 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
26Codestral-22B-v0.122BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,89 tok/s TG
Prefill 241 · TTFT 11.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →
27Devstral-Small-2-24B-Instruct-251224BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,85 tok/s TG
Prefill 276 · TTFT 9.964 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
28Mistral-Small-3.1-24B-Instruct-250324BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,73 tok/s TG
Prefill 245 · TTFT 10.366 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
29Mistral-Small-24B-Instruct-250123.6BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,47 tok/s TG
Prefill 226 · TTFT 10.329 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →