Created bymario-alka.dePowered bynoob2claw.detricoma.de
DEEN
Anmelden Registrieren
Beta
Ol
Modellserver / Inferenz-Engine

Ollama

Alle Performance-Benchmarks, die mit diesem Modellserver gemessen wurden – Durchsatz, Prefill und Latenz.

22Benchmarks
22Modelle
Beste Generation
256,3 tok/s
Qwen2.5-0.5B-Instruct
∅ Generation
66,1 tok/s
Schnitt aus 22 Läufen
Bester Prefill
10.267 tok/s
Prompt-Verarbeitung
Bester TTFT
249 ms
Time to First Token

Top-Modelle auf Ollama

Bester Durchsatz je Modell – umschaltbar nach Generation, Prefill oder Kombiwert.

Qwen2.5-0.5B-Instruct0.5B
256,3 tok/s
Qwen3-0.6B0.6B
164,8 tok/s
gemma-4-E2B-it5B
119,9 tok/s
Qwen2.5-1.5B-Instruct1.5B
113,3 tok/s
Llama-3.2-1B-Instruct1B
97,1 tok/s
Qwen3-1.7B1.7B
85,5 tok/s
gemma-4-E4B-it8B
85,2 tok/s
gemma-2-2b-it2B
68,1 tok/s
Qwen2.5-3B-Instruct3B
57,5 tok/s
SmolLM2-1.7B-Instruct1.7B
55,0 tok/s
Performanceprofil

Durchsatz mit Ollama

Jede Blase steht für eine CPU, GPU bzw. ein Modell – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

CPUnach Prozessor

2822692562432312.2212.3152.4102.504Prefill (tok/s)Generation (tok/s)Apple M6 (unified CPU/GPU SoC) - 256,3 tok/s Generation, 2.363 tok/s Prefill, TTFT 2.583 ms (22 Laufe)Apple M6 (unified CPU...
Apple M6 (unified CPU/GPU SoC) 256,3 tok/s

GPUnach Grafikkarte

2822692562432312.2212.3152.4102.504Prefill (tok/s)Generation (tok/s)CPU-only - 256,3 tok/s Generation, 2.363 tok/s Prefill, TTFT 2.583 ms (22 Laufe)CPU-only
CPU-only 256,3 tok/s

MODnach Modell

33124916682,80,004.2298.45812.687Prefill (tok/s)Generation (tok/s)Qwen2.5-0.5B-Instruct - 256,3 tok/s Generation, 10.267 tok/s Prefill, TTFT 249 ms (1 Lauf)Qwen2.5-0.5B-InstructQwen3-0.6B - 164,8 tok/s Generation, 7.014 tok/s Prefill, TTFT 390 ms (1 Lauf)Qwen3-0.6Bgemma-4-E2B-it - 119,9 tok/s Generation, 4.370 tok/s Prefill, TTFT 507 ms (1 Lauf)gemma-4-E2B-itQwen2.5-1.5B-Instruct - 113,3 tok/s Generation, 3.554 tok/s Prefill, TTFT 706 ms (1 Lauf)Qwen2.5-1.5B-InstructLlama-3.2-1B-Instruct - 97,1 tok/s Generation, 5.083 tok/s Prefill, TTFT 500 ms (1 Lauf)Llama-3.2-1B-InstructQwen3-1.7B - 85,5 tok/s Generation, 3.232 tok/s Prefill, TTFT 815 ms (1 Lauf)Qwen3-1.7Bgemma-4-E4B-it - 85,2 tok/s Generation, 2.052 tok/s Prefill, TTFT 990 ms (1 Lauf)gemma-4-E4B-itgemma-2-2b-it - 68,1 tok/s Generation, 2.491 tok/s Prefill, TTFT 768 ms (1 Lauf)gemma-2-2b-itQwen2.5-3B-Instruct - 57,5 tok/s Generation, 1.718 tok/s Prefill, TTFT 1.461 ms (1 Lauf)Qwen2.5-3B-InstructSmolLM2-1.7B-Instruct - 55,0 tok/s Generation, 2.966 tok/s Prefill, TTFT 1.194 ms (1 Lauf)SmolLM2-1.7B-InstructLlama-3.2-3B-Instruct - 52,9 tok/s Generation, 1.698 tok/s Prefill, TTFT 1.508 ms (1 Lauf)Llama-3.2-3B-InstructQwen3-4B - 41,8 tok/s Generation, 1.253 tok/s Prefill, TTFT 2.021 ms (1 Lauf)Qwen3-4Bgemma-3n-E4B-it - 39,4 tok/s Generation, 838 tok/s Prefill, TTFT 2.388 ms (1 Lauf)gemma-3n-E4B-itgpt-oss-20b - 34,8 tok/s Generation, 1.030 tok/s Prefill, TTFT 2.057 ms (1 Lauf)gpt-oss-20bgemma-4-12B-it - 31,7 tok/s Generation, 616 tok/s Prefill, TTFT 3.219 ms (1 Lauf)gemma-4-12B-itQwen2.5-7B-Instruct - 28,3 tok/s Generation, 637 tok/s Prefill, TTFT 3.902 ms (1 Lauf)Qwen2.5-7B-InstructQwen2.5-Coder-7B-Instruct - 28,2 tok/s Generation, 722 tok/s Prefill, TTFT 3.452 ms (1 Lauf)Qwen2.5-Coder-7B-Inst...Meta-Llama-3.1-8B-Instruct - 26,1 tok/s Generation, 582 tok/s Prefill, TTFT 4.250 ms (1 Lauf)Meta-Llama-3.1-8B-Ins...Qwen3-8B - 25,0 tok/s Generation, 745 tok/s Prefill, TTFT 3.467 ms (1 Lauf)Qwen3-8Bgemma-2-9b-it - 20,3 tok/s Generation, 550 tok/s Prefill, TTFT 3.503 ms (1 Lauf)gemma-2-9b-itQwen3-14B - 14,0 tok/s Generation, 321 tok/s Prefill, TTFT 8.097 ms (1 Lauf)Qwen3-14BCodestral-22B-v0.1 - 8,9 tok/s Generation, 241 tok/s Prefill, TTFT 11.388 ms (1 Lauf)Codestral-22B-v0.1
Qwen2.5-0.5B-Instruct 256,3 tok/sQwen3-0.6B 164,8 tok/sgemma-4-E2B-it 119,9 tok/sQwen2.5-1.5B-Instruct 113,3 tok/sLlama-3.2-1B-Instruct 97,1 tok/sQwen3-1.7B 85,5 tok/sgemma-4-E4B-it 85,2 tok/sgemma-2-2b-it 68,1 tok/sQwen2.5-3B-Instruct 57,5 tok/sSmolLM2-1.7B-Instruct 55,0 tok/sLlama-3.2-3B-Instruct 52,9 tok/sQwen3-4B 41,8 tok/sgemma-3n-E4B-it 39,4 tok/sgpt-oss-20b 34,8 tok/sgemma-4-12B-it 31,7 tok/sQwen2.5-7B-Instruct 28,3 tok/sQwen2.5-Coder-7B-Instruct 28,2 tok/sMeta-Llama-3.1-8B-Instruct 26,1 tok/sQwen3-8B 25,0 tok/sgemma-2-9b-it 20,3 tok/sQwen3-14B 14,0 tok/sCodestral-22B-v0.1 8,9 tok/s
Alle Messungen

22 Benchmark-Läufe

Jeder veroeffentlichte Lauf mit Ollama.

← Alle Engines
Messwert:
Größe
#Modell / HerstellerMesswerteParallelGPU / CPU / RAMRuntime
1Qwen2.5-0.5B-Instruct0.5BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
256,25 tok/s TG
Prefill 10.267 · TTFT 249 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
2Qwen3-0.6B0.6BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
164,83 tok/s TG
Prefill 7.014 · TTFT 390 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
3gemma-4-E2B-it5BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
119,93 tok/s TG
Prefill 4.370 · TTFT 507 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
4Qwen2.5-1.5B-Instruct1.5BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
113,29 tok/s TG
Prefill 3.554 · TTFT 706 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
5Llama-3.2-1B-Instruct1BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
97,11 tok/s TG
Prefill 5.083 · TTFT 500 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ8_0Details →
6Qwen3-1.7B1.7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
85,48 tok/s TG
Prefill 3.232 · TTFT 815 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
7gemma-4-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
85,23 tok/s TG
Prefill 2.052 · TTFT 990 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
8gemma-2-2b-it2BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
68,10 tok/s TG
Prefill 2.491 · TTFT 768 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →
9Qwen2.5-3B-Instruct3BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
57,50 tok/s TG
Prefill 1.718 · TTFT 1.461 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
10SmolLM2-1.7B-Instruct1.7BHuggingFaceTB PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
54,97 tok/s TG
Prefill 2.966 · TTFT 1.194 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ8_0Details →
11Llama-3.2-3B-Instruct3BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
52,92 tok/s TG
Prefill 1.698 · TTFT 1.508 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
12Qwen3-4B4BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
41,82 tok/s TG
Prefill 1.253 · TTFT 2.021 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
13gemma-3n-E4B-it8BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
39,36 tok/s TG
Prefill 838 · TTFT 2.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
14gpt-oss-20b20BOpenAI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
34,84 tok/s TG
Prefill 1.030 · TTFT 2.057 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliMXFP4Details →
15gemma-4-12B-it12BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
31,69 tok/s TG
Prefill 616 · TTFT 3.219 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliNVFP4Details →
16Qwen2.5-7B-Instruct7BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,30 tok/s TG
Prefill 637 · TTFT 3.902 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
17Qwen2.5-Coder-7B-Instruct7.6BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
28,21 tok/s TG
Prefill 722 · TTFT 3.452 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
18Meta-Llama-3.1-8B-Instruct8BMeta PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
26,14 tok/s TG
Prefill 582 · TTFT 4.250 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
19Qwen3-8B8.2BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
24,96 tok/s TG
Prefill 745 · TTFT 3.467 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
20gemma-2-9b-it9BGoogle PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
20,31 tok/s TG
Prefill 550 · TTFT 3.503 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →
21Qwen3-14B14BQwen (Alibaba) PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
14,03 tok/s TG
Prefill 321 · TTFT 8.097 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_K_MDetails →
22Codestral-22B-v0.122BMistral AI PerformancebenchmarkTimebench 3 - Kombi (Prefill + Generation)
8,89 tok/s TG
Prefill 241 · TTFT 11.388 ms
1×Keine GPU (CPU-only)Apple M6 (unified CPU/GPU SoC) · 24 GB RAMOllamaopenclaw_cliQ4_0Details →