⚡
Performance benchmark
How fast is the model?
Raw throughput on real hardware – how many tokens a model generates per second, how quickly it processes the prompt and how short the time to first token is. Higher is better.
⚡ Generation (tok/s)🔄 Prefill⏱️ TTFT👥 Concurrency
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | Qwen2.5-72B-Instruct-Q4_K_M.ggufQwen (Alibaba) Performance benchmarkPerformancetest Small 1.0 | 2,28 tok/s TG Prefill 81 · TTFT 569 ms | 1× | Keine GPU (CPU-only)4x Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz · 504 GB RAM | llama.cppgodclawQ4_K_M | Details → |