M
Best token generation
1.593,1 tok/s
Meta-Llama-3.1-8B-Instruct
Best prefill
27.508,3 tok/s
Meta-Llama-3.1-8B-Instruct
∅ Token generation
322,7 tok/s
Average of 97 runs
∅ Prefill
7.773,3 tok/s
Average of 97 runs
Best TTFT
219 ms
Meta-Llama-3.1-8B-Instruct
Best harness rate
0%
no harness run yet
🏆
Meta-Llama-3.1-8B-Instruct · fastest model at 1.593,1 tok/s token generation
Best prefill of this manufacturer: 27.508,3 tok/s (Meta-Llama-3.1-8B-Instruct)
Best prefill of this manufacturer: 27.508,3 tok/s (Meta-Llama-3.1-8B-Instruct)
Top models by token generation
Best measured generation throughput per model (tok/s), published performance benchmarks.