Meta
Meta-Llama-3.1-8B-Instruct8B
Benchmark profile and published results.
Dense8B
Position in the field
Best values compared
Best metrics of this model against the minimum, average and maximum of all published systems.
Generation1.593,1 tok/s
Min 0,0Ø 248,3Max 2.491,2
Ueber dem Durchschnitt · 3969 Systeme im Feld
Prefill27.508 tok/s
Min 4Ø 4.741Max 55.260
Ueber dem Durchschnitt · 3969 Systeme im Feld
Time to First Token219 ms
Min 20Ø 33.962Max 535.235
Ueber dem Durchschnitt · 3967 Systeme im Feld
Performance profile
Throughput by hardware & engine
Each bubble is a GPU, CPU or engine – position shows prompt processing (X) and output speed (Y), bubble size the number of runs.
GPUby graphics card
NVIDIA GeForce RTX 5090 1.593,1 tok/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 1.533,0 tok/sNVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 1.433,0 tok/sNVIDIA RTX A6000 1.026,4 tok/sNVIDIA GeForce RTX 5070 Ti 913,4 tok/sNVIDIA GeForce RTX 3090 Ti 837,9 tok/sAMD Radeon AI PRO R9700 518,3 tok/sAMD Radeon PRO W7800 48GB 323,1 tok/sNVIDIA GeForce RTX 2060 130,9 tok/sNVIDIA Tesla P100 PCIe 16GB 67,6 tok/sIntel Arc Pro B70 66,4 tok/s
CPUby processor
AMD Ryzen 7 5800X3D 8-Core Processor 1.593,1 tok/sAMD Ryzen 9 9950X 16-Core Processor 1.533,0 tok/sAMD Ryzen Threadripper PRO 9965WX 24-Cores 1.433,0 tok/sAMD Ryzen Threadripper PRO 7955WX 16-Cores 1.026,4 tok/sAMD Ryzen Threadripper PRO 5975WX 32-Cores 913,4 tok/sAMD Ryzen 9 8945HX with Radeon Graphics 837,9 tok/sIntel(R) Core(TM) i5-7400 CPU @ 3.00GHz 130,9 tok/sAMD Ryzen 9 7945HX with Radeon Graphics 67,6 tok/sIntel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz 3,6 tok/s
Throughput & latency
Performance benchmark
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.593,10 tok/s TG Prefill 16.376 · TTFT 6.849 ms | 10× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 2 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.532,95 tok/s TG Prefill 17.861 · TTFT 6.225 ms | 10× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 3 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.432,98 tok/s TG Prefill 21.012 · TTFT 6.892 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 4 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.383,23 tok/s TG Prefill 17.796 · TTFT 7.072 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 5 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.267,62 tok/s TG Prefill 13.497 · TTFT 8.310 ms | 10× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 6 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.026,44 tok/s TG Prefill 26.805 · TTFT 1.371 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawAWQ | Details → | |
| 7 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 913,39 tok/s TG Prefill 8.119 · TTFT 11.247 ms | 10× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 8 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 876,87 tok/s TG Prefill 11.325 · TTFT 2.242 ms | 5× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 9 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 837,85 tok/s TG Prefill 9.559 · TTFT 12.952 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 10 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 832,96 tok/s TG Prefill 27.508 · TTFT 1.327 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawFP8 | Details → | |
| 11 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 823,64 tok/s TG Prefill 12.521 · TTFT 2.492 ms | 5× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 12 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 774,67 tok/s TG Prefill 20.868 · TTFT 2.009 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 13 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 766,19 tok/s TG Prefill 15.268 · TTFT 2.352 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 14 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 699,71 tok/s TG Prefill 13.042 · TTFT 2.524 ms | 5× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 15 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 675,29 tok/s TG Prefill 20.479 · TTFT 781 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawAWQ | Details → | |
| 16 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 576,78 tok/s TG Prefill 14.305 · TTFT 2.330 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliAWQ | Details → | |
| 17 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 534,82 tok/s TG Prefill 11.826 · TTFT 7.114 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 18 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 518,27 tok/s TG Prefill 13.775 · TTFT 18.702 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 19 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 510,30 tok/s TG Prefill 21.461 · TTFT 752 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawFP8 | Details → | |
| 20 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 508,08 tok/s TG Prefill 5.204 · TTFT 21.109 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 21 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 499,39 tok/s TG Prefill 11.446 · TTFT 7.175 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 22 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 489,71 tok/s TG Prefill 8.631 · TTFT 3.252 ms | 5× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 23 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 489,43 tok/s TG Prefill 10.618 · TTFT 18.898 ms | 10× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 24 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 482,61 tok/s TG Prefill 12.125 · TTFT 5.913 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 25 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 467,83 tok/s TG Prefill 11.804 · TTFT 6.038 ms | 10× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 26 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 456,70 tok/s TG Prefill 8.966 · TTFT 3.773 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 27 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 444,58 tok/s TG Prefill 15.076 · TTFT 2.208 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliFP8 | Details → | |
| 28 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 387,59 tok/s TG Prefill 10.645 · TTFT 1.347 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliAWQ | Details → | |
| 29 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 343,70 tok/s TG Prefill 9.999 · TTFT 3.230 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 30 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 323,12 tok/s TG Prefill 5.337 · TTFT 5.911 ms | 10× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 31 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 322,77 tok/s TG Prefill 5.717 · TTFT 5.160 ms | 10× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 32 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 281,80 tok/s TG Prefill 10.273 · TTFT 5.323 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 33 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 276,49 tok/s TG Prefill 9.874 · TTFT 3.124 ms | 10× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 34 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 275,72 tok/s TG Prefill 13.406 · TTFT 4.952 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 35 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 274,94 tok/s TG Prefill 6.188 · TTFT 6.209 ms | 5× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 36 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 274,89 tok/s TG Prefill 8.972 · TTFT 1.446 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 37 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 272,75 tok/s TG Prefill 11.264 · TTFT 1.275 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliFP8 | Details → | |
| 38 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 262,93 tok/s TG Prefill 8.472 · TTFT 4.535 ms | 10× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | vLLMgodclawAWQ | Details → | |
| 39 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 253,38 tok/s TG Prefill 9.752 · TTFT 1.444 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 40 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 246,17 tok/s TG Prefill 6.676 · TTFT 369 ms | 1× | NVIDIA GeForce RTX 5090AMD Ryzen 7 5800X3D 8-Core Processor · 126 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 41 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 239,32 tok/s TG Prefill 11.230 · TTFT 219 ms | 1× | NVIDIA RTX PRO 6000 Blackwell Workstation EditionAMD Ryzen 9 9950X 16-Core Processor · 92 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 42 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 237,04 tok/s TG Prefill 7.169 · TTFT 1.831 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 43 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 225,01 tok/s TG Prefill 9.669 · TTFT 257 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 44 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 222,93 tok/s TG Prefill 8.663 · TTFT 286 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 45 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 212,79 tok/s TG Prefill 8.116 · TTFT 304 ms | 1× | 3x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionAMD Ryzen Threadripper PRO 9965WX 24-Cores · 125 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 46 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 208,70 tok/s TG Prefill 9.730 · TTFT 1.396 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 47 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 197,82 tok/s TG Prefill 3.730 · TTFT 3.658 ms | 5× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 48 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 197,34 tok/s TG Prefill 6.940 · TTFT 1.849 ms | 5× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 49 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 189,29 tok/s TG Prefill 10.070 · TTFT 1.358 ms | 5× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 50 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 181,22 tok/s TG Prefill 9.264 · TTFT 266 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawAWQ | Details → | |
| 51 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 159,48 tok/s TG Prefill 4.475 · TTFT 2.882 ms | 5× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 52 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 156,61 tok/s TG Prefill 3.757 · TTFT 4.751 ms | 10× | AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 53 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 148,21 tok/s TG Prefill 5.615 · TTFT 439 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 54 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 146,19 tok/s TG Prefill 6.653 · TTFT 2.534 ms | 5× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | vLLMgodclawAWQ | Details → | |
| 55 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 140,90 tok/s TG Prefill 2.783 · TTFT 1.005 ms | 1× | NVIDIA GeForce RTX 5070 TiAMD Ryzen Threadripper PRO 5975WX 32-Cores · 247 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 56 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 138,90 tok/s TG Prefill 3.102 · TTFT 4.748 ms | 5× | AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 57 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 130,88 tok/s TG Prefill 4.122 · TTFT 7.946 ms | 10× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 58 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 127,43 tok/s TG Prefill 9.480 · TTFT 260 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMgodclawFP8 | Details → | |
| 59 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 122,07 tok/s TG Prefill 3.576 · TTFT 3.191 ms | 5× | AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 60 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 116,23 tok/s TG Prefill 4.923 · TTFT 502 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 61 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 116,02 tok/s TG Prefill 4.973 · TTFT 496 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 62 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 102,47 tok/s TG Prefill 4.866 · TTFT 508 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 63 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 102,39 tok/s TG Prefill 5.871 · TTFT 419 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliAWQ | Details → | |
| 64 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 90,37 tok/s TG Prefill 3.202 · TTFT 4.555 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 65 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 89,46 tok/s TG Prefill 3.210 · TTFT 4.511 ms | 5× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 66 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 87,92 tok/s TG Prefill 4.116 · TTFT 600 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 67 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 87,03 tok/s TG Prefill 3.930 · TTFT 629 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 68 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 85,98 tok/s TG Prefill 6.398 · TTFT 386 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 69 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 84,89 tok/s TG Prefill 8.039 · TTFT 307 ms | 1× | 3x AMD Radeon AI PRO R9700AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 70 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 82,52 tok/s TG Prefill 2.347 · TTFT 1.048 ms | 1× | AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 71 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 81,91 tok/s TG Prefill 2.340 · TTFT 1.052 ms | 1× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ4_K_M | Details → | |
| 72 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 75,09 tok/s TG Prefill 4.946 · TTFT 499 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 73 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 75,09 tok/s TG Prefill 5.190 · TTFT 476 ms | 1× | 2x NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 74 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 72,25 tok/s TG Prefill 1.990 · TTFT 1.240 ms | 1× | AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 75 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 69,31 tok/s TG Prefill 1.997 · TTFT 1.235 ms | 1× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | llama.cppgodclawQ8_0 | Details → | |
| 76 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 67,58 tok/s TG Prefill 888 · TTFT 15.224 ms | 5× | NVIDIA Tesla P100 PCIe 16GBAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 77 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 66,99 tok/s TG Prefill 5.119 · TTFT 482 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 78 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 66,46 tok/s TG Prefill 6.040 · TTFT 408 ms | 1× | NVIDIA RTX A6000AMD Ryzen Threadripper PRO 7955WX 16-Cores · 184 GB RAM | vLLMopenclaw_cliFP8 | Details → | |
| 79 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 66,43 tok/s TG Prefill 898 · TTFT 2.781 ms | 1× | Intel Arc Pro B70AMD Ryzen 9 7945HX with Radeon Graphics · 29 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 80 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 48,95 tok/s TG Prefill 1.278 · TTFT 1.932 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 81 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 47,41 tok/s TG Prefill 1.800 · TTFT 1.368 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 82 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 45,73 tok/s TG Prefill 1.206 · TTFT 2.050 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 83 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 42,11 tok/s TG Prefill 821 · TTFT 2.298 ms | 1× | Intel Arc Pro B70AMD Ryzen 9 7945HX with Radeon Graphics · 29 GB RAM | openclaw_cli | Details → | |
| 84 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 37,29 tok/s TG Prefill 1.115 · TTFT 27.826 ms | 10× | NVIDIA Tesla P100 PCIe 16GBAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 85 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 35,46 tok/s TG Prefill 589 · TTFT 4.189 ms | 1× | NVIDIA Tesla P100 PCIe 16GBAMD Ryzen 9 7945HX with Radeon Graphics · 60 GB RAM | llama.cppopenclaw_cliQ4_K_M | Details → | |
| 86 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 34,53 tok/s TG Prefill 2.874 · TTFT 857 ms | 1× | 2x AMD Radeon PRO W7800 48GBAMD Ryzen Threadripper PRO 5975WX 32-Cores · 62 GB RAM | vLLMgodclawAWQ | Details → | |
| 87 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 31,83 tok/s TG Prefill 1.653 · TTFT 1.506 ms | 1× | 2x NVIDIA GeForce RTX 2060Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz · 7 GB RAM | llama.cppopenclaw_cliQ8_0 | Details → | |
| 88 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3,55 tok/s TG Prefill 36 · TTFT 59.044 ms | 1× | 2x NVIDIA GeForce RTX 20604x Intel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz · 504 GB RAM | llama.cppgodclawQ4_K_M | Details → |
Agent & chat rating
Harness benchmark
Noch keine Harnessbenchmarks fuer dieses Modell.