⚡
Performance benchmark
How fast is the model?
Raw throughput on real hardware – how many tokens a model generates per second, how quickly it processes the prompt and how short the time to first token is. Higher is better.
⚡ Generation (tok/s)🔄 Prefill⏱️ TTFT👥 Concurrency
| # | Model / Maker | Metrics | Parallel | GPU / CPU / RAM | Runtime | ||
|---|---|---|---|---|---|---|---|
| 1 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.492,78 tok/s TG Prefill 11.476 · TTFT 7.075 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 2 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 1.113,51 tok/s TG Prefill 6.554 · TTFT 10.391 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 3 | Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 992,11 tok/s TG Prefill 5.920 · TTFT 12.083 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 4 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 986,05 tok/s TG Prefill 5.179 · TTFT 13.058 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 5 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 983,68 tok/s TG Prefill 7.362 · TTFT 10.840 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 6 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 981,72 tok/s TG Prefill 5.166 · TTFT 13.134 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 7 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 979,74 tok/s TG Prefill 5.175 · TTFT 13.143 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 8 | North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 939,51 tok/s TG Prefill 4.611 · TTFT 12.995 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 9 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 930,61 tok/s TG Prefill 5.307 · TTFT 13.161 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 10 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 928,48 tok/s TG Prefill 7.663 · TTFT 12.219 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 11 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 927,42 tok/s TG Prefill 5.040 · TTFT 13.415 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 12 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 926,20 tok/s TG Prefill 3.875 · TTFT 14.083 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 13 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 920,16 tok/s TG Prefill 3.818 · TTFT 13.771 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 14 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 905,10 tok/s TG Prefill 7.532 · TTFT 12.527 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 15 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 903,53 tok/s TG Prefill 7.168 · TTFT 12.726 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 16 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 901,86 tok/s TG Prefill 12.965 · TTFT 12.663 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 17 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 897,47 tok/s TG Prefill 4.560 · TTFT 13.829 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 18 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 897,46 tok/s TG Prefill 4.457 · TTFT 13.871 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 19 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 896,55 tok/s TG Prefill 4.436 · TTFT 13.922 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 20 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 883,71 tok/s TG Prefill 12.981 · TTFT 12.917 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 21 | Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 866,12 tok/s TG Prefill 3.825 · TTFT 14.245 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 22 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 851,97 tok/s TG Prefill 3.890 · TTFT 14.433 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 23 | Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 849,15 tok/s TG Prefill 3.913 · TTFT 14.486 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 24 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 837,85 tok/s TG Prefill 9.559 · TTFT 12.952 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 25 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 830,46 tok/s TG Prefill 3.771 · TTFT 14.928 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | hermes_cli | Details → | |
| 26 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 829,45 tok/s TG Prefill 3.778 · TTFT 14.891 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | hermes_cli | Details → | |
| 27 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 828,36 tok/s TG Prefill 3.763 · TTFT 14.940 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppQ4_K_M | Details → | |
| 28 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 827,28 tok/s TG Prefill 9.803 · TTFT 2.242 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 29 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 797,18 tok/s TG Prefill 11.820 · TTFT 1.789 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 30 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 796,82 tok/s TG Prefill 11.892 · TTFT 1.781 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 31 | Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 786,22 tok/s TG Prefill 4.926 · TTFT 14.292 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 32 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 670,65 tok/s TG Prefill 17.357 · TTFT 3.808 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 33 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 656,13 tok/s TG Prefill 5.934 · TTFT 3.387 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 34 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 654,86 tok/s TG Prefill 2.799 · TTFT 19.412 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 35 | Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 596,31 tok/s TG Prefill 5.389 · TTFT 4.001 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 36 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 586,52 tok/s TG Prefill 4.479 · TTFT 4.845 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 37 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 584,39 tok/s TG Prefill 4.467 · TTFT 4.864 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 38 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 582,20 tok/s TG Prefill 4.480 · TTFT 4.869 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 39 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 572,10 tok/s TG Prefill 10.901 · TTFT 1.915 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm mxfp4 | Details → | |
| 40 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 558,18 tok/s TG Prefill 4.694 · TTFT 4.497 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 41 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 556,41 tok/s TG Prefill 4.468 · TTFT 4.620 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 42 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 554,75 tok/s TG Prefill 7.036 · TTFT 3.930 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 43 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 550,00 tok/s TG Prefill 3.904 · TTFT 4.878 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 44 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 548,84 tok/s TG Prefill 4.012 · TTFT 4.835 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 45 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 547,85 tok/s TG Prefill 3.914 · TTFT 4.883 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 46 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 543,05 tok/s TG Prefill 4.878 · TTFT 20.084 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 47 | North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 542,97 tok/s TG Prefill 4.226 · TTFT 4.294 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 48 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 542,88 tok/s TG Prefill 6.558 · TTFT 4.129 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 49 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 542,58 tok/s TG Prefill 6.522 · TTFT 3.407 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 50 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 542,30 tok/s TG Prefill 6.912 · TTFT 4.026 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 51 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 524,37 tok/s TG Prefill 7.751 · TTFT 4.130 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 52 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 522,97 tok/s TG Prefill 3.432 · TTFT 4.794 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 53 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 517,07 tok/s TG Prefill 3.355 · TTFT 4.929 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 54 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 496,79 tok/s TG Prefill 12.291 · TTFT 3.700 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 55 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 487,24 tok/s TG Prefill 12.271 · TTFT 3.758 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 56 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 483,69 tok/s TG Prefill 9.270 · TTFT 1.070 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 57 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 479,47 tok/s TG Prefill 9.242 · TTFT 1.075 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 58 | Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 479,14 tok/s TG Prefill 3.365 · TTFT 5.049 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 59 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 472,34 tok/s TG Prefill 3.434 · TTFT 5.070 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 60 | Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 471,29 tok/s TG Prefill 3.464 · TTFT 5.057 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 61 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 470,17 tok/s TG Prefill 6.563 · TTFT 3.093 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 62 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 468,74 tok/s TG Prefill 6.617 · TTFT 3.069 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 63 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 467,09 tok/s TG Prefill 3.349 · TTFT 5.143 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppQ4_K_M | Details → | |
| 64 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 463,67 tok/s TG Prefill 5.822 · TTFT 1.705 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 65 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 456,70 tok/s TG Prefill 8.966 · TTFT 3.773 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 66 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 451,05 tok/s TG Prefill 32.005 · TTFT 864 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm | Details → | |
| 67 | Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 430,04 tok/s TG Prefill 4.493 · TTFT 4.596 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 68 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 404,72 tok/s TG Prefill 4.793 · TTFT 5.226 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppIQ4_NL | Details → | |
| 69 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 403,67 tok/s TG Prefill 5.727 · TTFT 1.727 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 70 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 399,08 tok/s TG Prefill 14.492 · TTFT 984 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 71 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 397,18 tok/s TG Prefill 3.476 · TTFT 5.916 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 72 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 372,40 tok/s TG Prefill 1.508 · TTFT 13.832 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMllamacppAWQ | Details → | |
| 73 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 355,09 tok/s TG Prefill 2.494 · TTFT 6.574 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 74 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 353,84 tok/s TG Prefill 924 · TTFT 13.139 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 75 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 346,29 tok/s TG Prefill 16.028 · TTFT 1.776 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm bf16 | Details → | |
| 76 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 338,30 tok/s TG Prefill 5.943 · TTFT 2.299 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 77 | Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 331,64 tok/s TG Prefill 4.743 · TTFT 31.297 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 78 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 330,94 tok/s TG Prefill 4.379 · TTFT 7.353 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 79 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 325,20 tok/s TG Prefill 3.507 · TTFT 31.980 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 80 | Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 317,44 tok/s TG Prefill 6.072 · TTFT 23.382 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 81 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 308,36 tok/s TG Prefill 9.336 · TTFT 1.062 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm mxfp4 | Details → | |
| 82 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 306,80 tok/s TG Prefill 9.358 · TTFT 1.060 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm mxfp4 | Details → | |
| 83 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 300,10 tok/s TG Prefill 3.941 · TTFT 2.860 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppIQ4_NL | Details → | |
| 84 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 294,82 tok/s TG Prefill 4.538 · TTFT 6.199 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 85 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 287,13 tok/s TG Prefill 3.082 · TTFT 3.211 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 86 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 277,86 tok/s TG Prefill 3.781 · TTFT 10.183 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllmAWQ | Details → | |
| 87 | Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 264,22 tok/s TG Prefill 1.610 · TTFT 42.712 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 88 | Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 247,69 tok/s TG Prefill 5.149 · TTFT 6.214 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 89 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 246,06 tok/s TG Prefill 5.285 · TTFT 374 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 90 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 245,58 tok/s TG Prefill 5.520 · TTFT 359 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 91 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 238,87 tok/s TG Prefill 5.096 · TTFT 388 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 92 | Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 236,91 tok/s TG Prefill 4.079 · TTFT 528 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 93 | OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 236,79 tok/s TG Prefill 2.409 · TTFT 46.406 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 94 | Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 236,42 tok/s TG Prefill 2.420 · TTFT 46.503 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 95 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 235,80 tok/s TG Prefill 8.654 · TTFT 230 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 96 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 228,02 tok/s TG Prefill 11.489 · TTFT 2.605 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm | Details → | |
| 97 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 226,20 tok/s TG Prefill 2.929 · TTFT 4.189 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 98 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 223,82 tok/s TG Prefill 2.436 · TTFT 44.343 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 99 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 222,01 tok/s TG Prefill 2.436 · TTFT 45.755 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketAWQ | Details → | |
| 100 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 221,98 tok/s TG Prefill 3.364 · TTFT 641 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 101 | Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 221,81 tok/s TG Prefill 3.230 · TTFT 666 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 102 | Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 221,80 tok/s TG Prefill 3.314 · TTFT 655 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 103 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 214,10 tok/s TG Prefill 4.633 · TTFT 8.462 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 104 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,76 tok/s TG Prefill 4.024 · TTFT 605 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 105 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,44 tok/s TG Prefill 4.311 · TTFT 565 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 106 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,34 tok/s TG Prefill 3.833 · TTFT 717 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 107 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,29 tok/s TG Prefill 3.814 · TTFT 720 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 108 | Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,25 tok/s TG Prefill 3.820 · TTFT 720 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 109 | Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 198,07 tok/s TG Prefill 3.972 · TTFT 613 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 110 | Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 197,73 tok/s TG Prefill 4.076 · TTFT 598 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 111 | Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 196,02 tok/s TG Prefill 3.211 · TTFT 670 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppIQ4_NL | Details → | |
| 112 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 195,39 tok/s TG Prefill 4.296 · TTFT 567 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 113 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 184,33 tok/s TG Prefill 2.857 · TTFT 5.676 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllmAWQ | Details → | |
| 114 | Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 184,15 tok/s TG Prefill 2.849 · TTFT 676 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 115 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 183,69 tok/s TG Prefill 2.125 · TTFT 4.654 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMllamacppAWQ | Details → | |
| 116 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 183,68 tok/s TG Prefill 853 · TTFT 1.050 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 117 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 178,38 tok/s TG Prefill 3.257 · TTFT 10.368 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 118 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 177,40 tok/s TG Prefill 11.718 · TTFT 941 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm bf16 | Details → | |
| 119 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 176,18 tok/s TG Prefill 8.074 · TTFT 245 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 120 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 176,06 tok/s TG Prefill 7.970 · TTFT 248 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 121 | Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 175,32 tok/s TG Prefill 3.605 · TTFT 9.714 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 122 | North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 175,06 tok/s TG Prefill 3.551 · TTFT 557 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 123 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 174,07 tok/s TG Prefill 1.206 · TTFT 16.925 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 124 | Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 173,35 tok/s TG Prefill 4.438 · TTFT 8.093 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 125 | Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 171,72 tok/s TG Prefill 7.637 · TTFT 319 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 126 | Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 168,65 tok/s TG Prefill 7.081 · TTFT 345 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 127 | Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 168,59 tok/s TG Prefill 6.991 · TTFT 350 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 128 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 166,99 tok/s TG Prefill 3.555 · TTFT 556 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 129 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 163,95 tok/s TG Prefill 2.970 · TTFT 649 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 130 | Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 163,95 tok/s TG Prefill 2.997 · TTFT 644 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 131 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 162,37 tok/s TG Prefill 2.928 · TTFT 659 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | hermes_cli | Details → | |
| 132 | Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 162,17 tok/s TG Prefill 2.928 · TTFT 660 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacppQ4_K_M | Details → | |
| 133 | Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 161,86 tok/s TG Prefill 2.922 · TTFT 660 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 134 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 156,49 tok/s TG Prefill 182 · TTFT 97.092 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llama.cpp q3_k_m, q4-kv | Details → | |
| 135 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 155,81 tok/s TG Prefill 6.094 · TTFT 325 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 136 | Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 148,21 tok/s TG Prefill 5.615 · TTFT 439 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 137 | Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 142,65 tok/s TG Prefill 1.474 · TTFT 13.865 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 138 | Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 133,84 tok/s TG Prefill 3.838 · TTFT 503 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 139 | OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 127,74 tok/s TG Prefill 2.245 · TTFT 14.449 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 140 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 127,64 tok/s TG Prefill 2.262 · TTFT 14.457 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 141 | Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 127,44 tok/s TG Prefill 2.255 · TTFT 14.493 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 142 | Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 125,87 tok/s TG Prefill 3.206 · TTFT 618 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 143 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 125,21 tok/s TG Prefill 3.410 · TTFT 4.809 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 144 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 124,33 tok/s TG Prefill 2.267 · TTFT 13.440 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketAWQ | Details → | |
| 145 | Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 123,94 tok/s TG Prefill 3.957 · TTFT 3.439 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 146 | Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 111,64 tok/s TG Prefill 1.065 · TTFT 92.938 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 147 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 93,10 tok/s TG Prefill 2.557 · TTFT 773 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_XL | Details → | |
| 148 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 89,91 tok/s TG Prefill 3.497 · TTFT 648 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 149 | Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 89,62 tok/s TG Prefill 2.949 · TTFT 726 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 150 | gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 85,33 tok/s TG Prefill 270 · TTFT 166.013 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 151 | gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 83,56 tok/s TG Prefill 2.303 · TTFT 859 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMllamacppAWQ | Details → | |
| 152 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 81,35 tok/s TG Prefill 998 · TTFT 9.924 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 153 | Qwen3-Coder-NextQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 78,23 tok/s TG Prefill 354 · TTFT 169.650 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 154 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 74,79 tok/s TG Prefill 1.062 · TTFT 9.322 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llama.cpp q3_k_m, q4-kv | Details → | |
| 155 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 61,64 tok/s TG Prefill 5.671 · TTFT 366 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm mxfp4 | Details → | |
| 156 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 59,06 tok/s TG Prefill 405 · TTFT 20 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 157 | Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 58,76 tok/s TG Prefill 987 · TTFT 28.850 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 158 | Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 58,52 tok/s TG Prefill 351 · TTFT 222.335 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 159 | Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 58,03 tok/s TG Prefill 1.644 · TTFT 1.302 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocketAWQ | Details → | |
| 160 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 57,55 tok/s TG Prefill 1.941 · TTFT 1.402 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 161 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 57,45 tok/s TG Prefill 1.958 · TTFT 1.391 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 162 | Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 57,12 tok/s TG Prefill 3.392 · TTFT 922 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 163 | Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 56,94 tok/s TG Prefill 3.347 · TTFT 1.030 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 164 | Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 56,18 tok/s TG Prefill 2.655 · TTFT 1.019 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 165 | gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 51,30 tok/s TG Prefill 232 · TTFT 64.035 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 166 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 47,58 tok/s TG Prefill 10.520 · TTFT 189 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllmBF16 | Details → | |
| 167 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 46,20 tok/s TG Prefill 11.058 · TTFT 180 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm | Details → | |
| 168 | gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 46,01 tok/s TG Prefill 11.757 · TTFT 169 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllmBF16 | Details → | |
| 169 | Qwen3-Coder-NextQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 45,84 tok/s TG Prefill 299 · TTFT 64.947 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 170 | Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 45,38 tok/s TG Prefill 1.437 · TTFT 1.875 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vllm | Details → | |
| 171 | Qwen3.6-27B27BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 43,62 tok/s TG Prefill 1.290 · TTFT 1.497 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 172 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 40,26 tok/s TG Prefill 1.449 · TTFT 1.698 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 173 | OpenReasoning-Nemotron-32B32BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 40,24 tok/s TG Prefill 1.453 · TTFT 1.685 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 174 | Qwen2.5-Coder-32B-Instruct32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 40,22 tok/s TG Prefill 1.445 · TTFT 1.703 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 175 | Qwen2.5-32B-Instruct-AWQ32BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 40,18 tok/s TG Prefill 1.444 · TTFT 1.705 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketAWQ | Details → | |
| 176 | gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 37,86 tok/s TG Prefill 6.286 · TTFT 315 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | vLLMvllm bf16 | Details → | |
| 177 | Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 35,54 tok/s TG Prefill 233 · TTFT 92.559 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 178 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 32,82 tok/s TG Prefill 177 · TTFT 255.495 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketINT4 | Details → | |
| 179 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 31,55 tok/s TG Prefill 174 · TTFT 246.888 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 180 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 30,10 tok/s TG Prefill 1.020 · TTFT 1.964 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llama.cpp q3_k_m, q4-kv | Details → | |
| 181 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 29,73 tok/s TG Prefill 888 · TTFT 25.625 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 182 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 28,74 tok/s TG Prefill 938 · TTFT 2.110 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAM | llamacpp | Details → | |
| 183 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 25,54 tok/s TG Prefill 157 · TTFT 125.314 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 184 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 25,50 tok/s TG Prefill 157 · TTFT 125.739 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketINT4 | Details → | |
| 185 | Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 24,87 tok/s TG Prefill 138 · TTFT 262.075 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 186 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 21,98 tok/s TG Prefill 642 · TTFT 15.446 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 187 | gpt-oss-120b120BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 21,12 tok/s TG Prefill 213 · TTFT 9.303 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 188 | GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 20,78 tok/s TG Prefill 144 · TTFT 302.096 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 189 | Llama-3.3-Nemotron-Super-49B-v1.549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 19,60 tok/s TG Prefill 427 · TTFT 182.825 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 190 | gemma-4-31B-it31BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 19,50 tok/s TG Prefill 339 · TTFT 80.766 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llmcodex_cli | Details → | |
| 191 | Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 18,86 tok/s TG Prefill 117 · TTFT 322.251 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 192 | Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 18,77 tok/s TG Prefill 126 · TTFT 135.104 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 193 | Llama-3.3-Nemotron-Super-49B-v1.549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 18,48 tok/s TG Prefill 410 · TTFT 83.981 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 194 | Qwen3-Coder-NextQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 18,14 tok/s TG Prefill 279 · TTFT 8.732 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 195 | Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 16,87 tok/s TG Prefill 213 · TTFT 278.913 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 196 | Seed-OSS-36B-Instruct36BByteDance Seed Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 16,70 tok/s TG Prefill 924 · TTFT 2.388 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 197 | Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 16,07 tok/s TG Prefill 104 · TTFT 170.078 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 198 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 15,90 tok/s TG Prefill 157 · TTFT 163.779 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 199 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 15,80 tok/s TG Prefill 156 · TTFT 171.016 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 200 | GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 14,86 tok/s TG Prefill 133 · TTFT 150.792 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 201 | Mistral-Small-4-119B-2603119BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 14,47 tok/s TG Prefill 190 · TTFT 14.242 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 202 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 12,63 tok/s TG Prefill 153 · TTFT 195.676 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 203 | Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 12,02 tok/s TG Prefill 219 · TTFT 125.268 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 204 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 11,25 tok/s TG Prefill 148 · TTFT 197.892 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 205 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 9,12 tok/s TG Prefill 157 · TTFT 17.486 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketINT4 | Details → | |
| 206 | Laguna-S-2.1-INT4Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 9,06 tok/s TG Prefill 156 · TTFT 17.644 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 207 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 8,22 tok/s TG Prefill 70 · TTFT 38.073 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 208 | MiniMax-M2.5230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 8,20 tok/s TG Prefill 72 · TTFT 35.231 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 209 | gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 7,66 tok/s TG Prefill 533 · TTFT 3.719 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | vLLMwebsocket | Details → | |
| 210 | GLM-4.5-Air106BZ.ai (Zhipu) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 7,11 tok/s TG Prefill 133 · TTFT 16.005 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 211 | Qwen3.5-122B-A10B122BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 6,96 tok/s TG Prefill 123 · TTFT 15.650 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 212 | Nemotron-3-Super-120B-A12B120BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 6,67 tok/s TG Prefill 99 · TTFT 21.860 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 213 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,90 tok/s TG Prefill 239 · TTFT 40.388 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 214 | Qwen2.5-72B-Instruct72BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,84 tok/s TG Prefill 239 · TTFT 40.453 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 215 | Llama-3.3-Nemotron-Super-49B-v1.549BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,61 tok/s TG Prefill 384 · TTFT 6.371 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 216 | command-a-reasoning-08-2025111BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,18 tok/s TG Prefill 91 · TTFT 93.827 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 217 | command-a-reasoning-08-2025111BCohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 4,11 tok/s TG Prefill 85 · TTFT 100.325 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 218 | MiniMax-M2.7230BMiniMax Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3,95 tok/s TG Prefill 10 · TTFT 246.180 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketUD-Q4_K_M | Details → | |
| 219 | Nemotron-H-47B-Reasoning-128K47BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3,69 tok/s TG Prefill 212 · TTFT 10.143 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 220 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3,08 tok/s TG Prefill 8 · TTFT 281.078 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 221 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 3,01 tok/s TG Prefill 5 · TTFT 397.363 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 222 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,98 tok/s TG Prefill 5 · TTFT 410.515 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 223 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,98 tok/s TG Prefill 5 · TTFT 408.262 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 224 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,98 tok/s TG Prefill 5 · TTFT 416.638 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 225 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,97 tok/s TG Prefill 5 · TTFT 395.460 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 226 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,96 tok/s TG Prefill 5 · TTFT 418.804 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 227 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,95 tok/s TG Prefill 5 · TTFT 428.772 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 228 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,92 tok/s TG Prefill 5 · TTFT 406.612 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 229 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,91 tok/s TG Prefill 5 · TTFT 408.023 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 230 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,91 tok/s TG Prefill 5 · TTFT 413.865 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 231 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,90 tok/s TG Prefill 5 · TTFT 429.316 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 232 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,90 tok/s TG Prefill 5 · TTFT 409.419 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 233 | Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,90 tok/s TG Prefill 147 · TTFT 70.727 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 234 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,89 tok/s TG Prefill 6 · TTFT 385.101 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 235 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,88 tok/s TG Prefill 5 · TTFT 401.414 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 236 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,88 tok/s TG Prefill 5 · TTFT 405.191 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 237 | Devstral-2-123B-Instruct-2512123BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,88 tok/s TG Prefill 147 · TTFT 70.636 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 238 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,86 tok/s TG Prefill 5 · TTFT 399.841 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 239 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,85 tok/s TG Prefill 5 · TTFT 407.941 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 240 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,84 tok/s TG Prefill 5 · TTFT 423.222 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 241 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,84 tok/s TG Prefill 5 · TTFT 401.206 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 242 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,83 tok/s TG Prefill 5 · TTFT 410.851 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 243 | Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,83 tok/s TG Prefill 109 · TTFT 85.257 ms | 5× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 244 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,81 tok/s TG Prefill 5 · TTFT 459.880 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppcodex_cliQ4_K_M | Details → | |
| 245 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,81 tok/s TG Prefill 5 · TTFT 410.587 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 246 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,80 tok/s TG Prefill 5 · TTFT 407.953 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 247 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,79 tok/s TG Prefill 5 · TTFT 414.339 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 248 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,76 tok/s TG Prefill 5 · TTFT 397.216 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 249 | Mistral-Medium-3.5-128B128BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,74 tok/s TG Prefill 145 · TTFT 71.210 ms | 10× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → | |
| 250 | Laguna-M.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation) | 2,71 tok/s TG Prefill 5 · TTFT 392.723 ms | 1× | NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAM | llama.cppwebsocketQ4_K_M | Details → |