Created bymario-alka.dePowered bygodcore.denoob2claw.detricoma.de
DEEN
Sign in Sign up
Beta
🎮
Graphics card

NVIDIA GeForce RTX 3090 Ti

NVIDIA24 GB VRAM
★★★★★5,0/ 5 · Bewertung
264Benchmarks
49Models
264Performance
0Harness
25 WIdle power
450 WMax power
Best generation
1.492,8 tok/s
gemma-4-E2B-it
∅ Generation
256,4 tok/s
Schnitt aus 264 Läufen
Best prefill
32.005 tok/s
Prompt processing
Best TTFT
20 ms
Time to First Token
Best harness rate
–%
noch kein Harness
🏆
gemma-4-E2B-it läuft am schnellsten auf dieser GPU · 1.492,8 tok/s
Bester Prefill: 32.005 tok/s · 49 Modelle getestet

Top models

Bester gemessener Durchsatz je Modell auf NVIDIA GeForce RTX 3090 Ti – umschaltbar nach Generation, Prefill oder Kombiwert.

gemma-4-E2B-it5B
1.492,8 tok/s
gpt-oss-20b20B
1.113,5 tok/s
Nemotron-3-Nano-4B4B
992,1 tok/s
Laguna-XS-2.1
986,1 tok/s
gemma-4-E4B-it8B
983,7 tok/s
North-Mini-Code-1.0
939,5 tok/s
Qwen3-30B-A3B-Thinking-250730B
930,6 tok/s
Qwen3-Coder-30B-A3B-Instruct30B
928,5 tok/s
Qwen3-Omni-30B-A3B-Thinking30B
927,4 tok/s
Gemma-4-26B-A4B-it26B
926,2 tok/s
Anzeige
Performance profile

Durchsatz auf NVIDIA GeForce RTX 3090 Ti

Jede Blase steht für ein Modell, eine Engine bzw. eine CPU – Position: Prompt-Verarbeitung (X) × Ausgabe (Y), Blasengröße: Anzahl Messläufe.

MODnach Modell

1.8771.4089394690,007.20514.41121.616Prefill (tok/s)Generation (tok/s)gemma-4-E2B-it - 1.492,8 tok/s Generation, 17.762 tok/s Prefill, TTFT 3.393 ms (3 Laufe)gemma-4-E2B-itgpt-oss-20b - 1.113,5 tok/s Generation, 8.213 tok/s Prefill, TTFT 2.818 ms (11 Laufe)gpt-oss-20bNemotron-3-Nano-4B - 992,1 tok/s Generation, 5.655 tok/s Prefill, TTFT 8.042 ms (2 Laufe)Nemotron-3-Nano-4BLaguna-XS-2.1 - 986,1 tok/s Generation, 4.824 tok/s Prefill, TTFT 8.985 ms (6 Laufe)Laguna-XS-2.1gemma-4-E4B-it - 983,7 tok/s Generation, 9.971 tok/s Prefill, TTFT 5.341 ms (3 Laufe)gemma-4-E4B-itNorth-Mini-Code-1.0 - 939,5 tok/s Generation, 4.419 tok/s Prefill, TTFT 8.644 ms (2 Laufe)North-Mini-Code-1.0Qwen3-30B-A3B-Thinking-2507 - 930,6 tok/s Generation, 8.813 tok/s Prefill, TTFT 8.583 ms (4 Laufe)Qwen3-30B-A3B-Thinkin...Qwen3-Coder-30B-A3B-Instruct - 928,5 tok/s Generation, 11.637 tok/s Prefill, TTFT 5.235 ms (4 Laufe)Qwen3-Coder-30B-A3B-I...Qwen3-Omni-30B-A3B-Thinking - 927,4 tok/s Generation, 4.754 tok/s Prefill, TTFT 9.018 ms (2 Laufe)Qwen3-Omni-30B-A3B-Th...Gemma-4-26B-A4B-it - 926,2 tok/s Generation, 3.594 tok/s Prefill, TTFT 8.264 ms (3 Laufe)Gemma-4-26B-A4B-itOrnith-1.0-35B - 920,2 tok/s Generation, 3.586 tok/s Prefill, TTFT 9.350 ms (2 Laufe)Ornith-1.0-35BQwen3-30B-A3B-Instruct-2507 - 905,1 tok/s Generation, 9.925 tok/s Prefill, TTFT 8.229 ms (4 Laufe)Qwen3-30B-A3B-Instruc...Qwen3-VL-30B-A3B-Instruct - 903,5 tok/s Generation, 6.863 tok/s Prefill, TTFT 8.427 ms (2 Laufe)Qwen3-VL-30B-A3B-Inst...Nemotron-Cascade-2-30B-A3B - 897,5 tok/s Generation, 4.286 tok/s Prefill, TTFT 9.332 ms (2 Laufe)Nemotron-Cascade-2-30...Nemotron-3-Nano-30B-A3B - 897,5 tok/s Generation, 4.384 tok/s Prefill, TTFT 7.991 ms (3 Laufe)Nemotron-3-Nano-30B-A...Nemotron-3-Nano-Omni-30B-A3B-Reasoning - 896,6 tok/s Generation, 4.175 tok/s Prefill, TTFT 9.402 ms (2 Laufe)Nemotron-3-Nano-Omni-...Qwen3.5-35B-A3B - 866,1 tok/s Generation, 3.595 tok/s Prefill, TTFT 9.647 ms (2 Laufe)Qwen3.5-35B-A3BQwen3.6-35B-A3B - 852,0 tok/s Generation, 3.664 tok/s Prefill, TTFT 11.567 ms (6 Laufe)Qwen3.6-35B-A3BQwen-AgentWorld-35B-A3B - 849,2 tok/s Generation, 3.689 tok/s Prefill, TTFT 9.771 ms (2 Laufe)Qwen-AgentWorld-35B-A...Meta-Llama-3.1-8B-Instruct - 837,9 tok/s Generation, 9.262 tok/s Prefill, TTFT 8.362 ms (2 Laufe)Meta-Llama-3.1-8B-Ins...Ornith-1.0-9B - 786,2 tok/s Generation, 4.710 tok/s Prefill, TTFT 9.444 ms (2 Laufe)Ornith-1.0-9Bgemma-4-12B-it - 654,9 tok/s Generation, 2.267 tok/s Prefill, TTFT 13.272 ms (3 Laufe)gemma-4-12B-itMinistral-3-14B-Reasoning-2512 - 543,1 tok/s Generation, 6.191 tok/s Prefill, TTFT 8.838 ms (3 Laufe)Ministral-3-14B-Reaso...Magistral-Small-2509 - 353,8 tok/s Generation, 2.652 tok/s Prefill, TTFT 10.246 ms (2 Laufe)Magistral-Small-2509Devstral-Small-2-24B-Instruct-2512 - 331,6 tok/s Generation, 4.743 tok/s Prefill, TTFT 31.297 ms (1 Lauf)Devstral-Small-2-24B-...Codestral-22B-v0.1 - 325,2 tok/s Generation, 3.507 tok/s Prefill, TTFT 31.980 ms (1 Lauf)Codestral-22B-v0.1Devstral-Small-2507 - 317,4 tok/s Generation, 6.072 tok/s Prefill, TTFT 23.382 ms (1 Lauf)Devstral-Small-2507
gemma-4-E2B-it 1.492,8 tok/sgpt-oss-20b 1.113,5 tok/sNemotron-3-Nano-4B 992,1 tok/sLaguna-XS-2.1 986,1 tok/sgemma-4-E4B-it 983,7 tok/sNorth-Mini-Code-1.0 939,5 tok/sQwen3-30B-A3B-Thinking-2507 930,6 tok/sQwen3-Coder-30B-A3B-Instruct 928,5 tok/sQwen3-Omni-30B-A3B-Thinking 927,4 tok/sGemma-4-26B-A4B-it 926,2 tok/sOrnith-1.0-35B 920,2 tok/sQwen3-30B-A3B-Instruct-2507 905,1 tok/sQwen3-VL-30B-A3B-Instruct 903,5 tok/sNemotron-Cascade-2-30B-A3B 897,5 tok/sNemotron-3-Nano-30B-A3B 897,5 tok/sNemotron-3-Nano-Omni-30B-A3B-Reasoning 896,6 tok/sQwen3.5-35B-A3B 866,1 tok/sQwen3.6-35B-A3B 852,0 tok/sQwen-AgentWorld-35B-A3B 849,2 tok/sMeta-Llama-3.1-8B-Instruct 837,9 tok/sOrnith-1.0-9B 786,2 tok/sgemma-4-12B-it 654,9 tok/sMinistral-3-14B-Reasoning-2512 543,1 tok/sMagistral-Small-2509 353,8 tok/sDevstral-Small-2-24B-Instruct-2512 331,6 tok/sCodestral-22B-v0.1 325,2 tok/sDevstral-Small-2507 317,4 tok/s

ENGnach Engine

1.7751.4681.1628555491.6425.8039.96514.126Prefill (tok/s)Generation (tok/s)llama.cpp - 1.492,8 tok/s Generation, 5.243 tok/s Prefill, TTFT 9.703 ms (57 Laufe)llama.cppvLLM - 901,9 tok/s Generation, 11.947 tok/s Prefill, TTFT 4.454 ms (17 Laufe)vLLMunbekannt - 830,5 tok/s Generation, 3.821 tok/s Prefill, TTFT 10.174 ms (6 Laufe)unbekannt
llama.cpp 1.492,8 tok/svLLM 901,9 tok/sunbekannt 830,5 tok/s

CPUnach Prozessor

1.7751.4681.1628555495.4426.6957.9499.203Prefill (tok/s)Generation (tok/s)AMD Ryzen 9 8945HX with Radeon Graphics - 1.492,8 tok/s Generation, 6.307 tok/s Prefill, TTFT 8.720 ms (70 Laufe)AMD Ryzen 9 8945HX wi...AMD Ryzen 5 5600X 6-Core Processor - 830,5 tok/s Generation, 8.337 tok/s Prefill, TTFT 7.943 ms (10 Laufe)AMD Ryzen 5 5600X 6-C...
AMD Ryzen 9 8945HX with Radeon Graphics 1.492,8 tok/sAMD Ryzen 5 5600X 6-Core Processor 830,5 tok/s
Throughput & latency

Performancebenchmark

80 veroeffentlichte Performance-Läufe auf NVIDIA GeForce RTX 3090 Ti.

In the leaderboard →
Metric:
Size
#Model / MakerMetricsParallelGPU / CPU / RAMRuntime
1gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.492,78 tok/s TG
Prefill 11.476 · TTFT 7.075 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
2gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
1.113,51 tok/s TG
Prefill 6.554 · TTFT 10.391 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
3Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
992,11 tok/s TG
Prefill 5.920 · TTFT 12.083 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
4Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
986,05 tok/s TG
Prefill 5.179 · TTFT 13.058 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
5gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
983,68 tok/s TG
Prefill 7.362 · TTFT 10.840 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
6Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
981,72 tok/s TG
Prefill 5.166 · TTFT 13.134 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
7Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
979,74 tok/s TG
Prefill 5.175 · TTFT 13.143 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
8North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
939,51 tok/s TG
Prefill 4.611 · TTFT 12.995 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
9Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
930,61 tok/s TG
Prefill 5.307 · TTFT 13.161 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
10Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
928,48 tok/s TG
Prefill 7.663 · TTFT 12.219 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
11Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
927,42 tok/s TG
Prefill 5.040 · TTFT 13.415 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
12Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
926,20 tok/s TG
Prefill 3.875 · TTFT 14.083 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_XLDetails →
13Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
920,16 tok/s TG
Prefill 3.818 · TTFT 13.771 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
14Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
905,10 tok/s TG
Prefill 7.532 · TTFT 12.527 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
15Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
903,53 tok/s TG
Prefill 7.168 · TTFT 12.726 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
16Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
901,86 tok/s TG
Prefill 12.965 · TTFT 12.663 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
17Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
897,47 tok/s TG
Prefill 4.560 · TTFT 13.829 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
18Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
897,46 tok/s TG
Prefill 4.457 · TTFT 13.871 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
19Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
896,55 tok/s TG
Prefill 4.436 · TTFT 13.922 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
20Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
883,71 tok/s TG
Prefill 12.981 · TTFT 12.917 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
21Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
866,12 tok/s TG
Prefill 3.825 · TTFT 14.245 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
22Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
851,97 tok/s TG
Prefill 3.890 · TTFT 14.433 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
23Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
849,15 tok/s TG
Prefill 3.913 · TTFT 14.486 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
24Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
837,85 tok/s TG
Prefill 9.559 · TTFT 12.952 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
25Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
830,46 tok/s TG
Prefill 3.771 · TTFT 14.928 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMhermes_cliDetails →
26Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
829,45 tok/s TG
Prefill 3.778 · TTFT 14.891 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMhermes_cliDetails →
27Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
828,36 tok/s TG
Prefill 3.763 · TTFT 14.940 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMllamacppQ4_K_MDetails →
28gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
827,28 tok/s TG
Prefill 9.803 · TTFT 2.242 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
29gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
797,18 tok/s TG
Prefill 11.820 · TTFT 1.789 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketDetails →
30gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
796,82 tok/s TG
Prefill 11.892 · TTFT 1.781 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketDetails →
31Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
786,22 tok/s TG
Prefill 4.926 · TTFT 14.292 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
32Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
670,65 tok/s TG
Prefill 17.357 · TTFT 3.808 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
33gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
656,13 tok/s TG
Prefill 5.934 · TTFT 3.387 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
34gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
654,86 tok/s TG
Prefill 2.799 · TTFT 19.412 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_XLDetails →
35Nemotron-3-Nano-4B4BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
596,31 tok/s TG
Prefill 5.389 · TTFT 4.001 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
36Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
586,52 tok/s TG
Prefill 4.479 · TTFT 4.845 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
37Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
584,39 tok/s TG
Prefill 4.467 · TTFT 4.864 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
38Laguna-XS-2.1Poolside Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
582,20 tok/s TG
Prefill 4.480 · TTFT 4.869 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
39gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
572,10 tok/s TG
Prefill 10.901 · TTFT 1.915 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllm mxfp4Details →
40Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
558,18 tok/s TG
Prefill 4.694 · TTFT 4.497 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
41Qwen3-Omni-30B-A3B-Thinking30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
556,41 tok/s TG
Prefill 4.468 · TTFT 4.620 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
42Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
554,75 tok/s TG
Prefill 7.036 · TTFT 3.930 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
43Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
550,00 tok/s TG
Prefill 3.904 · TTFT 4.878 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
44Nemotron-Cascade-2-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
548,84 tok/s TG
Prefill 4.012 · TTFT 4.835 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
45Nemotron-3-Nano-Omni-30B-A3B-Reasoning30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
547,85 tok/s TG
Prefill 3.914 · TTFT 4.883 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
46Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
543,05 tok/s TG
Prefill 4.878 · TTFT 20.084 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
47North-Mini-Code-1.0Cohere Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
542,97 tok/s TG
Prefill 4.226 · TTFT 4.294 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
48Qwen3-VL-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
542,88 tok/s TG
Prefill 6.558 · TTFT 4.129 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
49gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
542,58 tok/s TG
Prefill 6.522 · TTFT 3.407 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
50Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
542,30 tok/s TG
Prefill 6.912 · TTFT 4.026 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
51Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
524,37 tok/s TG
Prefill 7.751 · TTFT 4.130 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
52Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
522,97 tok/s TG
Prefill 3.432 · TTFT 4.794 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_XLDetails →
53Ornith-1.0-35B35BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
517,07 tok/s TG
Prefill 3.355 · TTFT 4.929 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
54Qwen3-30B-A3B-Instruct-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
496,79 tok/s TG
Prefill 12.291 · TTFT 3.700 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
55Qwen3-30B-A3B-Thinking-250730BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
487,24 tok/s TG
Prefill 12.271 · TTFT 3.758 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
56gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
483,69 tok/s TG
Prefill 9.270 · TTFT 1.070 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketDetails →
57gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
479,47 tok/s TG
Prefill 9.242 · TTFT 1.075 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketDetails →
58Qwen3.5-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
479,14 tok/s TG
Prefill 3.365 · TTFT 5.049 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
59Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
472,34 tok/s TG
Prefill 3.434 · TTFT 5.070 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
60Qwen-AgentWorld-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
471,29 tok/s TG
Prefill 3.464 · TTFT 5.057 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_MDetails →
61gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
470,17 tok/s TG
Prefill 6.563 · TTFT 3.093 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
62gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
468,74 tok/s TG
Prefill 6.617 · TTFT 3.069 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
63Qwen3.6-35B-A3B35BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
467,09 tok/s TG
Prefill 3.349 · TTFT 5.143 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMllamacppQ4_K_MDetails →
64gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
463,67 tok/s TG
Prefill 5.822 · TTFT 1.705 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
65Meta-Llama-3.1-8B-Instruct8BMeta Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
456,70 tok/s TG
Prefill 8.966 · TTFT 3.773 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
66gemma-4-E2B-it5BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
451,05 tok/s TG
Prefill 32.005 · TTFT 864 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllmDetails →
67Ornith-1.0-9B9BDeepReinforce Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
430,04 tok/s TG
Prefill 4.493 · TTFT 4.596 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
68Nemotron-3-Nano-30B-A3B30BNVIDIA Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
404,72 tok/s TG
Prefill 4.793 · TTFT 5.226 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMllamacppIQ4_NLDetails →
69gpt-oss-20b20BOpenAI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
403,67 tok/s TG
Prefill 5.727 · TTFT 1.727 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
70Qwen3-Coder-30B-A3B-Instruct30BQwen (Alibaba) Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
399,08 tok/s TG
Prefill 14.492 · TTFT 984 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
71Gemma-4-26B-A4B-it26BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
397,18 tok/s TG
Prefill 3.476 · TTFT 5.916 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMllamacppDetails →
72gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
372,40 tok/s TG
Prefill 1.508 · TTFT 13.832 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMllamacppAWQDetails →
73gemma-4-12B-it12BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
355,09 tok/s TG
Prefill 2.494 · TTFT 6.574 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketUD-Q4_K_XLDetails →
74Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
353,84 tok/s TG
Prefill 924 · TTFT 13.139 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
75gemma-4-E4B-it8BGoogle Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
346,29 tok/s TG
Prefill 16.028 · TTFT 1.776 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 5 5600X 6-Core Processor · 30 GB RAMvLLMvllm bf16Details →
76Ministral-3-14B-Reasoning-251214BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
338,30 tok/s TG
Prefill 5.943 · TTFT 2.299 ms
5×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
77Devstral-Small-2-24B-Instruct-251224BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
331,64 tok/s TG
Prefill 4.743 · TTFT 31.297 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
78Magistral-Small-250924BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
330,94 tok/s TG
Prefill 4.379 · TTFT 7.353 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMvLLMwebsocketAWQDetails →
79Codestral-22B-v0.122BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
325,20 tok/s TG
Prefill 3.507 · TTFT 31.980 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
80Devstral-Small-250724BMistral AI Performance benchmarkTimebench 3 - Kombi (Prefill + Generation)
317,44 tok/s TG
Prefill 6.072 · TTFT 23.382 ms
10×NVIDIA GeForce RTX 3090 TiAMD Ryzen 9 8945HX with Radeon Graphics · 92 GB RAMllama.cppwebsocketQ4_K_MDetails →
Agent & chat rating

Harnessbenchmark

0 veroeffentlichte Harness-Läufe auf NVIDIA GeForce RTX 3090 Ti.

In the leaderboard →
Noch keine Harnessbenchmarks auf dieser Hardware.

← All hardware

Technische Daten

Technische Daten – NVIDIA GeForce RTX 3090 Ti

HerstellerNVIDIA
ArchitekturAmpere
VRAM24 GB
Tensor-Recheneinheiten336
Speicherbandbreite1.008 GB/s
TDP450 W
PCIe4.0
Release03/2022
Bewertung★★★★★5,0
Best generation1.492,8 tok/s
Best prefill32.005 tok/s
Models49
Messläufe (Performance)264