Tool
VRAM and GPU cost calculator
How much GPU memory an open-weight model needs, how many cards of each type it takes and what a million output tokens costs on your own rented card, once idle time is counted. Qwen3 32B with FP8 weights and a 32,768-token context needs 54.7 GB: 1 × NVIDIA H100 SXM 80 GB at $3.20 an hour (lowest listed on-demand price, Hyperstack; community-cloud and spot offers are excluded). With one request at a time a million output tokens costs $13.37 if the card is busy every hour you pay for, and $89.16 if it is busy 15% of the time.
Calculated Runs in your browser. Your inputs stay on this device. All calculators · Task cost calculator
Result Calculated Qwen3 32B · NVIDIA H100 SXM 80 GB · 8-bit weights · 32,768 tokens · 1 request · data of 9 Nov 2026
- Fits on
- 1 × NVIDIA H100 SXM 80 GB
- Memory needed
- 54.7 GB
- Per hour
- $3.20
- Per 1M output tokens, busy all the time
- $13.37
- Per 1M output tokens, busy 15% of the time
- $89.16
| Where the memory goes | GB |
|---|---|
| Weights | 34.4 |
| KV cache | 8.6 |
| Engine, multi-card buffers and 10% kept free | 11.7 |
| Total needed | 54.7 |
| Memory on 1 card | 80 |
Output speed, all requests together: 66.5 tokens per second Calculated (memory-bandwidth estimate, not a measurement). A card that is busy only part of each paid hour makes every token dearer: the cost per million is divided by the busy share.
- KV cache: Counted on all 64 layers, as the model’s configuration describes them.
- Speed:
- Context:
- Price: $3.20 per card-hour, Hyperstack NVIDIA H100 SXM 80 GB (on-demand), as listed in the United States on ; lowest on-demand price of this GPU type in our data, on-demand offers only. Type your own price above to recalculate.
How the estimate works
- Weights: parameters × bytes per parameter (2 for 16-bit, 1 for 8-bit, 0.5 for 4-bit), plus 5%.
- KV cache: for every token of context and every parallel request the model keeps keys and values: 2 × layers × key-value heads × head size × bytes per value. Models with latent attention (MLA) keep one compressed vector per layer instead: layers × (latent rank + rotary size) × bytes. When every layer uses a sliding window, the context is capped at the window.
- Engine overhead: vLLM 2.2 GB + 8% of the weights + 15% of the KV cache; llama.cpp 0.8 GB + 4% of the weights.
- More than one card: 0.8 GB per card with NVLink, 1.5 GB per card over PCIe, for communication buffers.
- Headroom: the sum is divided by 0.9, so 10% of card memory stays free.
- Cards: the smallest of 1, 2, 4 or 8 cards whose memory covers the total.
- Speed: each output token reads the active weights from memory once, so one request runs at memory bandwidth × cards × efficiency (vLLM 0.65, llama.cpp 0.55) ÷ weight bytes. Parallel requests raise the total by 1 + 0.62 × (requests − 1), never above the number of requests.
- Cost per 1M output tokens: price per card-hour × cards ÷ (tokens per second × 3,600) × 1,000,000 when the card is busy all the time; divided by the busy share of the hour otherwise. Reading the prompt (input tokens) is outside this estimate.
The same formula runs in your browser and in the tables below; the task cost calculator does the same for API prices.
Three open-weight models on each GPU type
Calculated FP8 and BF16 weights, KV cache 16-bit, 32,768-token context, one request at a time, vLLM, every paid hour busy. Per hour: cards × the lowest on-demand price per card-hour of that type (sources in the table of GPU types). Per 1M output tokens with one request is the most expensive way to run a model; more parallel requests in the calculator above lower it, idle hours raise it. Prices as shown in the United States on .
Qwen3 32B · 32.8 billion parameters · weights 34.4 GB (FP8) or 68.8 GB (BF16) · KV cache 8.6 GB at 32,768 tokens
KV cache: Counted on all 64 layers, as the model’s configuration describes them.
| GPU | FP8 cards | FP8 per hour | FP8 per 1M output tokens | BF16 cards | BF16 per hour | BF16 per 1M output tokens |
|---|---|---|---|---|---|---|
| NVIDIA B300 (Blackwell Ultra) | 1 | $7.40 | $12.95 | 1 | $7.40 | $25.90 |
| NVIDIA B200 | 1 | $6.00 | $10.50 | 1 | $6.00 | $21.00 |
| NVIDIA H200 141 GB | 1 | $3.99 | $11.64 | 1 | $3.99 | $23.28 |
| NVIDIA GH200 Grace Hopper 96 GB | 1 | $2.59 | $9.07 | 1 | $2.59 | $18.13 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 1 | $1.85 | $16.22 | 1 | $1.85 | $32.44 |
| NVIDIA H100 SXM 80 GB | 1 | $3.20 | $13.37 | 2 | $6.40 | $26.75 |
| NVIDIA A100 SXM 80 GB | 1 | $1.60 | $10.99 | 2 | $3.20 | $21.97 |
| NVIDIA H100 PCIe 80 GB | 1 | $2.50 | $17.50 | 2 | $5.00 | $35.00 |
| NVIDIA A100 PCIe 80 GB | 1 | $1.35 | $9.77 | 2 | $2.70 | $19.54 |
| NVIDIA L40 | 2 | $2.00 | $16.20 | 4 | $4.00 | $32.41 |
| NVIDIA L40S | 2 | $2.18 | $17.66 | 4 | $4.36 | $35.33 |
| NVIDIA RTX A6000 | 2 | $1.00 | $9.12 | 4 | $2.00 | $18.23 |
| NVIDIA A40 | 2 | $1.18 | $11.87 | 4 | $2.36 | $23.74 |
| NVIDIA A100 PCIe 40 GB | 2 | $3.98 | $17.92 | 4 | $7.96 | $35.83 |
| NVIDIA A100 SXM 40 GB | 2 | $3.98 | $17.92 | 4 | $7.96 | $35.83 |
| NVIDIA GeForce RTX 4090 | 4 | $3.56 | $12.36 | 8 | $7.12 | $24.72 |
| NVIDIA GeForce RTX 3090 | 4 | $2.00 | $7.48 | 8 | $4.00 | $14.96 |
| NVIDIA Quadro RTX 6000 | 4 | $2.76 | $14.38 | 8 | $5.52 | $28.75 |
| NVIDIA A10 | 4 | $5.16 | $30.10 | 8 | $10.32 | $60.20 |
| NVIDIA L4 | 4 | $2.36 | $27.54 | 8 | $4.72 | $55.07 |
| NVIDIA RTX A4000 | 4 | $0.60 | $4.69 | 8 | $1.20 | $9.38 |
Hermes 3 70B Instruct · 70.6 billion parameters · weights 74.1 GB (FP8) or 148.2 GB (BF16) · KV cache 10.7 GB at 32,768 tokens
KV cache: Counted on all 80 layers, as the model’s configuration describes them.
| GPU | FP8 cards | FP8 per hour | FP8 per 1M output tokens | BF16 cards | BF16 per hour | BF16 per 1M output tokens |
|---|---|---|---|---|---|---|
| NVIDIA B300 (Blackwell Ultra) | 1 | $7.40 | $27.89 | 1 | $7.40 | $55.78 |
| NVIDIA B200 | 1 | $6.00 | $22.61 | 2 | $12.00 | $45.23 |
| NVIDIA H200 141 GB | 1 | $3.99 | $25.06 | 2 | $7.98 | $50.13 |
| NVIDIA GH200 Grace Hopper 96 GB | 2 | $5.18 | $19.52 | 4 | $10.36 | $39.05 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 2 | $3.70 | $34.93 | 4 | $7.40 | $69.86 |
| NVIDIA H100 SXM 80 GB | 2 | $6.40 | $28.80 | 4 | $12.80 | $57.60 |
| NVIDIA A100 SXM 80 GB | 2 | $3.20 | $23.66 | 4 | $6.40 | $47.32 |
| NVIDIA H100 PCIe 80 GB | 2 | $5.00 | $37.69 | 4 | $10.00 | $75.38 |
| NVIDIA A100 PCIe 80 GB | 2 | $2.70 | $21.04 | 4 | $5.40 | $42.07 |
| NVIDIA L40 | 4 | $4.00 | $34.90 | 8 | $8.00 | $69.79 |
| NVIDIA L40S | 4 | $4.36 | $38.04 | 8 | $8.72 | $76.08 |
| NVIDIA RTX A6000 | 4 | $2.00 | $19.63 | 8 | $4.00 | $39.26 |
| NVIDIA A40 | 4 | $2.36 | $25.56 | 8 | $4.72 | $51.12 |
| NVIDIA A100 PCIe 40 GB | 4 | $7.96 | $38.59 | 8 | $15.92 | $77.17 |
| NVIDIA A100 SXM 40 GB | 4 | $7.96 | $38.59 | 8 | $15.92 | $77.17 |
| NVIDIA GeForce RTX 4090 | 8 | $7.12 | $26.62 | more than 8 | — | — |
| NVIDIA GeForce RTX 3090 | 8 | $4.00 | $16.11 | more than 8 | — | — |
| NVIDIA Quadro RTX 6000 | 8 | $5.52 | $30.96 | more than 8 | — | — |
| NVIDIA A10 | 8 | $10.32 | $64.83 | more than 8 | — | — |
| NVIDIA L4 | 8 | $4.72 | $59.30 | more than 8 | — | — |
| NVIDIA RTX A4000 | 8 | $1.20 | $10.10 | more than 8 | — | — |
Qwen3 235B A22B Instruct 2507 · 235.1 billion parameters · weights 246.8 GB (FP8) or 493.7 GB (BF16) · KV cache 6.3 GB at 32,768 tokens
KV cache: Counted on all 94 layers, as the model’s configuration describes them.
| GPU | FP8 cards | FP8 per hour | FP8 per 1M output tokens | BF16 cards | BF16 per hour | BF16 per 1M output tokens |
|---|---|---|---|---|---|---|
| NVIDIA B300 (Blackwell Ultra) | 2 | $14.80 | $8.70 | 4 | $29.60 | $17.39 |
| NVIDIA B200 | 2 | $12.00 | $7.05 | 4 | $24.00 | $14.10 |
| NVIDIA H200 141 GB | 4 | $15.96 | $7.82 | 8 | $31.92 | $15.63 |
| NVIDIA GH200 Grace Hopper 96 GB | 4 | $10.36 | $6.09 | 8 | $20.72 | $12.18 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 4 | $7.40 | $10.89 | 8 | $14.80 | $21.78 |
| NVIDIA H100 SXM 80 GB | 4 | $12.80 | $8.98 | 8 | $25.60 | $17.96 |
| NVIDIA A100 SXM 80 GB | 4 | $6.40 | $7.38 | 8 | $12.80 | $14.76 |
| NVIDIA H100 PCIe 80 GB | 4 | $10.00 | $11.75 | 8 | $20.00 | $23.50 |
| NVIDIA A100 PCIe 80 GB | 4 | $5.40 | $6.56 | 8 | $10.80 | $13.12 |
| NVIDIA L40 | 8 | $8.00 | $10.88 | more than 8 | — | — |
| NVIDIA L40S | 8 | $8.72 | $11.86 | more than 8 | — | — |
| NVIDIA RTX A6000 | 8 | $4.00 | $6.12 | more than 8 | — | — |
| NVIDIA A40 | 8 | $4.72 | $7.97 | more than 8 | — | — |
| NVIDIA A100 SXM 40 GB | 8 | $15.92 | $12.03 | more than 8 | — | — |
Does not fit on 8 cards even with FP8 weights: NVIDIA A100 PCIe 40 GB, NVIDIA GeForce RTX 4090, NVIDIA GeForce RTX 3090, NVIDIA Quadro RTX 6000, NVIDIA A10, NVIDIA L4, NVIDIA RTX A4000.
GPU types and prices used
Memory, memory bandwidth and interconnect from NVIDIA’s product pages, datasheets and reference architectures, read (hover a link for the quote). Hourly price: the lowest on-demand price per card among the offers listed for that type; spot and community-cloud offers are left out. Prices as shown in the United States on .
| GPU | Memory | Bandwidth | Interconnect | Specification source | Lowest on-demand price per card-hour |
|---|---|---|---|---|---|
| NVIDIA B300 (Blackwell Ultra) | 288 GB | 8 TB/s | NVLink 5, 1.8 TB/s GPU-to-GPU (HGX B300, 8 SXM GPUs) | memory, bandwidth, interconnect | $7.40 Published Hyperstack NVIDIA B300 288 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA B200 | 180 GB | 8 TB/s | NVLink 5, 1.8 TB/s GPU-to-GPU (HGX B200, 8 SXM GPUs) | memory, bandwidth, interconnect | $6.00 Published Hyperstack NVIDIA B200 192 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA H200 141 GB | 141 GB | 4.8 TB/s | SXM: NVLink 900 GB/s; NVL: 2- or 4-way NVLink bridge 900 GB/s per GPU | memory, bandwidth, interconnect | $3.99 Published Hyperstack NVIDIA H200 SXM 141 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA GH200 Grace Hopper 96 GB | 96 GB | 4 TB/s | NVLink-C2C CPU-to-GPU 900 GB/s; no GPU-to-GPU NVLink in a single GH200 | memory, bandwidth, interconnect, product page | $2.59 Published Lambda NVIDIA GH200 96 GB (on-demand) 1 GPU instance · the United States · · lambda.com |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | 1.597 TB/s | PCIe Gen 5 | memory, bandwidth, interconnect | $1.85 Published Hyperstack NVIDIA RTX Pro 6000 SE 96 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA H100 SXM 80 GB | 80 GB | 3.35 TB/s | NVLink 900 GB/s; PCIe Gen5 128 GB/s | memory, bandwidth, interconnect | $3.20 Published Hyperstack NVIDIA H100 SXM 80 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA A100 SXM 80 GB | 80 GB | 2.039 TB/s | NVLink 600 GB/s; PCIe Gen4 64 GB/s | memory, bandwidth, interconnect | $1.60 Published Hyperstack NVIDIA A100 SXM 80 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA H100 PCIe 80 GB | 80 GB | 2 TB/s | PCIe Gen5; NVLink bridge for two cards, 600 GB/s | memory, bandwidth, interconnect | $2.50 Published Hyperstack NVIDIA H100 80 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA A100 PCIe 80 GB | 80 GB | 1.935 TB/s | PCIe Gen4 64 GB/s; NVLink bridge for 2 GPUs 600 GB/s | memory, bandwidth, interconnect | $1.35 Published Hyperstack NVIDIA A100 80 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA L40 | 48 GB | 0.864 TB/s | PCIe Gen 4 | memory, bandwidth, interconnect | $1.00 Published Hyperstack NVIDIA L40 48 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA L40S | 48 GB | 0.864 TB/s | PCIe Gen4 x16 64 GB/s; no NVLink | memory, bandwidth, interconnect | $1.09 Published Runpod L40S 48 GB (Secure Cloud) · the United States · · runpod.io |
| NVIDIA RTX A6000 | 48 GB | 0.768 TB/s | PCIe 4.0 x16; NVLink bridge for 2 cards 112.5 GB/s | memory, bandwidth, interconnect | $0.50 Published Hyperstack NVIDIA A6000 48 GB (on-demand) · the United States · · hyperstack.cloud |
| NVIDIA A40 | 48 GB | 0.696 TB/s | PCIe Gen4 64 GB/s; NVLink bridge for 2 GPUs 112.5 GB/s | memory, bandwidth, interconnect | $0.59 Published Runpod A40 48 GB (Secure Cloud) · the United States · · runpod.io |
| NVIDIA A100 PCIe 40 GB | 40 GB | 1.555 TB/s | PCIe Gen4 64 GB/s; NVLink bridge for 2 GPUs 600 GB/s | memory, bandwidth, interconnect | $1.99 Published Lambda NVIDIA A100 PCIe 40 GB (on-demand) 1 GPU instance · the United States · · lambda.com |
| NVIDIA A100 SXM 40 GB | 40 GB | 1.555 TB/s | NVLink 600 GB/s; PCIe Gen4 64 GB/s | memory, bandwidth, interconnect | $1.99 Published Lambda NVIDIA A100 SXM 40 GB (on-demand) 1 GPU instance · the United States · · lambda.com |
| NVIDIA GeForce RTX 4090 | 24 GB | 1.008 TB/s | PCIe Gen 4; no NVLink | memory, bandwidth, interconnect | $0.89 Published Runpod RTX 4090 24 GB (Secure Cloud) · the United States · · runpod.io |
| NVIDIA GeForce RTX 3090 | 24 GB | 0.936 TB/s | PCIe Gen 4; NVLink bridge for two cards (bandwidth not stated) | memory, bandwidth, interconnect | $0.50 Published Runpod RTX 3090 24 GB (Secure Cloud) · the United States · · runpod.io |
| NVIDIA Quadro RTX 6000 | 24 GB | 0.672 TB/s | PCI Express 3.0 x16; NVLink bridge for 2 cards 100 GB/s | memory, bandwidth, interconnect | $0.69 Published Lambda NVIDIA Quadro RTX 6000 24 GB (on-demand) 1 GPU instance · the United States · · lambda.com |
| NVIDIA A10 | 24 GB | 0.6 TB/s | PCIe Gen4 64 GB/s | memory, bandwidth, interconnect | $1.29 Published Lambda NVIDIA A10 24 GB (on-demand) 1 GPU instance · the United States · · lambda.com |
| NVIDIA L4 | 24 GB | 0.3 TB/s | PCIe Gen4 x16 64 GB/s | memory, bandwidth, interconnect | $0.59 Published Runpod L4 24 GB (Secure Cloud) · the United States · · runpod.io |
| NVIDIA RTX A4000 | 16 GB | 0.448 TB/s | PCIe 4.0 x16 | memory, bandwidth, interconnect | $0.15 Published Hyperstack NVIDIA A4000 16 GB (on-demand) · the United States · · hyperstack.cloud |
B200: providers list 180 GB or 192 GB per card; the calculation uses 180 GB, the memory per GPU in NVIDIA’s HGX B200 reference architecture. NVIDIA's HGX B300 and DGX B300 product pages list 2.1 TB of GPU memory for eight GPUs; its reference architecture and DGX B300 user guide list 288 GB per GPU (2.3 TB for eight). The calculation uses 288 GB. Hyperstack lists its PCIe cards as “NVIDIA H100” and “NVIDIA A100”; its product pages name them H100 PCIe and A100 80GB PCIe. Its “H100 NVLink” and “A100 NVLink” offers do not state the card type and are left out.
Assumptions
| What | Value | Why |
|---|---|---|
| Price per card-hour | lowest on-demand, or yours | As listed by the provider; storage, network and taxes are extra. Type your own price in the form. |
| Busy share of the hour | 15%, or yours | A rented card is paid for every hour, busy or idle. |
| Weight margin Assumption | 5% | Layers kept in higher precision, embeddings and small tensors. |
| Engine overhead Assumption | vLLM 2.2 GB + 8% + 15%; llama.cpp 0.8 GB + 4% | Activations, graphs and buffers; an estimate, not a measurement. |
| Multi-card buffers Assumption | 0.8 GB or 1.5 GB per card | NVLink boards need less than PCIe cards. |
| Free memory Assumption | 10% | Engines do not fill a card to the last byte. |
| Efficiency Assumption | vLLM 0.65, llama.cpp 0.55 | Share of memory bandwidth that turns into tokens. |
| Parallel requests Assumption | 1 + 0.62 × (requests − 1) | Batching raises total speed, but not in proportion. |
| Cards Assumption | 1, 2, 4 or 8 | Common sizes for splitting a model across cards in one server; speed is taken to grow with the number of cards. Not every provider offers every type in every count. |
| Mixture-of-experts speed Assumption | all parameters | When the number of active parameters is not in the model’s configuration; speed is then a lower bound. |
Model architecture: each model’s config.json and parameter count on Hugging Face, read . The methodology explains how prices are read.