Serverless GPU pricing: Modal, Replicate and Together AI

Serverless GPUs cost more per hour than a rented GPU but nothing while idle. As of October 28, 2026, an H100 costs $3.9492 an hour on Modal (billed per second, GPU only), $5.49 an hour on Replicate (CPU and RAM included) and $5.49 an hour on a Together AI dedicated endpoint (billed per minute while it runs). Against an always-on H100 at $3.99 an hour, Modal’s GPU charge stays lower at almost any usage. Replicate is cheaper only if the GPU is busy less than about 530 hours a month.

We haven’t run workloads on these platforms ourselves. The figures come from each company’s pricing page and billing docs, read on October 28, 2026, and from the site’s own price data for the always-on GPUs (RunPod checked October 18, Lambda October 25, Hyperstack October 18, 2026). Prices change often, so check the source before you commit.

What does “serverless GPU” mean, and how is it billed?

On a rented GPU (a pod or VM), you pay for every hour the machine is on, busy or not. On a serverless platform, you deploy code or a model, and the platform starts GPU containers when requests arrive and shuts them down when traffic stops. You pay while a container runs.

The three platforms in this guide bill in different ways:

• Modal bills per second for the GPU, plus CPU and memory, while your containers run. Its pricing page says billing covers “application load time” and the time spent processing inputs, and that once an app scales to zero containers, “no charges apply.”
• Replicate bills per second by hardware type. The rate already includes a fixed amount of CPU and RAM. Public models are billed only for processing time. Private models and deployments are billed for all the time an instance is online, including setup and idle time.
• Together AI sells serverless inference per token, and dedicated endpoints per GPU. Its docs say dedicated endpoints “bill per minute by hardware” while the deployment runs, “regardless of model or request volume.”

So “serverless” covers two different deals: pay per second of compute (Modal, Replicate’s public models) or pay per token (Together’s serverless models). A dedicated endpoint on Together or a private deployment on Replicate behaves more like a rented GPU that the platform manages for you.

How much does a serverless GPU cost per hour?

These are the published on-demand rates. Modal and Replicate publish per-second prices; the hourly figures are those times 3,600.

GPU	Modal (GPU only)	Replicate (CPU and RAM included)	Together AI dedicated endpoint
T4 (16 GB)	$0.000164/sec = $0.5904/hr	$0.000225/sec = $0.81/hr	Not listed
L4	$0.000222/sec = $0.7992/hr	Not listed	Not listed
A10	$0.000306/sec = $1.1016/hr	Not listed	Not listed
L40S (48 GB)	$0.000542/sec = $1.9512/hr	$0.000975/sec = $3.51/hr	Not listed
A100 80 GB	$0.000694/sec = $2.4984/hr	$0.0014/sec = $5.04/hr	Not listed
H100 (80 GB)	$0.001097/sec = $3.9492/hr (SXM5)	$0.001525/sec = $5.49/hr	$5.49/hr (HGX H100)
H200	$0.001261/sec = $4.5396/hr (SXM)	$0.001525/sec = $5.49/hr	“Contact us”
B200	$0.001736/sec = $6.2496/hr	Not listed	$8.99/hr (HGX B200)
B300	$0.001972/sec = $7.0992/hr	Not listed	“Contact us”

Notes on the table:

• Modal’s prices are the GPU only. CPU is $0.0000131 per physical core per second (a physical core is 2 vCPUs, with a minimum of 0.125 cores per container), and memory is $0.00000222 per GiB per second. Modal bills the higher of what you request and what you use, with a minimum of 128 MiB of memory.
• Replicate’s prices include a fixed machine. The L40S comes with 10 CPUs and 65 GB of RAM, the H100 with 13 CPUs and 144 GB, the A100 80 GB with 10 CPUs and 144 GB, and the T4 with 4 CPUs and 16 GB. Replicate also sells 2-, 4- and 8-GPU versions at multiples of the single-GPU price (8x H100 is $43.92 an hour).
• Together AI’s H100 and B200 prices are for Dedicated Inference. Together also rents GPU clusters at $3.99 an hour per H100 on demand and $1.99 preemptible, which is closer to a rented VM than to serverless.
• Modal adds multipliers. Choosing a specific region costs 1.15 to 1.75 times the base price, and non-preemptible execution costs 3 times the base price, on all plans.

Matching Modal to Replicate’s bundled CPU and RAM

Modal’s GPU-only price looks much lower, so here is the like-for-like version. Give a Modal L40S container the same 10 vCPUs (5 physical cores) and 65 GiB of memory that Replicate includes:

• CPU: 5 cores × $0.0000131 × 3,600 = $0.2358 an hour
• Memory: 65 GiB × $0.00000222 × 3,600 = $0.5195 an hour
• Total: $1.9512 + $0.2358 + $0.5195 = $2.71 an hour, against $3.51 on Replicate

For the H100 with 13 vCPUs (6.5 cores) and 144 GiB:

• CPU: 6.5 × $0.04716 = $0.3065 an hour
• Memory: 144 × $0.007992 = $1.1508 an hour
• Total: $3.9492 + $0.3065 + $1.1508 = $5.41 an hour, against $5.49 on Replicate

So at matched specs, the H100 costs almost the same on both. The L40S is clearly cheaper on Modal. Replicate states RAM in GB and Modal in GiB, so the match is close, not exact. Most inference jobs need much less CPU and RAM than Replicate’s bundle, and on Modal you pay only for what you request.

What do you pay at 1, 10 and 100 hours a month?

Here’s the monthly bill for GPU time on each platform, compared with an always-on GPU at the providers in our GPU price index. For Modal, “with CPU and RAM” assumes a modest container of 4 physical cores and 32 GiB, which adds $0.4444 an hour ($0.18864 for CPU plus $0.25574 for memory).

L40S (48 GB)

Hours of GPU time a month	Modal, GPU only	Modal, with CPU and RAM	Replicate
1	$1.95	$2.40	$3.51
10	$19.51	$23.96	$35.10
100	$195.12	$239.56	$351.00
730 (always busy)	$1,424.38	$1,748.78	$2,562.30

An always-on L40S on RunPod Secure Cloud is $1.09 an hour, or $1.09 × 730 = $795.70 a month.

H100 (80 GB)

Hours of GPU time a month	Modal, GPU only	Modal, with CPU and RAM	Replicate	Together dedicated
1	$3.95	$4.39	$5.49	$5.49
10	$39.49	$43.94	$54.90	$54.90
100	$394.92	$439.36	$549.00	$549.00
730 (always busy)	$2,882.92	$3,207.32	$4,007.70	$4,007.70

An always-on H100 SXM is $3.99 an hour on RunPod Secure Cloud ($2,912.70 a month) and $4.29 on a single-GPU Lambda instance ($3,131.70 a month). We use 730 hours as an average month (24 × 365 ÷ 12).

For Together’s dedicated endpoint, “hours of GPU time” means hours the endpoint is running, not hours of traffic. It bills by the minute while deployed, so the 1 and 10 hour rows only apply if you start and stop the endpoint yourself.

When does serverless beat an always-on GPU?

The break-even point is the always-on monthly cost divided by the serverless hourly rate. Below that many busy hours, serverless is cheaper.

Comparison	Break-even (busy hours a month)	About this many hours a day
Modal L40S (GPU only) vs RunPod Secure L40S	$795.70 ÷ $1.9512 = 408	13.4
Modal L40S (4 cores, 32 GiB) vs RunPod Secure L40S	$795.70 ÷ $2.3956 = 332	10.9
Replicate L40S vs RunPod Secure L40S	$795.70 ÷ $3.51 = 227	7.5
Modal H100 (GPU only) vs RunPod Secure H100 SXM	$2,912.70 ÷ $3.9492 = 738	More than 24
Modal H100 (4 cores, 32 GiB) vs RunPod Secure H100 SXM	$2,912.70 ÷ $4.3936 = 663	21.8
Replicate H100 vs RunPod Secure H100 SXM	$2,912.70 ÷ $5.49 = 531	17.4

Two things stand out:

1. For the H100, Modal’s per-second GPU rate is below RunPod Secure Cloud’s hourly rate. Even fully busy, Modal’s GPU charge ($2,882.92) is below an always-on RunPod Secure H100 ($2,912.70). CPU and memory push Modal above it only once the GPU is busy more than about 22 hours a day. The always-on pod includes its CPU and RAM in the price, so compare full bills, not GPU lines.
2. Cheaper always-on GPUs move the line a lot. An H100 PCIe on Hyperstack is $2.50 an hour ($1,825 a month), and RunPod Community Cloud lists the H100 SXM at $2.69. PCIe and SXM are different versions of the H100, so that isn’t a like-for-like comparison, but if a PCIe card does your job, the break-even against Modal drops to $1,825 ÷ $3.9492 = 462 hours.

The always-on numbers assume one GPU handles your peak load. If traffic spikes beyond one GPU, an always-on setup has to keep extra GPUs running for the peak, while serverless adds containers only when needed. Modal’s Starter plan allows 10 GPUs at once, and its Team plan 50.

Worked example: an image API with 100 requests a day

Say an app gets 100 requests a day, spread out, and each takes 10 seconds of L40S time on Modal.

If every request finds a warm container: 100 × 10 seconds × 30 days = 30,000 seconds, or 8.33 hours. At $1.9512 an hour, that’s $16.26 a month in GPU time.

If each request starts a fresh container and the default idle window applies: Modal keeps a container alive for 60 seconds after the last input by default, and that time is billed. Each request now costs 10 + 60 = 70 seconds, so 100 × 70 × 30 = 210,000 seconds, or 58.33 hours. That’s $113.82 a month, seven times as much, before load time.

Modal says the idle timeout is configurable. Setting it to 5 seconds brings each request to 15 seconds: 100 × 15 × 30 = 45,000 seconds = 12.5 hours = $24.39 a month. The tradeoff is more cold starts. Load time is billed too, and it depends on your model’s size, which Modal doesn’t publish a figure for.

Compare that with an always-on RunPod Secure L40S at $795.70 a month. For light, uneven traffic, serverless wins by a wide margin. Modal’s Starter plan includes $30 of credits a month, which would cover the first two cases at $16.26 and $24.39 entirely ($30 ÷ $1.9512 = 15.4 hours of L40S GPU time).

On Replicate, the same 8.33 hours of active time on a public model would cost 8.33 × $3.51 = $29.25. If it’s your own private model, Replicate bills setup and idle time too, so the bill depends on how long instances stay online, which Replicate controls by scaling with demand.

What do you get free?

	Free allowance	What it covers
Modal	$30 a month on Starter ($0 plan fee); $100 a month on Team ($250 a month plus compute)	Compute only. Shared Endpoints are billed per token from the first request, including on Starter, since September 1, 2026, per Modal’s pricing page
Replicate	No credit amount on its pricing page; its billing docs say “you can run select models on Replicate for free” before you’re asked to set up billing	Select models
Together AI	No credit amount on its pricing page, which says “Start for free”	A few models are listed as free or $0.00 per token

At Modal’s rates, $30 buys about 50.8 hours on a T4 ($0.5904), 37.5 hours on an L4 ($0.7992), 15.4 hours on an L40S or 7.6 hours on an H100, less once CPU and memory are added. Modal’s Starter plan also includes 3 seats, 100 containers, 1 TiB of free egress a month and 1 TiB of volume storage, with 1-day log retention and no custom domains.

What to watch for

• Idle windows are billed. Modal’s default is 60 seconds after the last input. RunPod Serverless bills idle time until a worker scales down, with a default idle timeout of 5 seconds, and rounds usage up to the nearest second (RunPod docs).
• Cold starts cost money on some platforms. RunPod bills a worker from when it starts “until your handler is ready,” though pulling the image while scaled down isn’t billed. Modal bills application load time. Replicate’s public models bill only processing time but may cold boot when shared demand shifts.
• Private models on Replicate bill all online time. That includes setup and idle time. Deployments can keep a minimum number of instances running; the default minimum is 0.
• Modal’s multipliers. Pinning a region costs 1.15 to 1.75 times base prices, and non-preemptible execution 3 times. The headline rates assume neither.
• Egress and storage. Modal charges $0.04 per GiB of egress beyond the free allowance, and $0.09 per GiB a month for volumes beyond 1 TiB. Together’s shared filesystem is $0.16 per GiB a month.
• Dedicated endpoints don’t scale to zero on their own. Together’s docs say you can stop endpoints “when you don’t need them to reduce costs,” but the overview doesn’t describe automatic idle shutdown. Treat a running endpoint like a rented GPU.
• Prepaid credit. Replicate’s docs describe buying credit first, with usage drawn from the balance, or invoicing in arrears for some accounts.

Per token or per second?

If you only need to call a popular open model, per-token pricing can be cheaper than any GPU. Together AI lists, per million tokens (input / output), Llama 3 8B Instruct Lite at $0.14 / $0.14, Qwen3.5 9B at $0.17 / $0.25 and DeepSeek V4.1 Flash at $0.30 / $1.20. You pay nothing when idle and nothing for load time.

Per-second GPUs make sense when you run your own or fine-tuned model, an image or audio model, or a pipeline that isn’t offered per token. To compare a per-token bill with GPU hours for your traffic, use our AI token cost calculator.

Which should you choose?

• A side project or prototype with light traffic: Modal. The $30 monthly credit often covers it, and the per-second GPU rate is the lowest of the three. Shorten the idle timeout if requests are sparse.
• You want to call an existing model with no setup: Replicate’s public models or Together’s per-token serverless models. You pay only for processing time or tokens.
• Your own model, uneven traffic, under about 10 busy hours a day on an L40S: Modal. Above that, an always-on RunPod L40S at $795.70 a month is cheaper.
• Steady traffic near 24 hours a day: an always-on GPU. Compare RunPod, Lambda and Hyperstack in Modal vs RunPod and our cheapest cloud GPU guide.
• A team that needs a managed endpoint with no infrastructure work: Together’s Dedicated Inference at $5.49 an hour per H100, billed per minute while running. Plan for it to bill around the clock unless you stop it.

For the wider choice between pods, marketplaces and serverless, see how to rent a GPU for AI and the GPU cloud hub.

Sources

• Modal: Pricing, read October 28, 2026
• Replicate: Pricing, read October 28, 2026
• Replicate: Billing docs, read October 28, 2026
• Together AI: Pricing, read October 28, 2026
• Together AI: Dedicated Inference docs, read October 28, 2026
• RunPod: Serverless pricing docs, read October 28, 2026
• AskGPU price data (products.json): RunPod (checked October 18, 2026), Lambda (October 25, 2026), Hyperstack (October 18, 2026), Modal (October 25, 2026; matches the page read today)