Runpod makes Batch Inference generally available, up to 30% off on-demand

Runpod said on October 28, 2026 that Batch Inference is generally available on its Serverless platform. You send bulk jobs to an existing queue-based endpoint, and they run on separate batch workers that don’t take capacity from live requests. The company’s blog says batch requests cost up to 30% less than on-demand, depending on the GPU, and 20% less on H100, H200, B200 and B300.

What changed

Batch jobs use the model and container already on your Serverless endpoint and the GPU type set on that endpoint. You can’t pick a different GPU per batch or per request. The flow, per Runpod’s docs:

• Create a batch with a JSON array of inputs, then add more requests while it is a draft.
• Finalize it. A draft batch doesn’t run.
• Poll the batch status, then page through the results.

Runpod schedules batch workers by queue urgency and off-peak capacity, so start times aren’t guaranteed and requests don’t run in a set order. Both the blog and the docs say to plan for multi-hour latency.

Limit	Value
Requests per batch	5,000
Active batches per endpoint	10
Queued requests per endpoint	50,000
Size of each upload call	10 MiB

For higher volume, the blog says accounts can run up to 3 million requests a day, while the docs say the queue “can go up to 1,000,000 daily jobs.” Both say to contact support for custom limits.

What it costs

Billing is by compute time per request. Cancelling a batch stops charges for queued requests, but requests already running finish and are billed.

The two official pages don’t fully agree on price. The blog gives the discounts above. The docs page, still labeled beta when we read it, says batch jobs are billed at the same rate as standard Serverless requests and that Enterprise customers get flex worker discounts. Runpod’s pricing page doesn’t mention batch at all. For reference, it lists Serverless H100 at $4.79 an hour and B200 at $8.64.

Runpod shows no cost estimate before you finalize a batch. The blog suggests running a test batch of 100 requests and scaling up from that.

Who it affects

Teams running offline work on Runpod, such as embeddings, document processing or evaluations, can move it off their live endpoints. Anything a user is waiting on should stay on real-time requests.

What to do

Run a small test batch and check that the discount shows up on your bill before moving large jobs. To compare Runpod with other GPU clouds, see our GPU cloud comparison.