DigitalOcean makes prompt caching automatic on Serverless Inference

DigitalOcean made prompt caching generally available on its Serverless Inference service on October 23, 2026, after a public preview. The company says “cached input is billed at up to 90% off” and that time to first token drops by up to 80%, though it adds that the “actual discount and latency improvement vary by model and prompt.” Caching turns on by itself for eligible requests.

What changed

Serverless Inference runs open models such as Kimi K3 and GLM-5.3 behind an API at https://inference.do-ai.run/v1. With caching, the part of a prompt that repeats across requests, such as system instructions, tool definitions or a long document, is billed at a lower cache-read rate. Uncached tokens still cost the standard input rate.

DigitalOcean says its router now sends each request to the replica that already holds most of its prompt, with no session ID needed. An optional prompt_cache_key field keeps traffic within one account from sharing a cache. Caches are separated between accounts by default. Caching also works with the Responses API, and each response reports cached and uncached token counts. A new control panel view shows hit rate, a per-model breakdown and estimated savings.

The blog post does not say how long a cache lasts.

What it costs

DigitalOcean’s pricing docs list these standard rates per 1M tokens:

Model	Input	Cache read	Output
Kimi K3	$3.00	$0.30	$15.00
GLM-5.3	$1.40	$0.26	$4.40

So a cache hit on Kimi K3 costs a tenth of a fresh input token, while GLM-5.3’s cache read is about 81% below its input rate. DigitalOcean’s post reports cache hit rates of 85-95% on flagship open models in its own October testing, and says results “will vary.”

Who it affects

This matters most for agents and chat apps that resend the same long context on every turn. Since nothing needs to be turned on, existing Serverless Inference users should see the cache-read rate on repeated prompt prefixes. To compare these prices with other APIs, see our AI model price comparison or run the numbers in the token cost calculator.