OpenClaw with local models: Ollama, GPUs and when it pays (2026)
Run OpenClaw on a local model with Ollama, LM Studio or llama.cpp: the memory and context it needs, NVIDIA's NemoClaw, the security trade-off and the cost.
Yes, OpenClaw can run on a local model through Ollama, LM Studio, llama.cpp or vLLM, and its docs call that “the strongest privacy path”. But it’s demanding: the docs tell you to run the largest model you can, warn that local models lack hosted providers’ safety filters, and say smaller models are more open to prompt injection. On cost, a local model beats an API only at high volume or on hardware you already own.
We haven’t run OpenClaw on a local model ourselves. This guide rests on the documentation of OpenClaw, Ollama and NVIDIA, plus GPU and API prices we checked in October 2026; see how we work.
When a local model makes sense
| Your situation | Local model? |
|---|---|
| Your data must not leave your hardware | Yes. The docs call local-only “the strongest privacy path” |
| You already own a GPU with 24 GB or more, or a DGX Spark | Worth trying, at least as a fallback or for the heartbeat |
| You want the smallest bill | Usually no. Budget API models cost about $7 to $21 a month at moderate use |
| The agent reads email or web pages and can run commands | Use the strongest model you can get; small models are easier to hijack |
| A Raspberry Pi or a small VPS | No. The Pi guide says “do not run local LLMs on a Pi” |
What OpenClaw’s docs say about local models
The FAQ is cautious. Asked whether to use a local model, it answers “Usually no. OpenClaw needs large context + strong safety”, and warns that “small cards truncate context”. If you go ahead, run “the largest model build you can locally” (FAQ).
The local models page adds three points worth knowing before you buy anything:
- “Download the largest available model build (avoid “small”/heavily quantized variants)“.
- “A model that loads or answers a short prompt may still fail an agent turn.”
- “Memory requirements depend on the model weights, context size, runtime, and other work on the host.”
The reason is the workload. Every model call carries OpenClaw’s instructions, workspace files and tool definitions, about 17,600 tokens before any conversation in the docs’ own example (see what OpenClaw costs to run). The model must then call tools correctly, turn after turn. That’s harder than chat.
How to connect OpenClaw to a local model
The easiest start is openclaw onboard: its Quick start looks for AI access you already have, including Ollama and LM Studio, and checks it with a real completion (onboarding).
Ollama
Ollama has its own launcher. ollama launch openclaw installs OpenClaw if it’s missing and connects it to Ollama; it needs Ollama 0.17 or later (Ollama blog). Add --model <id> --yes to run it without prompts and pull the model automatically, or --config to change the model without starting the Gateway (Ollama docs). The older ollama launch clawdbot still works.
OLLAMA_CONTEXT_LENGTH=64000 ollama serve # start Ollama with a 64k context (see below)
ollama launch openclaw --model qwen3.6:27b --yes # NVIDIA's pick for 24 GB cardsA few details from OpenClaw’s Ollama provider pages:
- It talks to Ollama’s native
/api/chat. Pointing it at/v1instead “breaks tool calling and models can emit raw tool-call JSON as plain text.” - Automatic discovery needs a model that’s loaded, supports tools and has “at least 16K of context” (setup). Ollama can run on another machine, as long as OpenClaw can reach it.
- Models ending in
:cloudare Ollama’s cloud models, not local ones, and Ollama’s web search for local models needsollama signin. - Ollama’s blog covers macOS and Linux, and Windows through WSL. OpenClaw’s docs warn that on WSL2 GPU setups, autostart can reload the last model at boot and “pin host memory, causing repeated VM restarts”.
LM Studio, llama.cpp and vLLM
- LM Studio is the docs’ pick for a “first-time local setup” with a graphical loader. OpenClaw offers a model automatically “only when LM Studio reports tool training and at least 16K of effective context” (LM Studio). NVIDIA recommends it on Windows.
- llama.cpp gets an OpenClaw-managed server with hardware-aware model selection and verified downloads. Its recipes are text-only, support tools and use a 64K context (llama.cpp); they’re in the table below.
- vLLM, SGLang and MLX are for higher throughput. vLLM must expose
/v1and be started with the right tool parser for the model, such ashermesfor Qwen2.5 orqwen3_xmlfor Qwen3-Coder (vLLM). NVIDIA recommends vLLM on Linux and DGX Spark.
Context length: where local setups break
The sources disagree on how much context an agent needs:
- OpenClaw’s Ollama and LM Studio setup accepts a model with at least 16K of context. Its context check “warns below 20% with an 8k floor” and “hard-blocks below 10% with a 4k floor”; the page doesn’t say what the percentages are measured against.
- Ollama recommends “at least 64k tokens if using local models” for OpenClaw.
- NVIDIA says “32K tokens or more”, and 64K or more if your GPU memory allows (NVIDIA).
The trap is Ollama’s default, which depends on your GPU memory (Ollama):
| GPU memory | Ollama’s default context |
|---|---|
| Under 24 GiB | 4k tokens |
| 24 to 48 GiB | 32k tokens |
| 48 GiB or more | 256k tokens |
On a 16 GB card, then, Ollama gives the model 4k tokens unless you change it, well below what OpenClaw sends. One NVIDIA NemoClaw bug report measured 7,378 tokens for the base prompt and 28 tools, which overflowed Ollama’s default of 4,096; the fix was a 16,384-token context (GitHub). Set OLLAMA_CONTEXT_LENGTH, as above, and budget the memory: Ollama notes that “Setting a larger context length will increase the amount of memory required”.
Which model fits your hardware
Two official lists exist. OpenClaw’s llama.cpp recipes give a download size and a minimum amount of host (system) memory for each model, and put Qwen3.8 27B first when it fits, calling that order “a product default, not a claim that one model wins every task”. CPU-only recommendations stop at Qwen3.5 9B, and the memory figures are “selection floors, not guaranteed fit or speed.” NVIDIA’s guide and build playbook instead sort models by GPU memory. Filter the table by what you have:
Which local model fits your memory
Every figure is from the source named in the row. Host memory is system RAM; GPU memory is the graphics card's VRAM.
| Model | Download | Memory needed | Source |
|---|---|---|---|
| Qwen3.5 4B (Q4_K_M) | About 2.7 GB | 8 GiB host memory | OpenClaw llama.cpp recipe |
| Qwen3.5 9B (Q4_K_M) | About 5.7 GB | 16 GiB host memory; the largest CPU-only pick | OpenClaw llama.cpp recipe |
| Gemma 4 12B IT (Q4_K_M) | About 7.1 GB | 24 GiB host memory plus a GPU | OpenClaw llama.cpp recipe |
| Qwen3.8 27B (UD-Q4_K_M) | About 16.5 GB | 32 GiB host memory plus a GPU; first pick when it fits | OpenClaw llama.cpp recipe |
| Muse Glimmer 30B (Q4_K_M) | About 16.8 GB | 32 GiB host memory plus a GPU | OpenClaw llama.cpp recipe |
| Qwen 3.5 4B | Not stated | 6–8 GB GPU memory | NVIDIA RTX guide |
| Qwen 3.5 9B or Gemma 4 12B | Not stated | 12–16 GB GPU memory | NVIDIA RTX guide |
Qwen 3.6 27B (qwen3.6:27b) | Not stated | 24 GB or more of GPU memory | NVIDIA playbook |
glm-4.7-flash | Not stated | About 25 GB of GPU memory | Ollama blog |
qwen3-coder | Not stated | About 25 GB of GPU memory | Ollama blog |
| Qwen3.6-35B-A3B (NVFP4), via vLLM | Not stated | DGX Spark, 128 GB unified memory | NVIDIA playbook |
Nemotron 3 Super 120B (nemotron-3-super:120b), via Ollama | About 87 GB | DGX Spark | NVIDIA developer blog |
Showing all 12.
The OpenClaw recipes use a 64K context; NVIDIA asks for at least 32K. Memory floors don't guarantee a model fits with that context or runs fast. Sources read October 20, 2026.
OpenClaw’s own ranked list of recommended models includes some of these, such as qwen3.8-27b and nemotron-3-super-120b-a12b, but well below the hosted models at the top, led by Claude Opus 5.5 (recommended-models.json). See the best model for OpenClaw.
NVIDIA: RTX PCs, NemoClaw and DGX Spark
NVIDIA has published the most specific hardware guidance for local OpenClaw that we found. Its March guide is titled “Run OpenClaw For Free On NVIDIA RTX GPUs & DGX Spark”, and its blog calls RTX PCs and DGX Spark “ideal for running personal agents — privately and for free” (NVIDIA blog). Free here means no token bill; the hardware and power are yours to pay for.
NemoClaw was announced at NVIDIA’s GTC conference on March 16, 2026, where Jensen Huang said “OpenClaw is the operating system for personal AI” (NVIDIA). It isn’t a rival to OpenClaw. It’s an open-source stack that installs OpenClaw by default (or Hermes, Deep Agents or Pi) inside NVIDIA’s OpenShell runtime, whose parts are “the sandbox, the policy engine, and the privacy router” (NemoClaw docs, OpenShell), with local Nemotron models. OpenClaw itself can also use OpenShell as a sandbox backend.
- Status: “early preview” since March 16; the latest release-notes entry is v0.0.136, dated October 19, 2026 (release notes). Its GitHub repo calls it “an alpha project”.
- Requirements: at least 4 vCPU, 8 GB of RAM (16 GB recommended) and 20 GB of free disk (40 GB recommended), plus Docker or rootless Podman. NVIDIA lists DGX Spark and Ubuntu 24.04 as validated, and Apple Silicon Macs and WSL2 as working with limitations (prerequisites).
- Install:
curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash(NVIDIA developer blog).
DGX Spark has 128 GB of unified memory in the version NVIDIA’s OpenClaw playbook supports. A 64 GB model goes on sale from October 23 through Acer, ASUS, Dell, Gigabyte, HP and MSI, “starting at $4,999”, but the OpenClaw playbook is only “coming soon” for it (NVIDIA blog). We couldn’t confirm a current US price for the 128 GB model: nvidia.com shows none, and a February post on NVIDIA’s developer forums put the Founders Edition at $4,699, up from $3,999, from a poster whose NVIDIA role isn’t shown (forum).
The security trade-off
The docs are blunt here:
- “Local models do not provide hosted providers’ safety filters” (local models).
- “Smaller/heavily quantized models are more vulnerable to prompt injection” (models FAQ).
- The security guide says “smaller/cheaper models are more susceptible to tool misuse and instruction hijacking under adversarial prompts”. If you use one anyway, it calls for “read-only tools, strong sandboxing, minimal filesystem access, strict allowlists” (prompt injection).
The same page makes one exception: “For chat-only personal assistants with trusted input and no tools, smaller models are usually fine.” It also cites a 2026 study of 272,000 attacks with success rates from 0.5% on Claude Opus 4.5 to 8.5% on Gemini 2.5 Pro among the models listed, so hosted models differ too; it gives no figure for local ones.
NVIDIA’s own advice is to run OpenClaw on “a separate, clean PC with no personal data, or a virtual machine” and to “create dedicated accounts for the agent”, and Ollama’s blog also says to use an isolated environment. A local model fits a private, low-stakes assistant well, and an agent that reads your inbox and runs commands badly. Is OpenClaw safe? covers the wider risks.
Hybrid setups
You don’t have to choose one or the other. OpenClaw’s docs describe several mixes:
- Local fallback. The docs’ example uses Claude Sonnet as the primary, with a local LM Studio model and then Opus as fallbacks, which switch “per turn on provider errors” (local models). Your agent keeps working when the API is down.
- A local heartbeat. The heartbeat docs suggest a cheaper heartbeat model, giving
ollama/llama3.2:1bas the example (heartbeat). At the cost guide’s defaults, the heartbeat is $87.84 of a $145.08 monthly bill on Claude Sonnet 5.5. Heartbeats “run full agent turns”, though, so combine this withisolatedSessionandlightContext, and keep the warnings above in mind. - Local compaction and memory. A cheaper model can summarize long sessions (the docs’ example is
ollama/llama3.1:8b), and memory search can use local embeddings through Ollama or LM Studio instead of OpenAI’s default (compaction, memory). - Split machines. Keep the Gateway on a small, always-on server and the model on a GPU elsewhere, reached over a private network. Where to run OpenClaw covers the server side.
What it costs: local vs API
On hardware you already own, a local model costs only power, which depends on your card. On rented hardware it’s an hourly bill that runs whenever the agent might call the model.
The calculator’s defaults reproduce the moderate user in our cost guide: 20 requests a day at 3 model calls each, 30,000 input tokens per call, plus a heartbeat every 30 minutes around the clock. That’s about 97.2 million input and 1.2 million output tokens a month. In that guide, 60% of request input comes from the prompt cache and none of the heartbeat’s, a third overall, so the cache slider starts at 33%. The GPU runs 24 hours a day, because the heartbeat and scheduled jobs call the model at any hour. GPU prices are from our GPU cloud data for RunPod and Vast.ai; the Vast.ai figures are marketplace medians.
A rented GPU vs an API model for OpenClaw
Monthly cost of an API model against a GPU rented for the hours you choose. The defaults match the moderate OpenClaw user in our cost guide.
GPU cost is hours a day × the hourly price over an average month (730 hours when always on), before storage and data transfer. It assumes the GPU's model is good enough for your agent, which you need to check: open models that fit one GPU are smaller than the top API models. API cost counts cached input at the cache-read price where the model has one.
At these defaults, Claude Sonnet 5.5 costs about $146 a month (the cost guide’s $145.08 uses an exact one-third cache share), against $248.20 for a Community Cloud RTX 4090 running around the clock, 730 hours a month. The GPU wins only above about 166 million input tokens a month at the same mix. Around the clock, the other cards cost $357.70 (Vast.ai RTX 4090), $532.90 (Vast.ai A100), $795.70 (RunPod L40S) and $868.70 (RunPod RTX 5090) a month.
Against Claude Opus 5.5 (about $291 at these settings) or GPT-6 Astra (about $745), the 4090 looks cheaper, but the comparison flatters it: a 24 GB card runs a model like Qwen 3.6 27B, not one of that class. And open models are sold by API too. In our data, GPT-OSS 120B on Together AI costs $0.15 per million input and $0.60 per million output tokens, about $15 a month at this volume.
So rent a GPU for privacy or for very high volume, not to save money at personal scale. More on the options: how to rent a GPU, the cheapest cloud GPUs and the best GPU servers for AI.
FAQ
Can OpenClaw run fully locally?
The model can: the docs support local-only setups and call them “the strongest privacy path”. No page we read describes fully offline use, and some features, such as Ollama’s web search for local models, need an online sign-in.
What’s the best local model for OpenClaw?
The largest one your hardware runs well. OpenClaw’s llama.cpp recipes put Qwen3.8 27B first when it fits (32 GiB of host memory plus a GPU); NVIDIA suggests Qwen 3.6 27B for 24 GB cards and Qwen3.6-35B-A3B on DGX Spark.
How much VRAM does OpenClaw need?
None with an API model. For a local one, NVIDIA’s tiers start at 6–8 GB for Qwen 3.5 4B and reach 24 GB or more for Qwen 3.6 27B. Leave room for context: below 24 GiB, Ollama defaults to 4k tokens, while OpenClaw needs at least 16K.
What’s the difference between NemoClaw and OpenClaw?
OpenClaw is the agent. NemoClaw is NVIDIA’s open-source stack that installs OpenClaw, by default, inside its OpenShell sandbox with local Nemotron models. It’s in early preview.
Is a local model cheaper than an API for OpenClaw?
Not usually on rented hardware. At our cost guide’s default usage, Claude Sonnet 5.5 costs about $146 a month against $248.20 for an RTX 4090 rented around the clock, and budget API models cost far less. It can pay off at high volume, or on a GPU you already own.