How to Use Claude Code with a Local LLM (and When to Keep Claude in the Loop)

You can run Claude Code against a local LLM by pointing ANTHROPIC_BASE_URL at a local server that speaks the Anthropic Messages API. Ollama, llama.cpp, LM Studio and vLLM all do this natively today, so for most setups no proxy is needed. The catch is the model: Claude Code was built around Claude, and a model that fits on one consumer GPU is weaker, slower and shorter on context.

This guide covers the exact setup for each server, checked against each project’s documentation on November 13, 2026, then the limits you will hit and a hybrid pattern that keeps a strong cloud model in the loop.

How does Claude Code connect to a local model?

Claude Code sends every model request to the address in ANTHROPIC_BASE_URL, using the Anthropic Messages format. Any server that answers at /v1/messages in that format can stand in for Anthropic’s API.

The Claude Code gateway docs list the variables:

Two details matter for local use. First, the gateway overview says that setting only the base URL does not replace your claude.ai login: “a saved claude.ai login remains the active credential, so its usage limits and billing apply.” Set a credential variable as well. Second, you can put these values in the env block of ~/.claude/settings.json instead of your shell, and a settings-file value wins over a shell export. Run /status inside Claude Code to see which base URL and credential it is actually using.

Which local servers work with Claude Code?

Four popular local servers speak the Anthropic Messages API natively. Two proxies cover everything else.

Ollama

The shortest path. With Ollama installed, its Claude Code guide gives one command:

ollama launch claude

The manual setup from the same page:

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3.5

Swap qwen3.5 for any local model you have pulled that supports tools. Ollama’s compatibility page says it covers “a subset of the Anthropic Messages API”: tool calling, streaming, thinking and base64 images work, while count_tokens, prompt caching (cache_control), PDFs and tool_choice do not.

llama.cpp (llama-server)

llama.cpp added the Anthropic Messages API to llama-server in January 2026, with Claude Code compatibility as an explicit goal. The announcement shows this pair:

llama-server -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:Q4_K_M
ANTHROPIC_BASE_URL=http://127.0.0.1:8080 claude

Replace the model with one that fits your hardware, and set the context size with -c (for example -c 65536), because by default llama-server loads the model’s full trained context, which may not fit in memory. If you start llama-server with --api-key, set the same value in ANTHROPIC_AUTH_TOKEN.

LM Studio

LM Studio’s integration page starts the server and sets three variables:

lms server start --port 1234
export ANTHROPIC_BASE_URL=http://localhost:1234
export ANTHROPIC_AUTH_TOKEN=lmstudio
export CLAUDE_CODE_ATTRIBUTION_HEADER=0
claude --model openai/gpt-oss-20b

If Require Authentication is on in LM Studio, put your LM Studio API token in ANTHROPIC_AUTH_TOKEN.

vLLM

vLLM’s Claude Code page is the best template for mapping every Claude tier to one local model:

vllm serve openai/gpt-oss-120b --served-model-name my-model \
  --enable-auto-tool-choice --tool-call-parser openai

ANTHROPIC_BASE_URL=http://localhost:8000 \
ANTHROPIC_API_KEY=dummy \
ANTHROPIC_AUTH_TOKEN=dummy \
ANTHROPIC_DEFAULT_OPUS_MODEL=my-model \
ANTHROPIC_DEFAULT_SONNET_MODEL=my-model \
ANTHROPIC_DEFAULT_HAIKU_MODEL=my-model \
claude

The same page explains why CLAUDE_CODE_ATTRIBUTION_HEADER=0 keeps showing up: Claude Code adds a per-request hash to the system prompt, which can defeat prefix caching and slow a local server down a lot. vLLM handles this itself after version 0.17.1. On other servers, setting the variable is cheap insurance.

When you need a proxy

Use LiteLLM or claude-code-router when your server only speaks the OpenAI format, or when you want routing rules, such as sending one model name to a small local model and another to a bigger one, with fallbacks. Both add a moving part that has to keep up with Claude Code’s request format, so skip them when one of the native servers above already covers your case.

What are the limits of running Claude Code on a local LLM?

Expect a working setup, with a weaker model inside it. A local model on consumer hardware falls behind Claude on long agentic coding, context size, tool-calling reliability and speed, and Claude Code does nothing to hide those gaps.

The fixed prompt is large. Claude Code sends a long system prompt and dozens of tool schemas with every request. MCP servers and memory files add more. Anthropic does not publish the size. In a field test against llama-server on a 6 GB laptop, KUN measured the first request at about 15,800 prompt tokens, which took 48.8 seconds to process and used half of a 32K window. Automatic compaction then failed in 2 of 3 runs. Run /context in a session to see your own breakdown. On a model with a 32K window, little is left for your code.

Compaction is tuned for big windows. Claude Code’s docs note that CLAUDE_CODE_AUTO_COMPACT_WINDOW is clamped to at least 100,000 tokens. On a local model with a 64K window, plan to run /compact yourself before the context fills up.

Tool calls fail more often. Every edit, read and shell command is a structured tool call. Smaller and heavily quantized models get the format or the arguments wrong more often over a long session, and Ollama does not support tool_choice, so a client cannot force a specific tool.

It is slow. In our own agent runs (Atomic Agent, not Claude Code) of a 27B model at 4-bit on a 32 GB Mac, two multi-step tasks took 202 and 421 seconds, so a long Claude Code session on that hardware will be slow.

Some features switch off. While a credential variable or a non-Anthropic base URL is set, Remote Control and voice dictation are unavailable, the fast mode and WebFetch domain checks still call api.anthropic.com, and /feedback saves a local file instead of sending it.

None of this makes the setup useless. Bounded jobs (rename across a module, write tests for a known function, explain a file, run and fix a script) suit a local model well. Open-ended work across a large codebase is where it falls apart.

Which local model should you use with Claude Code?

Pick a model that supports tool calling and still leaves room for a 64K context after it loads. On a single GPU with about 24 GB of memory, a 27B-class model at 4-bit is the practical starting point in our own agent tests.

The numbers behind that guidance are in our Qwen 3.8 27B local agent guide. Per the official model card, it is a dense 27B model with a 262,144-token context and an Apache 2.0 license. The 4-bit file is about 17 GB, which leaves headroom on a 24 GB card for the KV cache that Claude Code’s long prompt needs. On Apple Silicon, the same sizes apply to unified memory, and 32 GB gives you slack.

Smaller models in the 4B to 8B range run on 8 to 16 GB, but between the fixed prompt and the shorter context they spend most of their window on Claude Code itself. Thinking modes cost time on every tool call, so if your model has a switch for thinking, try it off for routine edits.

How do you keep Claude in the loop with local models?

Split the work by role. A strong cloud model plans the job and reviews the result, while a local model does the bulk of the reading and writing. The reverse also works: a local model plans, and cloud models write the large outputs.

The split works because the two halves of a coding task need different things. Planning and review depend on judgment, which is where a frontier model earns its price. Execution is mostly mechanical once the brief is clear, which is where a local model is often good enough. Review rounds can still use a lot of cloud tokens, so cap the spend. Done by hand, this means running Claude Code for the plan, then pasting briefs into a second agent on a local model and checking its output yourself.

Claude Code can give plan mode and subagents their own model (the opusplan alias, a subagent’s model field), but every request goes to one base URL, so mixing a cloud Claude with a local model still needs a router that sends each model name to the right backend.

Fusion: the hybrid pattern built into Atomic Agent

Atomic Agent is an open-source, local-first agent (desktop app and terminal), and its Fusion run mode is a ready-made version of this split. An orchestrator model plans, writes a brief for each part and reviews the results. The orchestrator splits the job into up to 16 parts per fan-out and decides how many run at once: cloud workers run several in parallel, local workers share the machine’s request slots. Each worker runs one part in a throwaway session and writes files straight to disk.

During a Fusion turn the orchestrator cannot change your files or run shell commands in your project. It checks the workers’ output with verify.syntax and verify.run, which runs tests or opens the page in a throwaway copy of the folder, and sends weak parts out again with a more specific brief. The Fusion docs cover the settings, and our guide to multi-agent orchestration for coding walks through the pattern with measured runs.

In the terminal, /runmode fusion turns it on. In the desktop app, you pick Fusion in the composer, choose a provider for each role, and use the swap control to flip the direction.

On cost: local tokens count as $0, so only the cloud seat spends tokens, and turnBudgetUsd sets a money cap for one turn. That does not make a Fusion run cheaper than Claude Code by default. In one of our own long runs, the cloud planner’s review rounds pushed the bill above what a cloud-only run had spent by the time we stopped it, so set a budget cap. On privacy, the cloud seat sees your request and the files it works with, so your data does reach that provider. Parallel local workers share one GPU, so more workers on one card means each one runs slower.

What if Claude Code stops mid-task?

Carry the session into another agent instead of starting over. Atomic Agent’s handoff reads the session files Claude Code keeps on your computer and continues the task on a local or cloud model.

In the folder where Claude Code was working, run:

atag handoff --from claude-code

It saves the newest session as a new Atomic Agent chat with its history and sends a message asking the agent to pick up where Claude Code stopped. Tool calls that never got a result are marked so the agent checks them before relying on them. A long session usually will not fit a local model’s window, so handoff keeps the newest steps word for word and turns the older part into a short extract of picked lines, sized to your local model’s context (or 32,768 tokens if none is set). In the desktop app, the same thing is Continue from Claude Code / Codex… in the command palette. The handoff guide lists every flag.

The bottom line

Pointing Claude Code at a local LLM takes three environment variables and one of four servers that already speak its API. Whether it is worth it depends on the job: a 27B-class model on a 24 GB GPU handles bounded edits and scripts, and struggles with long autonomous sessions on a big codebase.

If you want local models doing the bulk of the work without giving up a strong planner, use a setup that keeps the roles separate, such as Fusion, and cap the cloud spend. If your limit runs out halfway through a task, hand the session off and finish it locally. To try either, download Atomic Agent or start with the quickstart.

Commands and variables were checked against the Claude Code, Ollama, llama.cpp, LM Studio, vLLM, LiteLLM and claude-code-router documentation on November 13, 2026.