Architecture

Atomic Agent is a local-first AI agent that does real work on your machine — browsing, editing files, running shell commands, remembering things across sessions. This page explains what happens between the moment you send a message and the moment the agent replies.

The short version: one model inference produces one step. A step is a batch of tool calls. The agent runs them, compresses the results, updates its state, and asks the model again. That repeats until the agent decides it’s done.

Developer Preview

Atomic Agent is a Developer Preview (this page describes v0.5.0). Internals, commands, and config keys described here can still change. Pin a release if you need stability.

The core idea

Most of the cost of running an agent is the model. Atomic Agent is built around keeping each model call small and cheap so that long, multi-step sessions stay affordable on a quantized local model.

It does this with a few deliberate choices:

• One inference per step. The model is never asked to “think and act in a loop” inside a single call. Each call emits a JSON array of tool calls and stops. The runtime executes them and calls the model again.
• State lives outside the prompt. Conversation history, memory, browser snapshots, and tasks all live in SQLite, not in an ever-growing context window. The prompt only ever carries a compact slice.
• A stable prompt prefix. The top of the prompt (persona, rules, tool catalog, skills) is byte-stable across a session, so llama-server can reuse its KV-cache and skip recomputing it. Only the prefix is held byte-stable — the variable tail below it is expected to change every step.
• Compressed results. Verbose tool output (test logs, grep dumps, stack traces) is shrunk to a summary before it ever reaches the model.

The result is a tight, observable loop you can trace, replay, and reason about.

The agent loop at a glance

flowchart TD
    User["User message<br/>or scheduled task"] --> Prompt["Build prompt<br/>(stable prefix + bounded tail)"]
    Prompt --> Infer["One LLM inference<br/>(GBNF grammar-constrained)"]
    Infer --> Calls["JSON array of<br/>1..N tool calls"]
    Calls --> Exec["Execute batch<br/>(parallel reads, approval gates)"]
    Exec --> Compress["Compress results"]
    Compress --> State["Update durable state<br/>(session, memory, world)"]
    State --> Terminal{"reply or finish?"}
    Terminal -->|no| Prompt
    Terminal -->|reply| Reply["Turn ends,<br/>session pending"]
    Terminal -->|finish| Done["Session completed"]
    State -.->|async, never awaited| Reflect["Memory reflection<br/>(learns from the turn)"]

Two terms recur:

• A turn is one user message (or task trigger) processed to completion. It can span many steps.
• A step is one model inference plus the tool batch it produced.

Walking through one turn

When you send a message, the runtime calls AgentLoop.runTurn(). Here is what it does, in order.

1. Set up the turn

• Your message is appended to the session transcript.
• Memory context is refreshed: relevant notes, profile facts, lessons, and procedures are recalled and staged into the prompt tail.
• The model profile and tool-calling grammar are synced with llama-server (in case the operator swapped models).
• A fresh ToolLoopTracker is created to watch for the agent getting stuck.

2. Run the step loop

The loop runs from step 0 up to agent.maxSteps (default 25). Each step:

1. Re-probes llama-server’s /props if a model-profile manager is wired (this is how a mid-turn model swap is detected).
2. Calls executeStep, which builds the prompt, runs one inference, parses the tool calls, and executes them.
3. Inspects the outcome:
   • If the model emitted reply, the turn ends (session left pending — the user can continue).
   • If it emitted finish, the session is completed.
   • Otherwise, loop-detection signals are processed and memory context is refreshed for the next step.

3. Close out

When the loop exits, the session status is set, the turn count is incremented, and a lesson-lifecycle hook records whether the turn succeeded or failed.

A session is always in one of eight states:

Status	Meaning
pending	Idle and ready — the last turn replied, or none has run yet
running	A turn is executing right now
awaiting_approval	Blocked on you approving an approval-gated call
awaiting_llm	Blocked waiting on the model’s response
completed	The model emitted finish
failed	The turn ended in an unrecoverable error
cancelled	The turn was cancelled
stalled	The turn hit agent.maxSteps without emitting reply or finish

stalled is not an error state: the session is otherwise healthy and your next message can drive a new turn. It exists so operators can tell that condition apart from a clean pending idle.

Finally, memory reflection fires — asynchronously, never awaited. It runs in the background on a separate llama-server slot so it never disturbs the main agent’s KV-cache or blocks your reply.

Inside a single step

executeStep is where the model meets the tools. The pipeline:

flowchart TD
    Build["buildPrompt()<br/>stable prefix + tail"] --> Slot["acquire KV-cache slot<br/>(prefix hash)"]
    Slot --> Complete["LLM completion<br/>(stream or unary)"]
    Complete --> Reason["resolveReasoning()<br/>(server channel or inline think)"]
    Reason --> Parse["parse tool calls<br/>(grammar or native_tools)"]
    Parse --> Validate["validate batch"]
    Validate -->|ok| Run["executeBatch()"]
    Validate -->|parse failed| Repair["one-shot repair completion"]
    Repair --> Run
    Run --> Effects["apply state effects<br/>(world snapshot, rare-tool autoload)"]
    Effects --> Record["append transcript turns"]

A few details worth knowing:

• One inference, an array of calls. The grammar constrains the model’s output to a JSON array. A single action is just an array of length one. This removes the model’s temptation to “choose” between solo and batched calls.
• Reasoning is captured two ways. resolveReasoning() prefers the server’s dedicated reasoning_content channel when it’s available (QwQ, DeepSeek-R1 with --reasoning-format deepseek), and otherwise falls back to extracting inline <think> blocks from the content body for models that stream chain-of-thought inline.
• Validation enforces the batching rules (covered below). If it fails for a recoverable reason, the runtime injects a notice and retries once with a repair completion rather than burning a whole step.

Executing the batch

Once the calls are parsed and validated, executeBatch runs them in three phases, grouped by resource class. There are nine classes:

Class	Covers
pure_read	Side-effect-free reads — the only class that fans out in parallel
fs_write	Declared, but no tool is assigned it today (see below)
browser	browser.* — one Playwright backend per process
memory_write	Mutations to the memory store
tasks_write	Mutations to the task queue
vision	vision.*
approval_gated	Anything that needs your approval, including os.shell.run and os.fs.write
terminal	reply and finish
unknown	Default for unlisted tools; the validator rejects these from any batch (fail-closed)

fs_write is declared but unused

fs_write exists in the type union, but no tool currently maps to it. Mutating filesystem verbs such as os.fs.write are classified approval_gated instead, so they go through the approval gate rather than a write group.

The three phases:

1. Sync loop gate. Every non-terminal call passes through the loop detector first, in batch order. Vetoed calls get a synthetic error result instead of running.
2. Grouped async execute. Groups in distinct resource classes execute concurrently with each other, so the total step wall time is roughly max(group durations). Within a group, pure_read tools fan out in parallel, while the other classes serialize in batch order to keep their writes deterministic.
3. Terminal barrier. If the batch ends with reply or finish, that call runs solo, last, after every other call settles — even if an earlier tool failed. This preserves the model’s intent: “do the work, then close the turn.”

Results are collected back in batch-index order, then each one is compressed.

Batching rules

Two constraints are enforced by the validator:

• Terminal calls (reply, finish) are allowed only at the tail of a multi-call batch.
• Approval-gated tools are forbidden inside batches entirely. If the model emits one in a batch, the runtime trims the batch to just that first approval-gated call, drops the rest, and injects a notice so the model retries them one at a time. This avoids a deadlock, because the approval gate is per-call.

Compressing results

Every tool result is passed through compressToolResult() before it re-enters the conversation. The compressor:

1. Keeps the last N non-blank lines (the tail, default 12).
2. For error results, scans for an error signature (error:, failed:, traceback, exception:) and surfaces the first match.
3. Enforces a maximum summary length (default 400 chars), appending [truncated] if it overflows.

The full output is never sent to the model — only this compact summary. This is what keeps each step under budget even when a shell command spews thousands of lines.

Detecting and breaking loops

Local models can get stuck repeating the same call. ToolLoopTracker watches for this within the turn (it has no memory of prior turns):

• Args-only repeats raise a warning once per bucket of repeats.
• Args + result no-progress streaks trip a critical veto.
• Wandering tools (web.search, web.fetch, browser.*) are watched for a spreading set of distinct args; crossing the escalation threshold jumps straight to the breaker.

The breaker is graceful, not fatal

When the loop breaker trips, it does not mark the session failed. It synthesizes a graceful reply (formatForcedLoopReply()), the transcript shows that reply, and the turn closes normally. The session is left pending, so you can redirect the agent and keep going.

Thresholds are tunable via config / env:

Knob	Env var
Warning threshold	ATOMIC_AGENT_LOOP_WARNING_THRESHOLD
Critical threshold	ATOMIC_AGENT_LOOP_CRITICAL_THRESHOLD
Breaker veto streak	ATOMIC_AGENT_LOOP_BREAKER_VETO_STREAK
History window size	ATOMIC_AGENT_LOOP_HISTORY_SIZE
Max parallel tool calls	ATOMIC_AGENT_MAX_PARALLEL_TOOL_CALLS

How state stays small

The prompt is split into two zones:

• A stable prefix: persona, interaction rules, the tool catalog, the skill index, and environment capabilities. Byte-stable across the session, so it lives in llama-server’s KV-cache and is never recomputed. The runtime measures this scaffolding at roughly 5.2k tokens and budgets 6000 (AGENT_FIXED_PROMPT_TOKENS) as headroom for drift — that constant is only a startup sanity check against the model’s context window, and the real per-build figure comes from checkBudget.
• A variable tail: loaded skills, profile facts, recalled memory, the latest browser snapshot, and conversation history — ordered slowest-changing first so a hot section at the bottom never invalidates a slow section above it. It is sized by the remaining budget rather than a fixed allowance, and it is expected to change every step.

Everything durable lives outside the prompt entirely:

• Sessions (the full transcript) in sessions.sqlite.
• Memory (profile facts, notes, lessons, procedures, embeddings, link graph) in memory.sqlite.
• Tasks (deferred and recurring work) in tasks.sqlite.

The prompt only ever sees compact pointers into memory — never the whole store.

Where it all gets wired together

Every entry point — atomic-agent run, atomic-agent serve, atomic-agent tui, and the Tauri sidecar — constructs the runtime exactly once through createAgentRuntime(). That factory probes llama-server, builds the LLM provider registry and tool registry, opens the memory stores, wires the AgentLoop, and returns an AgentRuntime handle.

Turn serialization is the runtime’s job, via the TurnController:

• Per-session FIFO: turns submitted to the same session run sequentially, in order.
• Cross-session parallelism: different sessions never block each other.

graph TD
    Entry["run / serve / tui / sidecar"] --> Factory["createAgentRuntime()"]
    Factory --> Llama["probe llama-server"]
    Factory --> Tools["ToolRegistry"]
    Factory --> Memory["Memory stores (SQLite)"]
    Factory --> Loop["AgentLoop"]
    Factory --> Turn["TurnController"]
    Turn -->|per-session FIFO| Loop
    Loop -->|runTurn| Step["executeStep → executeBatch"]
    Step --> Tools
    Step --> Memory
    Step -.->|events| Trace["TraceRecorder (NDJSON)"]

Try it

The loop is easiest to watch with tracing on. Run a turn, then replay or inspect it:

# Run an interactive session
atomic-agent run --cwd . --max-steps 25

# List recorded traces, then inspect one step-by-step
atomic-agent trace list
atomic-agent trace show <sessionId>

# Re-run the prompts against llama-server to check for drift
atomic-agent trace replay <sessionId>

Prompt system

How the stable prefix and variable tail are budgeted and cached.

Memory

Profile facts, notes, lessons, and procedures — and how recall avoids prompt bloat.

Tools

The tool namespaces (os, memory, browser, tasks, mcp, skill, vision, tool, plus the bare reply and finish verbs), resource classes, and the approval gate.

Configuration

Loop thresholds, parallel limits, and the agent.* config block.