Architecture

Atomic Agent is a local-first AI agent that does real work on your machine — browsing, editing files, running shell commands, remembering things across sessions. This page explains what happens between the moment you send a message and the moment the agent replies.

The short version: one model inference produces one step. A step is a batch of tool calls. The agent runs them, compresses the results, updates its state, and asks the model again. That repeats until the agent decides it’s done.

Developer Preview

Atomic Agent is a Developer Preview. Internals, commands, and config keys described here can still change. Pin a release if you need stability.

The core idea

Most of the cost of running an agent is the model. Atomic Agent is built around keeping each model call small and cheap so that long, multi-step sessions stay affordable on a quantized local model.

It does this with a few deliberate choices:

• One inference per step. The model is never asked to “think and act in a loop” inside a single call. Each call emits a JSON array of tool calls and stops. The runtime executes them and calls the model again.
• State lives outside the prompt. Conversation history, memory, browser snapshots, and tasks all live in SQLite, not in an ever-growing context window. The prompt only ever carries a compact slice.
• A stable prompt prefix. The top of the prompt (persona, rules, tool catalog, skills) is byte-identical across a session, so llama-server can reuse its KV-cache and skip recomputing it every step.
• Compressed results. Verbose tool output (test logs, grep dumps, stack traces) is shrunk to a summary before it ever reaches the model.

The result is a tight, observable loop you can trace, replay, and reason about.

The agent loop at a glance

flowchart TD
    User["User message<br/>or scheduled task"] --> Prompt["Build prompt<br/>(stable prefix + bounded tail)"]
    Prompt --> Infer["One LLM inference<br/>(GBNF grammar-constrained)"]
    Infer --> Calls["JSON array of<br/>1..N tool calls"]
    Calls --> Exec["Execute batch<br/>(parallel reads, approval gates)"]
    Exec --> Compress["Compress results"]
    Compress --> State["Update durable state<br/>(session, memory, world)"]
    State --> Terminal{"reply or finish?"}
    Terminal -->|no| Prompt
    Terminal -->|reply| Reply["Turn ends,<br/>session pending"]
    Terminal -->|finish| Done["Session completed"]
    State -.->|async, never awaited| Reflect["Memory reflection<br/>(learns from the turn)"]

Two terms recur:

• A turn is one user message (or task trigger) processed to completion. It can span many steps.
• A step is one model inference plus the tool batch it produced.

Walking through one turn

When you send a message, the runtime calls AgentLoop.runTurn(). Here is what it does, in order.

1. Set up the turn

• Your message is appended to the session transcript.
• Memory context is refreshed: relevant notes, profile facts, lessons, and procedures are recalled and staged into the prompt tail.
• The model profile and tool-calling grammar are synced with llama-server (in case the operator swapped models).
• A fresh ToolLoopTracker is created to watch for the agent getting stuck.

2. Run the step loop

The loop runs from step 0 up to agent.maxSteps (default 25). Each step:

1. Re-probes llama-server’s /props if a model-profile manager is wired (this is how a mid-turn model swap is detected).
2. Calls executeStep, which builds the prompt, runs one inference, parses the tool calls, and executes them.
3. Inspects the outcome:
   • If the model emitted reply, the turn ends (session left pending — the user can continue).
   • If it emitted finish, the session is completed.
   • Otherwise, loop-detection signals are processed and memory context is refreshed for the next step.

3. Close out

When the loop exits, the session status is set (completed, pending, stalled, cancelled, or failed), the turn count is incremented, and a lesson-lifecycle hook records whether the turn succeeded or failed.

Finally, memory reflection fires — asynchronously, never awaited. It runs in the background on a separate llama-server slot so it never disturbs the main agent’s KV-cache or blocks your reply.

Inside a single step

executeStep is where the model meets the tools. The pipeline:

flowchart TD
    Build["buildPrompt()<br/>stable prefix + tail"] --> Slot["acquire KV-cache slot<br/>(prefix hash)"]
    Slot --> Complete["LLM completion<br/>(stream or unary)"]
    Complete --> Reason["extract reasoning<br/>(server channel or inline think)"]
    Reason --> Parse["parse tool calls<br/>(grammar or native_tools)"]
    Parse --> Validate["validate batch"]
    Validate -->|ok| Run["executeBatch()"]
    Validate -->|parse failed| Repair["one-shot repair completion"]
    Repair --> Run
    Run --> Effects["apply state effects<br/>(world snapshot, rare-tool autoload)"]
    Effects --> Record["append transcript turns"]

A few details worth knowing:

• One inference, an array of calls. The grammar constrains the model’s output to a JSON array. A single action is just an array of length one. This removes the model’s temptation to “choose” between solo and batched calls.
• Reasoning is captured two ways. Either the server returns it on a dedicated channel, or the model emits inline <think> blocks that the parser strips out.
• Validation enforces the batching rules (covered below). If it fails for a recoverable reason, the runtime injects a notice and retries once with a repair completion rather than burning a whole step.

Executing the batch

Once the calls are parsed and validated, executeBatch runs them in three phases, grouped by resource class (pure_read, fs_write, browser, terminal, and so on).

1. Sync loop gate. Every non-terminal call passes through the loop detector first, in batch order. Vetoed calls get a synthetic error result instead of running.
2. Grouped async execute. Calls run concurrently across groups. Within a group, pure_read tools fan out in parallel; everything else serializes in batch order to keep writes deterministic.
3. Terminal barrier. If the batch ends with reply or finish, that call runs solo, last, after every other call settles — even if an earlier tool failed. This preserves the model’s intent: “do the work, then close the turn.”

Results are collected back in batch-index order, then each one is compressed.

Batching rules

Two constraints are enforced by the validator:

• Terminal calls (reply, finish) are allowed only at the tail of a multi-call batch.
• Approval-gated tools are forbidden inside batches entirely. If the model emits one in a batch, the runtime trims the batch to just that first approval-gated call, drops the rest, and injects a notice so the model retries them one at a time. This avoids a deadlock, because the approval gate is per-call.

Compressing results

Every tool result is passed through compressToolResult() before it re-enters the conversation. The compressor:

1. Keeps the last N non-blank lines (the tail, default 12).
2. For error results, scans for an error signature (error:, failed:, traceback, exception:) and surfaces the first match.
3. Enforces a maximum summary length (default 400 chars), appending [truncated] if it overflows.

The full output is never sent to the model — only this compact summary. This is what keeps each step under budget even when a shell command spews thousands of lines.

Detecting and breaking loops

Local models can get stuck repeating the same call. ToolLoopTracker watches for this within the turn (it has no memory of prior turns):

• Args-only repeats raise a warning once per bucket of repeats.
• Args + result no-progress streaks trip a critical veto.
• Wandering tools (web.search, web.fetch, browser.*) are watched for a spreading set of distinct args; crossing the escalation threshold jumps straight to the breaker.

The breaker is graceful, not fatal

When the loop breaker trips, it does not mark the session failed. It synthesizes a graceful reply (formatForcedLoopReply()), the transcript shows that reply, and the turn closes normally. The session is left pending, so you can redirect the agent and keep going.

Thresholds are tunable via config / env:

Knob	Env var
Warning threshold	ATOMIC_AGENT_LOOP_WARNING_THRESHOLD
Critical threshold	ATOMIC_AGENT_LOOP_CRITICAL_THRESHOLD
Breaker veto streak	ATOMIC_AGENT_LOOP_BREAKER_VETO_STREAK
History window size	ATOMIC_AGENT_LOOP_HISTORY_SIZE
Max parallel tool calls	ATOMIC_AGENT_MAX_PARALLEL_TOOL_CALLS

How state stays small

The prompt is split into two zones:

• A stable prefix (~5–10k tokens): persona, interaction rules, the tool catalog, the skill index, and environment capabilities. Byte-identical across the session, so it lives in llama-server’s KV-cache and is never recomputed.
• A variable tail (~0–3k tokens per step): loaded skills, profile facts, recalled memory, the latest browser snapshot, and conversation history — ordered slowest-changing first so a hot section at the bottom never invalidates a slow section above it.

Everything durable lives outside the prompt entirely:

• Sessions (the full transcript) in sessions.sqlite.
• Memory (profile facts, notes, lessons, procedures, embeddings, link graph) in memory.sqlite.
• Tasks (deferred and recurring work) in tasks.sqlite.

The prompt only ever sees compact pointers into memory — never the whole store.

Where it all gets wired together

Every entry point — atomic-agent run, atomic-agent serve, atomic-agent tui, and the Tauri sidecar — constructs the runtime exactly once through createAgentRuntime(). That factory probes llama-server, builds the LLM provider registry and tool registry, opens the memory stores, wires the AgentLoop, and returns an AgentRuntime handle.

Turn serialization is the runtime’s job, via the TurnController:

• Per-session FIFO: turns submitted to the same session run sequentially, in order.
• Cross-session parallelism: different sessions never block each other.

graph TD
    Entry["run / serve / tui / sidecar"] --> Factory["createAgentRuntime()"]
    Factory --> Llama["probe llama-server"]
    Factory --> Tools["ToolRegistry"]
    Factory --> Memory["Memory stores (SQLite)"]
    Factory --> Loop["AgentLoop"]
    Factory --> Turn["TurnController"]
    Turn -->|per-session FIFO| Loop
    Loop -->|runTurn| Step["executeStep → executeBatch"]
    Step --> Tools
    Step --> Memory
    Step -.->|events| Trace["TraceRecorder (NDJSON)"]

Try it

The loop is easiest to watch with tracing on. Run a turn, then replay or inspect it:

# Run an interactive session
atomic-agent run --cwd . --max-steps 25

# List recorded traces, then inspect one step-by-step
atomic-agent trace list
atomic-agent trace show <sessionId>

# Re-run the prompts against llama-server to check for drift
atomic-agent trace replay <sessionId>

Prompt system

How the stable prefix and variable tail are budgeted and cached.

Memory

Profile facts, notes, lessons, and procedures — and how recall avoids prompt bloat.

Tools

The eight tool families, resource classes, and the approval gate.

Configuration

Loop thresholds, parallel limits, and the agent.* config block.