Local memory: identity, capture, MCP, providers, recall

Config

merkhain config prints the resolved directories, database, listen address, provider and retention, not every setting. Settings come from defaults < config.json in config_dir < MERKHAIN_* environment variables, key by key: a file key overrides its default (including turning a boolean off), and an environment variable overrides that one key of the file.

MERKHAIN_HOME=/abs/dir roots both (/abs/dir/config, /abs/dir/data) for a separate instance; --config FILE or MERKHAIN_CONFIG picks another config file. Useful keys: local_listen (default 127.0.0.1:7781, must be loopback), log_level, provider, extract.lease, extract.attempt_timeout, extract.request_target_bytes, scheduler (see AI workers), retention.event_payload_days (see Raw event retention), and tracing (opt-in OpenTelemetry export, see Tracing).

Live reload. serve local, serve remote and mcp watch config.json (an editor's save-by-rename and a swapped symlink included) and reload it on change and on SIGHUP; hooks and other commands read it on every run. A reload is applied whole or not at all: a file that does not parse or validate is logged at warn level and the previous configuration stays active, and so does one that changes a startup-only key. Those are data_dir, local_listen, remote_listen, remote_trusted_proxy, remote_public_origin, switching relay TLS on or off, database.*, tracing.* except tracing.sql, metrics.*, scheduler.* and relay_quota.*; the log names them (restart_to_apply=[...], never their values) and they need a restart. Everything else applies live: log_level, provider (jobs already running finish with the provider they started with), extract.* (from the next job), recall_comparison and embeddings (the comparison worker restarts with the new judge and providers), recall_search (the next request uses the new backend; the service starts indexing its provider), prompt_context, search_context, retention, tracing.sql and the relay's certificate files. merkhain status reports a rejected reload as config_reload_failed with config.last_reload_error (invalid, restart_required or apply_failed); while the file does not load, it prints the error and still asks the running service (with the defaults and environment, so set MERKHAIN_LOCAL_LISTEN if local_listen is only in the file). Check an edit before saving it in place with merkhain config validate --config FILE.

Editing Memory & recall settings from the browser

/ui/settings/injection (Settings → Memory & recall) keeps a draft across internal app navigation in the authenticated session. Review changes before Save; Keep editing returns to the draft and Discard returns to the latest confirmed values. Drafts never enter URLs or history/browser storage. Reload/close warns while dirty, including while another page is visible; sign-out clears the draft. A lost response requires a re-read and deliberate reconciliation before retrying.

serve local lets the signed-in browser UI change a fixed list of live keys of the file it reads (the default config.json, or the file --config/MERKHAIN_CONFIG selected; the UI says which kind, never the path): provider.kind, the model of each provider kind (provider.openai.model, provider.codexcli.model, provider.claudecli.model), a loopback provider.openai.base_url, prompt_context.{enabled,planner,max_items,max_bytes,deadline,min_relevance}, search_context.{enabled,max_items,max_turn_attempts,max_bytes,max_turn_bytes,deadline} and recall_comparison.{disabled,embeddings_disabled,judge.provider,judge.daily_budget} (0 is unlimited) and agent_feedback.disabled (new agent sessions). Each key shows its resolved value, the file's value, the default and where it comes from; resetting a key removes it from the file so the default applies. Binaries, credentials, allow_remote, startup-only keys and everything else stay file-only: a remote OpenAI endpoint configured in the file is shown read-only and kept, and an endpoint's credentials, query or fragment are never shown (the UI marks the value redacted) and cannot be set from the browser, whichever provider is selected. An endpoint value that is not an absolute http(s) URL with a host, such as a bare key, is not shown at all. Switching the provider sends the backlog too: pending, queued, retried and new extraction jobs use the newly selected provider; only attempts already running finish with the previous one.

• Environment locks. A key an environment variable of the service sets is locked, and only that key: MERKHAIN_PROMPT_CONTEXT=off locks prompt_context.enabled, another value also planner; MERKHAIN_RECALL_COMPARISON=off locks disabled (off unless set to false), on/no-embeddings also embeddings_disabled; MERKHAIN_AGENT_FEEDBACK locks agent_feedback.disabled; MERKHAIN_SEARCH_CONTEXT locks search_context.enabled; MERKHAIN_EXTRACT_PROVIDER locks provider.kind. Hooks run with the agent's environment, which the service cannot see and may override the file differently.
• Saving. A save is checked as a whole configuration first (an invalid search_context change is rejected rather than ignored as on load). It is based on the revision the page read: if the file changed since, including a retargeted symlink, nothing is saved and the page gets the current values. Only the changed keys are edited; other keys, formatting aside, stay as they are. The file is replaced atomically (a symlink keeps pointing at its target, which keeps its permissions; a new file is owner-only). A file that does not load, a missing explicit file, a read-only file or a file with pending startup-only changes cannot be edited until it is fixed or the service restarted.
• Outcome. The service reloads right away and reports what was written, whether the new configuration was published, and which components still run an earlier one: applied, unchanged, saved_with_apply_errors (for example a provider that cannot be started: extraction pauses, without falling back to the previous provider, while the other settings apply) or saved_not_published. A component's failure is reported until it applies a later configuration; a save that changes nothing does not clear it. "Applied" means the provider or judge was built and installed, not that its CLI works; failures while running show under Processing and in merkhain status. The comparison worker restarts in the background.
• Turning search_context.enabled on or off needs merkhain claude install and merkhain codex install again; the UI says so and never installs hooks itself.

Raw event retention

Raw events are stored in plaintext and kept forever by default. To stop them growing without bound, tell serve local to remove an event's content a number of days after the extraction job covering it succeeded:

{"retention": {"event_payload_days": 90}}

MERKHAIN_RETENTION_EVENT_PAYLOAD_DAYS overrides the file. A change applies to a running service, which prunes what has expired on its next pass. Setting 0 later stops pruning but does not bring back content already removed.

What is removed: the payload of each event and, for prompts, the copy of their text in the prompt search index. What stays: the event row (ID, session, source key, kind, time), which capture needs to recognise a replayed transcript, so a pruned event is never captured again, and these fields of the payload, which hold no captured content: author (who wrote a prompt), sidechain and agent_id (a subagent's message or turn end), phase (a Codex message's phase) and prompt_id or turn_id (the agent's turn identity, which links a prompt to the memories it received). Memories are not touched. session_start payloads also stay whole, because later jobs and prompt listings read from them who opened the session.

What is never pruned: events of a job that is pending, leased or failed (retry it or leave it, its events wait), and events no job covers yet. A session whose extraction cannot run, for example without a provider, keeps its payloads.

What evidence you lose for a pruned event:

• memory_get with include_payloads returns the source as local with payload_pruned: true and no payload. The memory itself, its title, body and citations are unchanged.
• The UI shows "removed by retention" for events, prompts and replies. Pruned prompts stay listed with their author but no longer match a search.
• merkhain rules explain --event reports that there is nothing to explain, and rules replay counts these events as pruned instead of judging them.
• merkhain rules cleanup counts them as missing evidence. A memory whose remaining evidence is only chatter is unavailable, never chatter, so retention can never cause a retraction. New rules cannot be evaluated on them.

Disk space. Pruning stops growth: SQLite reuses the freed pages for new data, but the file does not shrink. merkhain never runs VACUUM on its own, because it rewrites the whole file and holds the writer lock throughout, which would make hooks fail. To shrink the file, stop serve local, make sure no agent session is running, take a backup, then run sqlite3 "<database_path>" "VACUUM". Until freed pages are reused or the file is vacuumed, removed text can remain in the file's free pages and the write-ahead log, and search terms in old index segments until they merge, so treat retention as a limit on growth and exposure, not as secure erasure. Backups, including the one taken before a migration, keep whatever they held. On PostgreSQL, autovacuum reclaims the space for reuse.

Load. The worker (event_retention in merkhain status) runs hourly under serve local's supervisor, so a panic restarts it without affecting capture or recall. It reads and parses payloads before taking the writer lock, then writes batches of at most 100 events or 1 MiB (one larger payload goes alone), leaving the writer lock free between batches so hooks are not starved. It resumes after a crash or restart from what is already committed.

Identity

merkhain identity init --name laptop                 # native store: macOS Keychain, Secret Service, Windows Credential Manager
merkhain identity init --name ci-box --key-store file  # passphrase-encrypted file in data_dir/keys
merkhain identity show                               # public record only
merkhain identity check                              # unlocks and validates the keys

An identity is needed for sync, pairing, teams and recovery, not for local capture/recall. There is no silent fallback between key stores. A new key-file or recovery passphrase needs at least 12 bytes and must not be guessable (zxcvbn strength 3 of 4): a stolen key file can be attacked offline. Use four or more random words or a generated passphrase. Passphrases of existing files keep working. Headless use of the file store: MERKHAIN_KEY_PASSPHRASE_FILE=/path/to/0600-file (never pass a passphrase in argv or a plain environment value). Restart serve local after creating the identity.

Capture

Claude Code (verified 2.1.308):

merkhain claude install                                    # --scope user (default): ~/.claude/settings.json
merkhain claude install --scope project --project-dir .    # .claude/settings.json (shared with the repo)
merkhain claude install --scope local --project-dir .      # .claude/settings.local.json

The project and local scopes stay inside the project directory, since a repository's .claude directory may come from someone else. Install and uninstall refuse, before changing either file, a settings file or skill that is a symlink, a hard link (checked on Linux and macOS, not on Windows) or not a regular file, and a path through a symlink that leaves the project or is absolute (a relative symlink inside the project is fine). The project directory must be readable. To edit such a file anyway, name it with --settings FILE; a refused skill path also blocks the settings change, so pass --no-skill to install or uninstall the hooks alone.

Install adds one merkhain hook per event (SessionStart, UserPromptSubmit, PostToolUse, PostToolUseFailure, Stop, SubagentStop, PreCompact, SessionEnd) and keeps every other setting and hook; re-running it is idempotent; claude uninstall removes only merkhain's entries. Hooks print nothing to the model unless prompt context is enabled (see Prompt context) or search context is enabled (see Search context), and never block a prompt. Sessions whose hooks did not run: merkhain claude import ~/.claude/projects/<dir>/<session>.jsonl. The browser UI's Settings → Agents shows, for Claude Code (user scope), Codex, OpenCode and pi, whether the hooks, plugin or extension and the agent skill are installed and current for this executable and --config, with the command to fix them (see Agent status).

Keeping hooks up to date

serve local keeps the hooks and skill you installed current, when it starts, when search_context.enabled changes and whenever Settings → Agent integrations is opened, as merkhain claude|codex install would:

• hooks that run this executable but differ from what it installs (a new version's events or timeouts, search context switched) are rewritten, keeping the path they run it under;
• hooks whose executable no longer exists (a Homebrew upgrade removes the previous version's directory) are pointed at the running executable;
• a Merkhain-written agent skill from another version is replaced when the agent's hooks run this executable.

It never installs hooks or a skill that are missing, replaces a skill merkhain did not write, or takes over hooks that run another existing executable (a second installation or a development build: run its install command, or use Update in Settings) or that pass a --config. Only the instance at the default locations does this: a service started with --config or a MERKHAIN_HOME, MERKHAIN_CONFIG_DIR, MERKHAIN_DATA_DIR or MERKHAIN_CONFIG override changes nothing on its own. When the executable was replaced since the service started, it changes nothing until restarted, since it may be older than the installed version; Settings asks for the restart. Changed Codex hooks must be trusted again in codex → /hooks, and Settings says so.

Codex (verified 0.163.0):

merkhain codex install            # appends merkhain groups to $CODEX_HOME/hooks.json (default ~/.codex), keeps others
codex                              # then open /hooks once and trust the merkhain hooks
merkhain codex import             # catch up from rollout files: untrusted hooks, downtime, ephemeral history

Codex does not run new or changed hooks until you trust them; merkhain does not write Codex trust state.

OpenCode (verified 2.2.5; the plugin also implements the 1.x plugin API):

merkhain opencode install         # writes $XDG_CONFIG_HOME/opencode/plugins/merkhain.js (default ~/.config/opencode)

OpenCode keeps sessions in a private database, so there is no transcript to replay: the plugin collects each turn's prompt, assistant text and tool results from the plugin API and hands them to merkhain opencode hook when the turn ends (and when OpenCode shuts down). Turns that ran while the plugin was not installed are not captured. The plugin also registers the merkhain MCP server, so no opencode.json edit is needed. Restart OpenCode (its background service too) after installing.

pi (verified 1.4.0):

merkhain pi install               # writes $PI_CODING_AGENT_DIR/extensions/merkhain.ts (default ~/.pi/agent)
merkhain pi import                # catch up from session files: downtime, sessions before install

The extension reports session start, each agent run and shutdown to merkhain pi hook, which reads pi's session file (<agent-dir>/sessions, or $PI_CODING_AGENT_SESSION_DIR); it also registers the merkhain MCP server. Run /reload or restart pi after installing. pi import is idempotent.

All install commands also add the merkhain agent skill (see below): Claude Code: skills/merkhain/SKILL.md next to the settings file of the chosen scope (~/.claude/skills, or <project>/.claude/skills for --scope project|local); Codex, OpenCode and pi: ~/.agents/skills/merkhain (one shared copy; --skills-dir DIR to change it). Pass --no-skill to skip it. A SKILL.md you wrote yourself at that path is never overwritten or removed. Uninstall removes the skill with the hooks (pass --no-skill to keep the shared copy while another agent uses it). codex import is idempotent — schedule it if you prefer not to use hooks.

codex import and pi import read regular files only: a link or a named pipe in the selected directory is not imported, and is listed in the output as skipped. Every link is listed, since it may lead to transcripts; one that does not is harmless. With --since, a recent file that cannot be opened is reported too, not left out; an old one needs no permission to be left out. pi import refuses a --sessions-dir that is itself a link. An imported transcript stays bound to that directory, as resolved at import, when it is read again later. Codex and pi sources stored before this binding existed, and imported sources whose selected directory now leads elsewhere, are paused until you run the import again (codex import --archived also covers archived rollouts); hooks do not resume them. An import reads every transcript in the directory it has not seen, which can start a large extraction backlog; to unpause only tracked sources, name each with merkhain capture reconcile --format FORMAT --path FILE and its agent and session. merkhain status lists them as capture_import_rebind_required. A paused session ends once the session has had no hook call and no stored activity for the idle time, and none of its copies can be read: a readable copy's own end check decides. If you move or delete an imported tree, its old sources report as missing and stop warning; run the import from the new location to capture from there. An import also settles a paused source whose file no longer exists below the directory it was given, if its old directory no longer leads to it. A hook that names a link stores nothing from it and shows as paused, even when the file it leads to is already read under its own path. A hard link to a file elsewhere is a regular file and is imported; only import trees you trust not to contain one. Every transcript read refuses a named pipe or device. Hooks and the background reconcile never follow a final symbolic link. Naming a link with capture reconcile --path or claude import stores the file it leads to, which later reads use. When you name a transcript, pass the agent's own session ID and its format (--format codex-rollout, pi-session or claude): without --format it is read as generic records. A source already stored at a link stays paused while the link is there, and is read once each time you name the link. To keep reading, name the file it leads to, which is not duplicated; the link's own entry clears only when you remove the link or put a regular file there.

Agent status

GET /ui/api/v1/agents (Settings → Agents) runs each install command's own logic as a dry run and writes nothing. Installed means the Claude Code and Codex hook entries, or the generated OpenCode plugin and pi extension byte for byte, are what this version writes for this executable and the same --config, and that executable exists. The hooks or plugin may name the same file under another path, such as a package manager's symlink; they count as installed, although the install command would rewrite that path to its own. Otherwise the hooks or plugin are missing, outdated (another executable, an executable that no longer exists even if the file is otherwise current, or hooks or a plugin file that differ from what this version writes), unmanaged (a plugin file without the managed-by: merkhain marker, which install refuses to replace: move it away) or unreadable (the error says why; the install command is still shown, and it fails on the same read error unless the file is merkhain's with damaged settings). The status follows the service's environment (HOME, CLAUDE_CONFIG_DIR, CODEX_HOME, XDG_CONFIG_HOME, PI_CODING_AGENT_DIR), which a background service may not share with your shell. Hooks installed with --scope project|local, --settings, --codex-home, --config-dir or --agent-dir elsewhere, and a skill installed with --skills-dir, are not checked and show as missing at the checked location. The shared ~/.agents/skills/merkhain skill shows the same state for Codex, OpenCode and pi. An agent counts as capturing when its hooks or plugin can run (installed, or outdated with an executable that still exists).

Raw events and transcripts stay on this device. The local service also replays known transcripts (merkhain repair reconcile does it on demand), so a missed hook is recovered with no duplicates.

Import existing memories

Memory already distilled by other tools can be copied into the local store once, so memory_recall sees it alongside what merkhain captures itself:

merkhain import claude-mem --dry-run        # ~/.claude-mem/claude-mem.db (claude-mem plugin); --db PATH overrides
merkhain import claude-mem                  # observations → decision/lesson/attempt, session summaries → summary (+ todo); merged keys included
merkhain import claude                      # ~/.claude/projects/*/memory/*.md (Claude Code auto-memory; MEMORY.md index skipped)
merkhain import codex                       # $CODEX_HOME/memories_*.sqlite and memories/rollout_summaries/

Imports are idempotent: every record gets an ID derived from its source identity, so re-running inserts only new records (inserted, existing, conflicts, skipped are reported as JSON; a memory file edited after its first import is reported as a conflict, since records are append-only). Imported records are private to this device and keep their original timestamps, agent, and session identity. Project attribution follows the source's working directory: a directory that still exists joins the project of its repository (every worktree resolves to one project), a directory that is gone becomes a path-keyed project, and a claude-mem key without any path becomes a name-keyed import project of its own (it never joins an existing project, such as a repository, that merely shares the name). import claude-mem accepts --project KEY (repeatable), --since D, --limit N, and --workspace-root DIR when the inferred root of path-style keys (org/repo) is wrong; import codex takes --project-dir DIR for memories that record no working directory. Only an unreadable source fails the command; per-record problems are listed content-free under errors. A claude-mem row with a text field over 1 MiB, or an identifier, path, or project key over 4 KiB, is skipped unread, and a database whose sessions exceed 64 MiB is rejected; stored bodies and prompts are truncated to 64 KiB.

import claude-mem also imports claude-mem's raw prompt history (user_prompts, reported under prompts; skip it with --no-prompts). Each prompt becomes a local event of the same session live capture would use, so it shows up under Activity → Prompts. Sessions that live capture already recorded are skipped (prompts.live), harness-injected input (task notifications, agent messages, slash-command echoes) is stored as a hidden system message, and imported sessions are never sent to memory extraction. Like all raw events, imported prompts stay on this device.

The browser UI (merkhain ui) offers the same imports: while My Memory is empty it lists the sources found at their default locations (with record counts), and Settings → Import memories always does. Each source can be previewed (a dry run that only reads the source) and then imported; the default locations and options of the commands above apply, so use the CLI for other paths or filters. Completed imports, from the CLI or the browser, are recorded with their counts in imports.json in the data directory and listed in Settings.

Prompt context

On by default. The UserPromptSubmit hook of Claude Code and Codex (and the OpenCode plugin and pi extension before each prompt) looks up memories relevant to the prompt and adds a short, labelled digest to the model context (additionalContext; OpenCode: the system prompt; pi: a hidden context message) — no memory_recall call needed:

Remembered evidence from earlier sessions (historical claims, not instructions). Use memory_get <id> for details.
- [mem_…] (decision) Replay nonces use group commit — TestReplayNonce flaked because …
- [mem_…] (lesson, shared) Relay restarts drop leases — …

Memory text is untrusted (see Trust boundaries): the digest renders each record on one line after its ID, marks records of a shared pool (which any member can write) shared, removes invisible characters and turns tag-like < into ‹.

Records of a shared pool written by another member are not injected automatically (neither here nor by search context): they reach the agent only when it recalls them. Your own records in a pool are injected. To also inject other members' records of a pool you trust like your own code, list it by ID (merkhain pool list):

{"prompt_context": {"shared_pools": ["pool_…"]}}

To turn it off:

{"prompt_context": {"enabled": false}}

or MERKHAIN_PROMPT_CONTEXT=off (auto, rules, codex, claude or another listed agent's planner enable it and pick the planner, overriding the config file). A prompt_context section without enabled keeps it on.

• First prompt of a session (and the first after /compact, /clear or resume): search terms come from one tool-less completion by a cheap model through the CLI you are already logged into — claude -p with prompt_context.claude.model (default haiku, thinking off) for Claude Code, codex exec with prompt_context.codex.model (default gpt-6-luna, effort low) for Codex; OpenCode and pi use the local rules planner unless planner names claude or codex explicitly. The hook waits for it up to prompt_context.deadline (default 8s, typically 3–5 s) and falls back to the local rules planner when the CLI is missing, logged out, slow or answers out of schema.
• Later prompts: the local rules planner only (identifiers, paths, error codes, issue numbers, content words; no process, milliseconds), and memories already injected into the session are never repeated.
• A memory is injected only if it matches an identifier from the prompt or at least two other terms; trivial prompts ("thanks", "continue") inject nothing. At most max_items (5) items and max_bytes (2 KiB); min_relevance (default 0, off) drops weaker bm25 matches.
• The prompt goes only to the agent CLI (and account) the session already uses; planner: "rules" sends it nowhere. Planner failures are reported on the hook's stderr.

Re-run merkhain codex install after upgrading: the Codex UserPromptSubmit hook timeout grows to 30 s to leave room for the planner.

Orchestration prompts (event filter, on by default)

Orchestrators such as Orca type their own text into an agent: a worker introduction before each assigned task, "You have 1 orchestration message" notices, and envelopes with workers' reports. event_filter.mode (or MERKHAIN_EVENT_FILTER) decides what automatic recall and memory extraction do with them:

• on (default): a recognized worker introduction is left out of the search terms, so recall searches for the assigned task itself. Inbox notices, acknowledgements and workers' reports start no automatic recall. Extraction is given the assigned task and reports without the introduction and envelope, and no inbox notices or acknowledgements; a job that holds nothing else completes without a model call. Prompts in any other format — including your own question that quotes such a notice — are handled as before.
• preview: prompts are handled as with off; the filter's decision is counted (prompt_filter_preview_* in usage_health) and recorded with the recall comparison, and each extraction job records what the filter would have left out (event_filter in merkhain extract jobs output).
• off: prompts are handled as described above, and extraction sees every captured event (the behavior before the filter was on by default).

The filter never changes what is captured, and it calls no model. Rules can be disabled or added in event-filter-rules.json next to config.json; merkhain rules check|explain|replay validate the rules, show what they decide about a prompt or event, and replay them on recorded sessions without changing anything. See Event filter.

Agent protocol events (A2A, AG-UI)

An application that talks to agents over A2A 1.0 can have merkhain remember those exchanges. merkhain does not run an A2A server, discover agents or send tasks; the application forwards what it observed — the requests it sent as well as the responses and stream events it received — with the project directory and a session of its choosing:

merkhain capture protocol < request.json      # or POST /v1/capture/protocol on the local service

{"agent": "my-app", "session": "ctx-1", "dir": "/path/to/project",
 "producer": "research-agent", "protocol": "a2a", "version": "1.0",
 "events": [
   {"stream": "req-1", "seq": 0, "direction": "request",  "data": {"jsonrpc": "2.0", "id": 1, "method": "SendStreamingMessage", "params": {"message": {…}}}},
   {"stream": "req-1", "seq": 1, "direction": "response", "data": {"jsonrpc": "2.0", "id": 1, "result": {"artifactUpdate": {…}}}}
 ]}

Every event needs an id (for example the SSE event ID) or a stream and seq (its position in the stream): the same delivery forwarded twice is stored once, while two identical chunks at different positions are both kept. author_kind (person, agent, automated) may state who wrote the client's messages; A2A's user role alone is not taken to mean a person. Recorded exchanges (one JSON-RPC message or A2A object per line, or records with id/stream/seq/direction/data) are imported the same way, and importing a file again adds nothing:

merkhain capture import --protocol a2a --producer research-agent --agent my-app --session ctx-1 \
  --dir /path/to/project exchange.jsonl

With event_filter.mode on (the default), extraction receives the messages, the assembled artifacts and the explanations attached to status updates (a blocker, a question that needs input), but not the bare state changes (working, completed) or task lookups. With the mode off every captured event is sent as it is.

AG-UI

Applications with an AG-UI frontend (protocol 1.0) forward the same way with "protocol": "ag-ui": the RunAgentInput they sent (direction request) and the events of the run's stream. AG-UI events do not name their thread, so use one session per AG-UI thread (for example "session": "<threadId>") and give each run's events one stream (for example the run ID) with their position as seq:

{"agent": "my-app", "session": "thread-1", "dir": "/path/to/project",
 "producer": "support-agent", "protocol": "ag-ui", "version": "1.0",
 "events": [
   {"stream": "run-1", "seq": 0, "direction": "request", "data": {"threadId": "thread-1", "runId": "run-1", "messages": […]}},
   {"stream": "run-1", "seq": 1, "data": {"type": "TEXT_MESSAGE_CONTENT", "messageId": "msg-2", "delta": "Checking "}}
 ]}

A recording with one AG-UI event or RunAgentInput per line is imported with --protocol ag-ui:

merkhain capture import --protocol ag-ui --producer support-agent --agent my-app --session thread-1 \
  --dir /path/to/project run.jsonl

With event_filter.mode on (the default), extraction receives each streamed message and tool call assembled once (a tool call is the call and its arguments; it does not say the tool succeeded), tool results, the question of a run that stopped for approval or input, and run error messages. Messages that later snapshots or run inputs repeat unchanged are not extracted again; a changed message is. Run, step and subagent markers, shared state and activity updates and the tools offered stay in history only. CUSTOM and RAW events are kept as they are.

Search context (experimental)

On by default and independent of prompt context. A PreToolUse hook looks at code searches the agent is about to run — Claude Code Grep (pattern, path, glob) and literal rg/grep commands in Claude Code or Codex Bash calls — combines their identifiers with the current prompt's terms, and adds a short digest of matching memories in the same format as prompt context. The search itself, its arguments, permissions and result are never changed.

To turn it off:

{"search_context": {"enabled": false}}

or MERKHAIN_SEARCH_CONTEXT=off (0, false, off, no disable; 1, true, on, yes enable; any case). A search_context section without enabled keeps it on. An invalid section or variable value never breaks capture or other commands: search context is then disabled, a warning goes to stderr and to the install notes, and merkhain status reports search_context.invalid. After changing the setting, re-run merkhain claude install and merkhain codex install: they register PreToolUse with merkhain claude search-hook / merkhain codex search-hook only while the setting is enabled (matcher Grep|Bash for Claude Code, Bash for Codex), and remove only merkhain's PreToolUse handler when it is disabled. Codex runs the new hook only after you trust it in /hooks.

• Only a bounded shell subset is understood: quoted patterns, -e/--regexp, --, -F, flags with operands, search roots, pipelines and a literal cd DIR && rg …. Substitutions, variables, unknown wrappers (timeout, bash -c, xargs), pattern files, rg --files and generic patterns (TODO, .*) are skipped; nothing is ever executed or read from disk.
• A search with an exact identifier (redirect_uri, proxyHost, pkg.Func) may retrieve on its own; plain words need the current prompt's terms, which the UserPromptSubmit hook records (rules planner, no model call) also when prompt context is off.
• Budgets: max_items 3 and max_bytes 2 KiB (response envelope included) per injection; max_turn_attempts 5 and max_turn_bytes 6 KiB per recipient (main agent or child agent) and user turn; deadline 500 ms for the whole hook, after which it prints nothing (retrieval stops 100 ms early so that state can still be recorded). After the output is written, recording the retrieval for Insights takes at most another 250 ms. A memory already injected into the same context (by prompt or search context) is not repeated; /compact and /clear reset that for the context that was compacted or cleared only; task and delivery state expires after retention (24 h) of inactivity.
• No model call, sync or network request runs on this path, and every failure exits 0 with empty output, including oversized, truncated or malformed hook input; search-hook ignores arguments. Codex calls without turn_id are not supported. Unquoted glob operands (rg -g *.go x) are skipped because the shell would expand them, and exclusion globs (-g '!vendor', --exclude, --exclude-dir) are never used as path hints.

Status: merkhain status reports search_context.enabled (the configuration, not proof of delivery: live delivery of the hook output to the model has not been verified for any client version yet, see docs/design/search-triggered-memory.md), invalid, and 24 h query outcome counts.

Seeing what was injected

The browser UI (merkhain ui) shows both injections. Settings → Memory injection lists whether prompt context and search context are on, the planner, its models and the limits (as merkhain serve local read them at start; the hooks read config.json on every call, and MERKHAIN_PROMPT_CONTEXT in an agent's environment overrides it there). Each session's page lists the memories injected into it, by source (prompt or code search), with when and how often, linking to each memory; the history is local and survives /compact and /clear. Insights counts the injections under their own callers, "Injected with prompts" and "Injected before code searches".

MCP (recall inside agents)

claude mcp add --scope user merkhain -- merkhain mcp

Codex, ~/.codex/config.toml:

[mcp_servers.merkhain]
command = "merkhain"
args = ["mcp"]

OpenCode and pi: registered by the plugin / extension that merkhain opencode install / merkhain pi install writes, with the absolute executable path.

merkhain mcp opens the local database directly (the service need not run) and scopes results to the project of the agent's working directory. Tools: memory_recall, memory_search, memory_timeline, memory_get, memory_save, memory_retract. Use an absolute command path if the agent's PATH lacks merkhain.

Agent instructions

The MCP server sends equivalent guidance at initialization; you can also add it to CLAUDE.md / AGENTS.md:

At the start of a task, call memory_recall with a few of its key terms (short keyword queries, not sentences). If recall is unavailable or returns nothing relevant, proceed normally. Expand only the most relevant hits with memory_get or memory_timeline (anchor). Memories are fallible background: verify what you rely on against the current code and tests; direct evidence wins. Before retrying an approach or after a failure, search kinds attempt/lesson. Save decisions with rationale, attempts and results, exact identifiers, constraints and truly unfinished work with memory_save. If the current task shows a memory is wrong, save the correction with supersedes; do not audit memory otherwise. Shared records are awareness of others' work, not locks.

Starting a fresh session for a new task, rather than continuing a very long one, keeps the conversation small while recall brings back what matters. merkhain does not restart sessions, detect prompt caches or promise savings.

What the agent can do for you

With the skill installed, ask Claude Code, Codex, OpenCode or pi in plain words and it runs the right merkhain commands, asking before anything that shares data or cannot be undone:

• "Open the merkhain UI": merkhain ui (Tailscale Personal sync setup is UI-only; the agent points you there).
• "Is my memory healthy?" / "Why isn't sync working?": status, sync status, extract jobs, and the matching repair command.
• "Connect my other laptop" / "Invite Alice to the team": device invite / member invite. The code is written to an owner-only file whose path the agent tells you; you run merkhain device join / member join on the other side.
• "Share this project with the team pool": pool bind, and publish --all only if you ask for existing history.
• "Remove my lost laptop": device list, then device remove after you confirm.

Everything an agent runs is captured into memory, so the skill keeps secrets out of its own context: it discards the UI's sign-in link, never reads enrollment codes, and leaves joins, recovery, and operator tokens to you.

Provider

Extraction turns captured sessions into memories (summary, decision, lesson, attempt, todo) through one explicitly selected provider: provider.kind = none (default: capture continues, jobs wait), codexcli, claudecli, or openai. Nothing falls back to another provider or endpoint. Change it in config.json (or MERKHAIN_EXTRACT_PROVIDER for the kind, which needs a restart of serve local): a running serve local switches to it with the next job. A provider that cannot be built (an unreadable api_key_file, say) pauses extraction: the previous provider is not used for any new job (captured sessions keep waiting as pending jobs; attempts already claimed finish) until a configuration whose provider can be built applies. merkhain status reports it.

Codex CLI — runs codex exec in a read-only sandbox with tools, hooks and user config disabled, in an empty temp directory, with your existing codex login:

{"provider": {"kind": "codexcli",
  "codexcli": {"model": "<a model your codex login can use>", "reasoning_effort": "low", "binary": "codex", "timeout_seconds": 240}}}

reasoning_effort is optional: omit it to keep the model default. When set, it is passed explicitly to Codex even though user configuration is disabled, and the provider is recorded as codexcli/<model>@<effort>. Configuration validation only checks that it is lowercase letters; the supported values depend on your Codex version and model, so Codex decides. When Codex rejects a value, the job error includes the API error (type, status if any, and message, at most 180 bytes so the whole stored job error stays within its 256-byte limit) so you can see why. That message comes from the API and could, rarely, quote request content.

Claude Code CLI — runs claude -p headless with no built-in tools (--tools ""), no MCP servers (--strict-mcp-config), safe mode (no CLAUDE.md, skills, plugins, hooks or custom agents), slash commands disabled, permission mode dontAsk and --no-session-persistence (no transcript under ~/.claude/projects), in an empty temp directory, with your existing Claude Code login. The prompt goes over stdin; the output schema is passed with --json-schema. Stdout is Claude Code's stream-json message stream: only its usage-limit events (recorded as quota observations) and the final result are read, the structured result at most max_output_bytes (default 1 MiB):

{"provider": {"kind": "claudecli",
  "claudecli": {"model": "haiku", "effort": "low", "binary": "claude", "timeout_seconds": 240}}}

model is anything claude --model accepts (an alias such as haiku or sonnet, or a full model name). effort is optional and passed as --effort; the provider is then recorded as claudecli/<model>@<effort>. --bare is not used because bare mode ignores the Claude Code login and accepts only ANTHROPIC_API_KEY; if ANTHROPIC_API_KEY is set in the service environment, Claude Code uses it instead of the login. A Claude Code too old for one of these options fails the job with unsupported and names the option. A failed run records only Claude Code's result status (subtype, terminal reason, API status), never its result text.

Local OpenAI-compatible server (Ollama ≥ 0.5, LM Studio, llama.cpp; /v1/chat/completions):

{"provider": {"kind": "openai",
  "openai": {"base_url": "http://127.0.0.1:11434/v1", "model": "qwen2.5:7b-instruct",
             "context_bytes": 98304, "timeout": "2m"}}}

A non-loopback base_url is rejected unless you add "allow_remote": true — that sends session content to that host (see Security) — and it must use https. A non-loopback http:// endpoint additionally needs "allow_insecure_http": true, which sends transcripts and the API key unencrypted to anyone on the network route; the configuration then reports a warning. Without it, the configuration still loads and everything else keeps working, but nothing is sent to that endpoint: merkhain config validate fails, serve local starts with extraction paused (jobs stay pending) and logs the reason, and status reports transport_refused. API keys only by reference: "api_key_env": "VAR" or "api_key_file": "file" (relative to config_dir).

A loopback http:// endpoint needs no opt-in, because merkhain treats local inference as trusted: use it on a machine whose local users and administrators you trust. A loopback TCP port does not identify the server: while it is down, any other user on the machine can bind its port and receive the transcripts and the API key sent to it. merkhain does not detect this.

If your server listens on a Unix socket itself (llama-server --host /path/to/llm.sock), set "socket_path": "/path/to/llm.sock" (relative to config_dir); base_url then only supplies the http scheme, Host header and path. Before each connection is used, merkhain checks the kernel's peer credentials and sends nothing unless the server runs as you or root; otherwise the job fails with endpoint socket is served by another user and is retried. Protect the socket's directory so other users cannot replace it. A bridge such as socat from a socket to a TCP port is not authentication: it runs as you, so it passes this check, and it forwards everything to whatever holds the port. Root is trusted in every case.

Opt-in live check: MERKHAIN_SMOKE_OPENAI_BASE_URL=… MERKHAIN_SMOKE_OPENAI_MODEL=… go test -run TestLiveLocalModelSmoke ./internal/provider/openai.

Each extraction attempt sends exactly one request per chunk; there is no second review pass. A job whose fully encoded request (instructions, events, earlier memories as context, and the schema) exceeds extract.request_target_bytes (default 262144, at least 16384) is split into chunks; each provider's hard limit caps the target (1 MiB for Codex, 4 MiB and a 120 KiB schema for Claude Code, context_bytes for OpenAI-compatible servers). Optional context is dropped, oldest and least relevant first, before events are split, and a split job keeps it to at most half of each request; an event too large for one request is sent in parts. Every event byte is sent exactly once. When the model's output is cut off at its output limit, that chunk is halved and the halves are sent instead (down to 4 KiB of events, and at most 50 requests per job); if that cannot help, the job fails with output_truncated and you should raise the model's output limit or lower its reasoning effort. A response above the local byte bound (output_too_large) and input no chunking can fit (input_too_large) fail distinctly. Each chunk's output must match a strict schema and cite only that chunk's events, or the attempt fails and is retried; the job completes only when every chunk succeeded. Jobs run asynchronously when a session ends (a session with events after its recorded end, as Codex records one per turn, still counts as active), plus checkpoints for long active sessions, so recent work can take a few minutes to become recallable. Inspect them with merkhain extract jobs; failures are retried with backoff, then merkhain repair retry-jobs.

Jobs are sized by what their events occupy in the provider's request, so a job fits the selected provider's input limit (codexcli 1 MiB, claudecli 4 MiB, openai at most 50 requests of context_bytes) together with its instructions and up to 32 KiB of earlier memories. A job that a provider still rejects as too large (input_too_large, for example one planned by an earlier release) is split automatically into smaller consecutive jobs that cover the same events, and the session's later jobs continue; repair retry-jobs splits such a job the same way. Only a job of a single event that is too large stays failed.

AI workers: concurrency and quota

Extraction, recall judging (recall_comparison.judge) and prompt-time planning share one limit per instance (all processes of one database, including every agent hook), set in the scheduler section:

{"scheduler": {"concurrency": 4,
  "extract": {"max_concurrency": 4, "daily_limit": 0},
  "judge": {"max_concurrency": 4, "daily_limit": 0},
  "quota_reserve_percent": 20, "unknown_quota": "continue"}}

• Concurrency. At most concurrency AI calls run at once (default 4). Extraction gets 3 slots and judging 1; either may borrow the other's unused slots while that work has nothing waiting, and a borrowed slot returns when its call ends (running calls are never interrupted). max_concurrency caps a worker below concurrency. Extraction runs sessions in parallel, taking turns between sessions, while each session's jobs still run strictly in order.
• Daily limits. None by default. A positive daily_limit bounds the jobs admitted per UTC day, retries included; the count survives restarts. recall_comparison.judge.daily_budget still limits judged comparisons when positive (0: unlimited).
• Quota. For the Codex and Claude Code CLIs, new calls stop while any usage window of the account has quota_reserve_percent or less remaining (default 20); calls already running finish. Codex usage is read with codex app-server (account/rateLimits/read) every minute while work waits or runs; Claude Code reports it with its calls. A low reading pauses the account until the window resets or a fresh reading clears it; a reading older than two minutes is stale. With no reading (quota_unknown) work continues with a warning, or waits with "unknown_quota": "pause". A rate-limited call pauses the account until the time the provider gave. Accounts are never switched.
• Prompt planning uses a model only when a slot is free and the account is above the reserve right now; otherwise the local rules planner answers. It never waits.
• Sleep. Nothing keeps the computer awake. After a resume, serve local rechecks pending work, recovers expired leases and reservations, and reads quota again.

merkhain status reports each background worker under workers (state, why it waits, jobs running of its concurrency, completions and failures in the last hour, last progress, next check) and the shared state under admission (calls in use, and each account's pause reason, until when, and how fresh its quota reading is). The browser UI shows the same under Processing → Technical details.

A panic in a background worker does not stop serve local or end browser sessions: the worker restarts with a growing delay (reported as failing, reason panic_restart), and panics and last_panic_at stay in its report. After 5 panics in 10 minutes it stays down, as failed with reason repeated_panic and the worker_failed diagnosis; a change to its configuration tries a configuration-driven worker again. The log records the panic's type and stack, never its value. A panic in a provisional first-time join exits the process instead, because it may leave identity setup half done; the next start resumes the join. A panic while reloading or watching the config file also exits, because it can leave the configuration store locked; so does a panic in a background loop of serve remote. In Personal sync hosting such a panic replaces the managed relay instead. All of these log type and stack only.

Recall

merkhain recall search "sqlite busy timeout"                     # current directory's project
merkhain recall search --kind attempt --kind lesson --since 2026-10-01 cache
merkhain recall search --pool private "release"                  # or --project ID, --pool POOL_ID
merkhain recall timeline --anchor mem_…                          # what happened around a record
merkhain recall get mem_…                                        # full record + provenance by ID (--project/--pool to restrict; --payloads: raw local events)
merkhain recall save --kind decision --title "Use WAL" --body "…" # manual memory (private unless --pool)
merkhain recall save --kind decision --title "Use WAL (v2)" --supersedes mem_… --body "…"  # correction
merkhain recall retract --reason "status noise" mem_…  # stop retrieving it (recall restore mem_… undoes)
merkhain rules cleanup --project .                     # preview older records only orchestration noise supports (--apply, --undo cln_…)

Queries: short keywords work best. Words are ANDed; "quoted phrase", a OR b, prefix* and -exclude are supported. When no record contains every word of a plain multi-word query (no quotes, OR or -), records matching any of the words are returned ranked by relevance, and the result carries "match": "any".

All worktrees of a Git repository share one project. Another clone of the same repository joins it only explicitly: merkhain project alias <PROJECT_ID> <REMOTE_URL> or merkhain project link <PROJECT_ID> <DIR> (merkhain project show prints the current project and its aliases as kind, value, project_id and created_at). Records are append-only; corrections supersede, never edit.

Recall backend

Prompt context, search context, MCP memory_search and memory_recall, and merkhain recall search rank with full-text search by default. recall_search.backend (or MERKHAIN_RECALL_BACKEND) selects another method:

{"recall_search": {"backend": "hybrid:ollama-nomic"}}

The built-in providers ollama-nomic and lmstudio-nomic need a running model server with nomic-embed-text v1.5:

ollama pull nomic-embed-text:v1.5                     # Ollama on 127.0.0.1:11434
lms server start && lms load text-embedding-nomic-embed-text-v1.5   # LM Studio on 127.0.0.1:1234

Add your own under embeddings (kind ollama or lmstudio, base_url, model; api_key_env names a variable holding a bearer token). base_url must be a loopback address unless allow_remote is set, because memory and query text is sent to it. See the comparison design for what is pinned in a generation.

serve local embeds memories in the background into recall-benchmarks/v1/index.db for the selected provider (and keeps the fuzzy vocabulary for fuzzy), even when recall comparison is off. Hooks, the MCP server and the CLI embed only the query and read those vectors read-only. Until the provider can serve, full-text search answers instead and the result says why in fallback:

Results of a non-default backend carry backend (the method that ranked) and higher-is-better scores; search cursors are bound to the backend's version (its embedding generation) and go at most 200 results deep. Insights → Method comparison marks the live backend's scorecard, and each card's Served count is the requests that method answered. Other backends ignore the prompt-context relevance floor: fuzzy keeps what its search finds, embed: needs cosine 0.35, and hybrid: keeps a memory that passes the full-text gate or the cosine floor.

Schema 306 drops the unused explicit alias flag. Back up the SQLite database before installing it; older binaries reject schema 306, so rollback requires the matching database backup and binary.