Event filter

Orchestrators and agent protocols put text into a session that is not about the work itself: worker introductions and routing instructions, "You have 1 orchestration message" notices, acknowledgements, task state changes. The event filter decides which content of captured events takes part in memory extraction and automatic recall (prompt context, search context and the recall comparison). It removes recognizable automation noise and keeps assignments, reports, decisions, constraints, results and blocker explanations.

The filter never changes what is captured: raw events stay in history unchanged unless retention, when you turn it on, removes their content. It is deterministic and calls no model. Content it does not recognize is processed exactly as before.

Mode

The filter is on by default (earlier releases defaulted to off). The built-in rules were evaluated on labeled development and held-out examples (internal/filtereval): they remove recognizable orchestration noise and lost no content labeled as substantive. With no rules file the built-ins apply; content no adapter recognizes is processed as before. To turn it off, or to only watch what it would do:

{"event_filter": {"mode": "off"}}

{"event_filter": {"mode": "preview"}}

MERKHAIN_EVENT_FILTER=off|preview|on overrides the file. A running serve local applies a change on save; agent hooks read it on their next prompt. To see what the rules decide about your own recorded sessions before or after the change, run merkhain rules replay --project . --since 7d (read-only; see below), and merkhain rules explain --event evt_… for one event. Local memory describes what on does to Orca prompts and to A2A and AG-UI events.

Rules

Built-in rules ship enabled; merkhain rules check lists them in evaluation order. Each has a stable ID (orca.…, a2a.…, agui.…, core.…), one action and a plain-language explanation. An optional rules file you own can disable built-ins and add rules. By default it is event-filter-rules.json next to config.json (it may be missing); event_filter.rules_file names another file by absolute path (which must then exist).

{
  "version": 1,
  "disabled_rules": ["orca.acknowledgement"],
  "rules": [
    {"id": "company.runner-heartbeat", "match": {"producer": "company-runner", "event_type": "heartbeat"}, "action": "history_only"},
    {"id": "company.worker-result", "match": {"producer": "company-runner", "event_type": "worker_result"},
     "action": "unwrap", "content_field": "payload.result", "recall": false}
  ],
  "projects": {
    "prj_01jexample": {"disabled_rules": ["core.wrapper"], "rules": []}
  }
}

For each part of an event, the first matching rule decides: the project's rules, then your global rules, then the enabled built-ins. A part no rule matches is kept. There are no chains: unwrapped content is not evaluated again.

Rule IDs are your own (company.runner-heartbeat); the namespaces core., orca., a2a., agui. and merkhain. are reserved. To change a built-in, disable it and add your own rule.

Match (every field given must match; at least one):

Text matchers never match an unfinished stream. merkhain rules explain shows the producer, protocol, event type and content type of every part, which is what you match on.

Filtering a CLI. orca.orchestration-command excludes agents driving orca orchestration … this way (not check or worker-read, whose output carries workers' reports), and any other CLI whose commands are noise takes one rule:

{"id": "company.deploy-status", "match": {"content_type": "result", "command": "deployctl\\s+status\\b.*"}, "action": "history_only"}

The line is split at ;, &&, || and newlines outside quotes, ignoring cd, sleep and variable assignments, so cd x && deployctl status | head matches, and deployctl status; go test ./... does not: output of other work stays. The result of a matching tool call is left out with it when both are in the same extraction job. A message that only mentions the command never matches.

Actions:

unwrap needs exactly one selector: content_field (a dotted path of JSON keys, such as payload.result), between ({"start": "<report>", "end": "</report>"}: the whole text must start and end with them) or a (?P<content>...) group in the rule's text pattern (what the group captured is kept). If the selector does not find its structure, the rule does not match and the next one is tried. "recall": false on a keep or unwrap rule lets the content reach extraction without starting automatic recall (as for workers' reports).

Avoid rules on length, on words that happen to occur in orchestration, or on who wrote a message: they remove useful content. Match complete structures.

An invalid file never stops capture. merkhain rules check names every problem with its location. Until the file is fixed, a running serve local keeps its last valid rules (and status reports event_filter_rules_invalid), and other processes run with the filter off (the built-ins included), whatever the mode.

Commands

check, explain and replay are local and read-only: they change no event, memory or job and call no model. cleanup previews read-only too; only its explicit --apply and --undo append local retractions and restores. explain and replay print full content; status and the job records carry only counts and rule IDs.

merkhain rules check

Validates the rules file and the configuration and prints the effective rules in evaluation order, with the rule-set version (rs1-…) that job records and status name, and what the running service uses:

merkhain rules check                         # the configured file
merkhain rules check --file candidate.json   # another file, before you save it in place
merkhain rules check --project .             # the effective rules of this project
merkhain rules check --json

Rules file: /home/me/.config/merkhain/event-filter-rules.json (valid)
Mode: on; a process started now runs with on
Running service: mode on, rule set rs1-6c0f…
Rule set: rs1-6c0f…

Effective rules in evaluation order (project, user, then built-in):
     1. company.runner-heartbeat         history_only             user
     2. orca.inbox-notification          history_only             built-in
     -  orca.acknowledgement             history_only             built-in, disabled
  …

It exits 1 when the file is invalid.

merkhain rules explain

Shows what the rules decide about some content: per part, the rule, its action, what it removed, what remains, and whether it may contribute to extraction and start automatic recall; for prompts, the cleaned query and the automatic recall decision with and without the filter.

merkhain rules explain "the text of a prompt"         # or on stdin: merkhain rules explain < prompt.txt
merkhain rules explain --event evt_…                  # a captured event, such as one of a memory's source_event_ids
merkhain rules explain --jsonl run.jsonl --protocol ag-ui --producer support-agent [--line 3]
merkhain rules explain --rules candidate.json --project . --json "…"

--agent (default claude), --producer and --protocol describe where text came from; a JSONL recording is read like capture import, with streamed chunks assembled as extraction assembles them (--line N explains only the N-th record). Flags come before the text. --example prints the input instead as an evaluation example for internal/filtereval, labeled with what the built-in rules (and --rules) decide now, for you to review and keep outside the repository (internal/filtereval/testdata/README.md). For an Orca dispatch:

Part 1 of 2: wrapper, orca dispatch_preamble
Rule: orca.worker-introduction
Action: history_only
Removed:
  You are working inside Orca, a multi-agent IDE. You are a dispatched worker. …
Remaining:
  (nothing)
Eligible for extraction: no
Eligible to trigger recall: no

Part 2 of 2: message, orca dispatch_task
Rule: orca.dispatch-task
Action: keep
Removed:
  (nothing)
Remaining:
  Investigate why the migration fails on an empty database.
Eligible for extraction: yes
Eligible to trigger recall: yes

Cleaned query:
  Investigate why the migration fails on an empty database.

Content no adapter recognizes is reported as unrecognized and processed as before, unless a rule matches it: text patterns also read the typed text of an unrecognized prompt, so a rules file can strip another tool's wrapper or drop its notices without new code.

merkhain rules replay

Applies the current rules, or a candidate file, to recorded sessions with the code extraction and prompt recall use, and reports what would change:

merkhain rules replay --session ses_…
merkhain rules replay --project . --since 7d
merkhain rules replay --project . --rules candidate.json --verbose

It reports the extraction outcome of the events (kept, cleaned, excluded, streamed; unrecognized; items sent, assembled, partial, duplicates, replays), how many events and items each rule matched, and how many prompts would no longer start automatic recall or would search with a cleaned query — with --rules, also how many decisions and queries the candidate changes compared with the current rules. --verbose lists every event with its outcome and rules. Each session is replayed as one extraction job from its first event (or the first after --since), so content an earlier job already extracted may show as sent again.

merkhain rules cleanup

The filter applies to new extraction only. Memories extracted before it was on (or before a rule existed) may be about orchestration noise. rules cleanup classifies a project's existing memories by their evidence: it reads the captured events each memory cites on this device and renders them with the current rules as extraction would (the configured rules, whatever event_filter.mode is; an invalid rules file stops it).

merkhain rules cleanup --project .                    # preview (read-only, no model calls)
merkhain rules cleanup --session ses_… --since 30d    # one session, memories of the last 30 days
merkhain rules cleanup --project . --json             # counts, per-rule counts, per-memory lines
merkhain rules cleanup --project . --apply            # exclude the chatter-only memories, prints the run ID
merkhain rules cleanup --undo cln_…                   # restore what that run excluded
merkhain rules cleanup --undo mem_… mem_…             # or individual memories

Each line names the memory, its class, why, and the rules that excluded its chatter events; the summary counts the classes and the chatter-only memories per rule, with the rule-set version used.

--apply classifies again and excludes only the chatter memories, each with a retraction (the mechanism of recall retract) whose reason names the cleanup run (cln_…), the rules and the rule-set version. An excluded memory leaves automatic recall, search, prompt and extraction context, but memory_get, merkhain recall get, the timeline and the UI still show it with the reason. Nothing is edited or deleted. Applying again changes nothing; memories already retracted are left alone. The retractions always stay on this device, also for records you published: a cleanup never changes what a pool shares. To retract a published record for the pool, use recall retract.

--undo restores the memories a cleanup run excluded (cln_…, optionally narrowed with --project) or the given memories, as long as their latest local retraction is the cleanup's. A retraction you made otherwise is not undone here; merkhain recall restore does that.

Superseded records and resolved todos are skipped: recall already leaves them out. A chatter-only correction is listed for review rather than excluded, since excluding it would leave the record it corrects superseded and out of recall as well.

For review memories, read the record and its sources (merkhain recall get mem_… --payloads) and retract it with merkhain recall retract if it is noise. Re-extracting such a session with the current rules would use the configured extractor and cost model calls; cleanup never does that.

Status

merkhain status has an event_filter section: the configured and effective mode, the rules file state (none, loaded, invalid, and rules_kept), the global rule-set version and rule counts, the outcome counts of jobs completed in the last 7 days (jobs_7d: events kept/cleaned/excluded, items, per-rule and per-version counts) and of prompts (prompts_7d: <mode>_<excluded|cleaned|unchanged>). These are numbers and rule IDs only, never content.