Protocol

Every capture source — the OpenClaw plugin buffer, the Claude Code and Cursor disk parsers — feeds the same server endpoint and obeys the same crash-safe protocol. This page is the contract they share.

The shape of an interview

durable trail ──▶ window (events + cursor range) ──▶ POST /interview/submit
                                                          │
                                                    mask credentials
                                                          │
                                              chunked map-reduce (LLM)
                                                          │
                                        report sections ──▶ typed memories
                                                          │
                                            idempotent bulk write
                                                          │
                                          advance per-node watermark

1. The cursor and the window

Each capture source reads its trail as an append-only sequence and assigns every event a monotonic seq (the disk parsers use the raw line index; the plugin buffer uses its buffer offset). A window is a contiguous slice — the events plus a cursor_from/cursor_to range — submitted together.

The source keeps no local cursor. Where to resume comes from the server's watermark document (below); a fresh node reads back "never interviewed" and starts at zero. This is what makes the client side crash-safe: there is no local state to lose or corrupt.

Windows are bounded so a single interview stays inside the server's time budget — the disk parsers cap at ≤400 events and ≤150k characters per window and submit at most a few windows per run, resuming the rest next time.

2. Submit and seq validation

POST /api/v1/interview/submit takes { node_id, agent_id, cursor_from, cursor_to, events[] }. The server validates that the event seqs are strictly ascending and lie within [cursor_from, cursor_to] — seqs[0] >= cursor_from and seqs[-1] <= cursor_to. Sparse seqs are legal (most trail lines are noise and get filtered), and cursor_to may point past a filtered tail, so the cursor advances over noise without wasting a window on it.

Only real user prompts and assistant replies are submitted as events. Thinking, tool calls, tool results, and meta lines are dropped at parse time — they still advance the cursor, but they aren't interviewed.

3. Masking

Before synthesis, the server masks credential-shaped strings across each event's content, tool, and outcome fields — a second layer behind the disk parser's local scrub. What the LLM sees, and what could ever be stored, is already redacted.

4. Chunked map-reduce

A 12-hour window can exceed the model's context, so synthesis is a map-reduce. Events are split into char-budgeted chunks; each chunk is interviewed into a mini-report (map); the mini-reports merge into one final report (reduce). A single-chunk window skips the reduce. One shared prompt drives every chunk, asking the model to fill the six report sections and to use only timestamps that appear in the events — never invented ones.

5. Sections become typed memories

The final report's six sections map onto MemClaw's memory-type enum:

Each item becomes a memory written through the normal pipeline — enriched, embedded, governed, and recallable like any other. Event timestamps are carried through, so the memories reflect real wall-clock time, not when the interview ran.

6. Idempotency and the watermark

The write is ordered to be safe across crashes and retries:

1. Interview → bulk write. The memories are written via the idempotent bulk endpoint under a deterministic attempt id — sha1(node_id:cursor_from:cursor_to). Re-submitting the exact same window resolves the already-committed rows as duplicate_attempt instead of writing them twice.
2. Then advance the watermark. Only after the write commits does the per-node watermark document (collection interview_watermarks) move forward to cursor_to.

Because the watermark advances last, a crash between submit and watermark update simply re-submits the same window next run — and the deterministic attempt id dedups it. The result is never a gap, never a duplicate.

Scheduling

On the plugin-buffer path, the server decides when a node is due: it compares each node's last interview time against the tenant's period_hours (default 12) and issues a capture command to due nodes. On the disk-parser path, cadence is external — a cron job and/or a session-end hook — but the same watermark makes any cadence safe to combine.

Where to look in the source

• Submit endpoint + seq validation: routes/interview.py
• Masking, chunk_events, build_prompt, section map, watermark, scheduler: services/interview_service.py
• Client windowing + runner + watermark read: clients/python/src/memclaw_client/interviewer/