fix(issue-19): bound the subscriber daemon to its workspace, forward SIGTERM, keep sandboxes from leaking it

create_session.sh detaches `reconcile.sh --subscribe --idle-timeout 0` per
workspace; test sandboxes never reaped it and the daemon neither noticed its
workspace vanish nor honoured SIGTERM (bash deferred the signal behind the
foreground python child). 202 orphans were found holding public-broker
wildcard subscriptions for deleted pytest tmpdirs.

- daemon.py / lib.sh: MAM_NO_SUBSCRIBER_DAEMON=1 (literal) skips the spawn but
  still reports a running daemon; cli prints a stderr note, stdout stays empty.
- reconcile.sh: python subscriber runs as a waited background child with a
  TERM/INT trap (exit 143); the MQTT loop exits when the workspace dir is gone
  or .mam/monitor.lock is no longer the held inode; SIGTERM/SIGINT handled via
  the stop event so loop_stop/disconnect run; the polling fallback checks the
  same and sleeps in a waited child.
- conftest: sandbox default MAM_NO_SUBSCRIBER_DAEMON=1, per-sandbox reaper
  finalizer, session-scope leak guard over the pytest basetemp.
- tests: 11 lifecycle tests (tests/test_issue19_subscriber_daemon_lifecycle.py);
  phase-1 watchdog/daemon characterization tests opt back in.
- docs: monitor SKILL.md lifecycle contract + pitfalls, create SKILL.md note.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
GodopuandClaude Fable 5.1 committed 2026-09-13 15:34:56 +09:00
1 parent 8e7282c948
commit ce315d8c5e
9 files changed
+536 -12

No files matched your search

@@ -69,6 +69,23 @@ Run it under whatever supervisor you already use (a dedicated herdr pane, `nohup
or a background job). Nothing else needs to be running for the monitor to work —
it reconciles YAML ↔ herdr ↔ disk on its own.
### Daemon lifecycle contract (Issue #19)
`multi-agent-mux-create` starts one detached `reconcile.sh --subscribe --idle-timeout 0`
per workspace through `lib.sh::start_watchdog` (`lib_py.delegation.daemon`). That daemon:
- is **one per workspace** — `.mam/monitor.lock` (flock) rejects a second instance, and
`start_watchdog` returns the running PID instead of spawning again;
- **ends on its own only when its workspace is gone** — every wake-up (≤ 5 s on the MQTT
loop, every poll on the broker-down fallback) it checks that the workspace directory
still exists and that `.mam/monitor.lock` is still the inode it holds. A deleted
workspace, a purged `.mam/`, or a re-created lock file makes it disconnect and exit 0;
- **stops promptly on `SIGTERM`/`SIGINT`** — the bash wrapper forwards the signal to the
python subscriber and exits 143. `SIGKILL` is never required;
- is **not started when `MAM_NO_SUBSCRIBER_DAEMON=1`** — the literal `1` only. Use it when
the monitor is supervised elsewhere; a daemon that already runs is still reported. Test
sandboxes set it by default and additionally reap any daemon anchored in the sandbox.
The herdr commands the script issues (`herdr agent list`, `herdr agent get <session>`)
are real native herdr commands — do not substitute tmux-era names like `herdr ls` /
`herdr list-panes` outside a shell that has sourced `.agents/skills/lib.sh`.
@@ -141,6 +158,8 @@ YAML: no such session
- **Don't expect `--once` to stay alive** — it does a single pass and exits. Use `--subscribe` for continuous monitoring.
- **`--idle-timeout` defaults to 3600s** — a monitor meant to run indefinitely needs `--idle-timeout 0` explicitly, or it will quietly exit after an hour of broker silence.
- **`--idle-timeout 0` is bounded by the workspace, not by time** — the daemon exits when its workspace directory or `.mam/monitor.lock` disappears (Issue #19). Do not delete `.mam/monitor.lock` under a running monitor unless you mean to stop it.
- **Spawning from tests or ephemeral checkouts** — set `MAM_NO_SUBSCRIBER_DAEMON=1` so `create_session.sh` does not detach a daemon that outlives the checkout; the pytest sandbox does this for you.
- **The poll interval is a default** — `RECONCILE_POLL_INTERVAL` (15s) is what the broker-down fallback uses. A workspace with 5+ agent sessions can bump it to reduce noise.
- **Coalesce repeated drifts** — the same drift re-appears on every pass until it is resolved. A caller that acts on `drifts[]` should compare against the previous pass and act only on *new* entries; the script does not deduplicate for you.
- **Don't fight the user's explicit action** — if `multi-agent-mux-stop` is mid-flight and the monitor sees the same session in two states within 5s, prefer the user's most recent action. The monitor should not auto-revert a fresh `terminated` to `running` because of a stale `herdr has-session` check.