feat(issue-18): decide liveness and delivery by the event protocol, not the screen

Contract (no new event types): heartbeat every N s (automatic timer),
unresponsive after K x N s without an event, `started` within T s of the
injection, retained per-job state, and a remote transport.

- registry: heartbeat_sec / heartbeat_misses / ack_timeout_sec / transport
  fields; idle_timeout_sec derives K x N unless explicit (defaults 30/4/60
  keep the historical 120 s idle window).
- heartbeat.py + publish_event --heartbeat N: detached fixed-cadence timer
  publishing signed `progress` heartbeats (threaded wire publish, local
  record first); stopped by the terminal event via pidfile, by re-dispatch
  and by lifetime. Terminal events refuse --heartbeat.
- publish_event: retained `<prefix>/state` (status, last_event, seq, HMAC)
  sent on the same connection after every event (--no-state to skip);
  publisher.publish_raw takes extra messages.
- job_subscriber: subscribes the state topic, recovers terminal status from
  retained state, resets idle on any accepted message, records seq_gap and
  reconnected in the audit log, idle budget before the first event is
  T + K x N (--ack-timeout), --remote mirrors started/terminal events into
  the registry through the transition boundary.
- lib.sh send_keys_safe: an accepted-but-unconfirmed submission is confirmed
  by agent_status working or the job's own `started` (_sks_job_started);
  pane change and busy token are no longer delivery signals.
- wrapper: --heartbeat/--heartbeat-misses/--ack-timeout (MAM_* env),
  injected start command arms the timer, brief states the contract,
  _wait_started (registry / events.log / subscriber log, bounded by T and
  the wall clock) replaces the 10 s inject grace, R1 grace uses only the
  on-disk event age (herdr/pane logged once as a diagnostic),
  remote:<name> sessions skip preflight/idle/pane and dispatch over the
  broker (remote_dispatch).
- remote.py (+ scripts shim): signed retained dispatch (MAM_DISPATCH_SECRET)
  and a runner that mirrors the job into its own registry with the
  orchestrator's identity, publishes started --heartbeat, runs --exec and
  publishes the terminal event.
- loop runner: passes MAM_HEARTBEAT_SEC / MAM_HEARTBEAT_MISSES /
  MAM_ACK_TIMEOUT_S through; YAML rows with transport: remote delegate as
  remote:<name>.
- docs: MESSAGING.md 2.5 contract + remote transport, delegate-job SKILL.md,
  registry.md, loop SKILL.md.
- tests: tests/test_issue18_liveness_contract.py (16); issue 14 pane-signal
  tests flipped to started-based; issue 6 working-agent tests acknowledge
  with started; CLI snapshot/help captures refreshed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff

MESSAGING.md

@@ -154,7 +154,7 @@ Every event payload must adhere to the following schema structure:
 * **Example Detail**: `"Job 918b0612 started"`
 
 #### 2. `progress`
-* **Emit Trigger**: Optional. Emitted at major check-points, long loops, or sub-task boundaries.
+* **Emit Trigger**: At least every `heartbeat_sec` while working (Issue #18 heartbeat obligation; `publish_event.py --event started --heartbeat N` automates it with `detail: heartbeat`), plus any check-point the agent wants to report.
 * **Payload Constraints**: None.
 * **Example Detail**: `"Section 1: MQTT Broker Architecture completed"`
 
@@ -185,6 +185,53 @@ To prevent unauthorized users from hijacking or spoofing events on public broker
 
 ---
 
+### 2.5 Liveness and Delivery Contract (Issue #18)
+
+No new event types: the existing events are promoted to a contract with
+obligations, periods and timeouts, so that an agent's life and the delivery of
+its prompt are decided by what the agent **publishes**, never by scraping a
+terminal (`herdr agent_status`, pane text, screen changes). This is what lets
+an agent on another host, which shares neither the pane nor the
+orchestrator's disk, take part on equal terms.
+
+| Rule | Contract | Where |
+|---|---|---|
+| **Heartbeat obligation** | While working, the agent publishes `progress` at least every **N** s (`heartbeat_sec`, default 30). `publish_event.py --event started --heartbeat N` starts a detached timer that publishes `progress` (`detail: heartbeat`) every N s with the same `dispatch_token`/`attempt_id`, so the agent does not have to remember. The timer stops itself at the terminal event (which also kills it through `<registry>/<job>.heartbeat.pid`), on re-dispatch, or after `timeout_sec`. | `lib_py/delegation/heartbeat.py`, `publish_event.py --heartbeat` |
+| **Unresponsive** | The orchestrator treats a job as unresponsive after **K x N** s without any event (`heartbeat_misses`, default 4 → 120 s; `--idle-timeout` overrides). The subscriber's idle clock counts every accepted message (`progress` and retained state included); the wrapper's post-idle grace re-reads only the job's events.log (`_recent_disk_heartbeat`, written before every wire publish). herdr status and the pane are logged once as a *diagnostic* (`liveness: ... herdr/pane says busy=yes/no (diagnostic only)`) and never decide. | `multi-agent-mux-delegate-job` (`finalize_unfinished`), `job_subscriber.py` |
+| **Delivery** | After the prompt is handed over (herdr accepted the submission, or the remote dispatch was published), the agent's `started` must arrive within **T** s (`ack_timeout_sec`, default 60): the registry leaves `pending`, a `started` line lands in events.log, or the subscriber prints one. No `started` within T → the inject fails closed: rc 1, registry `error` (`agent_blocked`), and the prompt is **never re-injected**. `send_keys_safe` confirms an accepted-but-unconfirmed submission only by `agent_status == working` or the job's `started`; pane changes and busy tokens are no longer signals. | `_wait_started`, `lib.sh: _sks_job_started` |
+| **Termination** | Only `completed` / `error` (and an operator `cancelled`) end a job. Terminal immutability (Issue #6 CAS) refuses anything that arrives after `deadline_timeout` closed the job. | `engine.py` |
+| **Wire durability** | Every event (QoS 1) is followed, on the same connection, by a **retained** `<topic_prefix>/state` message (`event: state`, `status`, `last_event`, `seq`, signed like an event). A subscriber that connects or reconnects receives it at once: a terminal status ends the wait, any state resets the idle window, and a `seq` beyond the last one seen is recorded as `seq_gap` in the audit log (also for gaps between events). paho reconnects on its own; `on_connect` re-subscribes both topics and logs `reconnected`. | `publish_event.py: build_state_payload`, `job_subscriber.py: _on_state` |
+| **Credentials** | Each host connects with its own `.mam.env` (Issue #12: the password is never in a record). Every event and state message carries the job's HMAC (`auth_token`). A remote dispatch is signed with the pre-shared `MAM_DISPATCH_SECRET`. | `mqtt_common.py`, `remote.py` |
+| **What screen scraping is still for** | (a) operations that need the screen: prompt injection into a pane, trust/permission dialogs, the pre-injection idle handshake; (b) diagnostics on the same host when events stop. Neither is a liveness or delivery decision. | `lib.sh` |
+
+Registry record fields: `heartbeat_sec`, `heartbeat_misses`, `ack_timeout_sec`,
+`idle_timeout_sec` (derived K x N unless explicit), `transport`
+(`herdr` / `remote`). Wrapper options: `--heartbeat`, `--heartbeat-misses`,
+`--ack-timeout` (also `MAM_HEARTBEAT_SEC`, `MAM_HEARTBEAT_MISSES`,
+`MAM_ACK_TIMEOUT_S` for the loop runner).
+
+#### Remote transport (`--agent-session remote:<name>`)
+
+```
+orchestrator ── retained <prefix>/dispatch (signed) ──▶ remote runner (other host)
+orchestrator ◀── <prefix>/events + retained <prefix>/state ── remote runner / agent
+```
+
+`remote.py dispatch` publishes the brief, the job's `auth_token`,
+`dispatch_token`, `attempt_id` and the contract as one retained, signed
+message. `remote.py run --session <name> [--exec CMD]` on the other host
+mirrors the job into its own registry (same job id and identity), retires the
+retained dispatch, publishes `started --heartbeat N`, runs the command (env:
+`JOB_ID`, `MAM_JOB_BRIEF`, `MAM_PUBLISH_CMD`, ...) and publishes the terminal
+event unless the agent already did. The orchestrator's subscriber runs with
+`--remote` (automatic for `transport: remote` jobs) and mirrors `started` and
+terminal events into the orchestrator's registry through the transition
+boundary, because nothing else on that host will. The wrapper takes no
+herdr/pane/lsof/ps path for a remote session. Because the dispatch carries the
+job's `auth_token`, use the remote transport only on a private broker.
+
+---
+
 ## 3. Job Lifecycle & State Transitions
 
 The lifecycle of a delegated job progresses through a highly coordinated state machine, involving file-based registry claiming, asynchronous message subscription, and multi-faceted event publishing.
@@ -291,7 +338,8 @@ The subscriber acts as the central execution watchdog:
   ```
 * **Dual Timeout Semantics**:
   1. **Wall-Clock Timeout**: Calculated relative to absolute startup time (`wall_deadline = start + wall_timeout`). It acts as a hard budget limit, guarding against an agent hanging indefinitely.
-  2. **Activity Idle Timeout**: Measured as the difference between the current monotonic time and the last packet arrival time (`idle_left = idle_timeout - (now - last_event)`). If the agent fails to print logs or publish progress updates for the duration of the idle window, the subscriber aborts and exits with code 2.
+  2. **Activity Idle Timeout**: Measured as the difference between the current monotonic time and the last packet arrival time (`idle_left = idle_timeout - (now - last_event)`). If the agent fails to publish any event (heartbeats and retained state included) for the duration of the idle window - K x N under the Issue #18 contract - the subscriber aborts and exits with code 2.
+* **Retained state and seq gaps (Issue #18)**: the subscriber also subscribes to `<topic_prefix>/state`; a retained terminal state resolves the job at (re)connect, any state resets the idle clock, and a seq jump on either topic is written to the audit log as `seq_gap`. With `--remote` (or `transport: remote`) accepted `started`/terminal events are mirrored into the local job record.
 
 ---