fix(issue-18): admit remote attempts atomically, end timed-out commands with their group, fence heartbeats off the terminal, renew liveness only on news

Review round 2 (planner-reviewer-codex-01, NOT PASS at d09c162):

- R1 remote.mirror_job: admission under the registry lock. The same
  dispatch_token is idempotent (a finished attempt never runs again), a
  lower attempt or the same attempt under another token is refused before
  any write, and only a strictly newer attempt replaces the record
  (register_job(supersede_older_attempt=True); events.log kept, seqs stay
  monotonic).
- R2 remote.handle_dispatch: the --exec command runs in its own session;
  at timeout_sec, or when the runner is interrupted (SIGTERM now unwinds
  it), the whole process group is terminated (TERM, then KILL) before the
  error is published.
- R3 terminal barrier: a terminal publish raises an attempt-scoped barrier
  under the lock that allocates seqs, stops the heartbeat timer, gives it
  MAM_HEARTBEAT_DRAIN_S (default 1 s) to exit and then kills it, and only
  then takes its own seq. Beats are admitted under that lock
  (next_seq(admit=...)); a beat still on the wire is cancelled with the
  timer rather than holding the terminal back; and a late non-terminal
  event never publishes a running state over a terminal record.
- R4 job_subscriber: a retained state renews the idle clock only when its
  seq is newer than everything already accepted; re-delivered or older
  snapshots still recover a terminal status.
- tests: five regressions, all failing on 77f2829; T1-14's wall clock
  15 -> 30 s against public-broker latency.
- docs: MESSAGING 2.5 (barrier, state renewal, detection vs final close),
  heartbeat docstring, loop SKILL.md (remote rows need the Python runner),
  VERSIONS v4.5.0 items 1/4/5 and the test count (25).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
This commit is contained in:
GodopuandClaude Opus 5 committed 2026-09-13 20:49:56 +09:00
1 parent 77f2829b8e
commit 135d76f903
11 files changed
+577 -67

No files matched your search

+12 -1
View File
@@ -88,6 +88,7 @@ def register_job(
dispatch_token: Optional[str] = None,
attempt_id: int = 1,
transport: str = "herdr",
supersede_older_attempt: bool = False,
) -> str:
"""Create a new ``pending`` job record and return its id.
@@ -147,7 +148,17 @@ def register_job(
}
with registry_lock(registry_dir):
if mqtt_common._job_path(job_id, registry_dir).exists():
raise FileExistsError(f"job already exists: {job_id}")
if not supersede_older_attempt:
raise FileExistsError(f"job already exists: {job_id}")
# A remote runner adopting the orchestrator's next attempt (review
# round 2, codex R1). Decided under this lock, so a stale or
# conflicting dispatch can never roll the record back.
old = mqtt_common.load_job(job_id, registry_dir)
old_attempt = int(old.get("attempt_id", old.get("iteration", 1)) or 1)
if int(attempt_id) <= old_attempt:
raise FileExistsError(
f"job {job_id} is already at attempt {old_attempt}; refusing attempt {attempt_id}")
record["last_seq"] = int(old.get("last_seq", 0) or 0) # seqs stay monotonic across attempts
_atomic_write_record(job_id, registry_dir, record)
# Seed the persistent audit log (meta.json + status.json + a "registered"
# event). Best-effort inside init_job_log — never blocks registration.