fix(issue-23): both ack waits share one contract - the wrapper's status bypass is gone; the G9 guard removes only the docstring node
Review round 3 (planner-reviewer-codex-01, reviewer-opencode-01): R1 blocking, R2 and the "ONE builder" wording non-blocking. All three reproduced; none rebutted. R1. _wait_started still returned 0 as soon as the record left `pending`, ahead of the shared predicate, so `error`, `cancelled` and another dispatch's `running` all acknowledged an injection that create's delegate_wait_started rejects - the two contracts the ack-finding brief said must not diverge. Both reviewers reproduced it with the real producer and no record manipulation; my S7n had hidden it by resetting the record to `pending`. The bypass and the console grep are gone. The wrapper's evidence is now delegate_started_recorded alone, the same call create makes. For a remote agent - no shared disk, so no line in events.log - the predicate reads the subscriber's audit record of the `started` it received. That record is attributable by construction: the subscriber verifies the HMAC and drops any event whose data.dispatch_token differs from the job's expected token before it writes anything. The token itself cannot go into the record (the audit redactor masks token-named keys), so the subscriber now stamps a boolean dispatch_verified, and with a token demanded only a verified record counts; an untokened legacy record fails closed. The registry status is never evidence; a terminal job is preserved by the terminal logic, not turned into an acknowledgement. Tests: W1/W2/W3 are the reviewers' three negatives and the positive, real producer, record left exactly as it produced it; W4 runs create's wait and the wrapper's wait side by side on all four and requires the same answer; W5 pins the remote route (a verified record counts, an unverified or non-started one does not); W6 drives the real _Watcher and shows it stamps the flag on a matching started and records nothing for a mismatch; S7n no longer resets the record. test_issue14's "registry ack over unconfirmed inject" produced its ack with a bare status flip - the very bypass - and now produces it the way an agent does, publish_event with the dispatch token; its scenario and every other assertion are unchanged. test_issue18's remote e2e accepted the status messages and now expects the shared one. R2. _code_only blanked a docstring's whole line range, so code sharing that line - `"""doc"""; call()` - vanished from the scan. It now removes only the docstring node from the AST and unparses the rest (comments drop out with it, code strings stay); test_g9b pins the four same-line shapes. The builder comment claimed one builder for create and the wrapper; the wrapper keeps _pub_cmd and S7l pins the two equal argument-for-argument. The comment now says that. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
4805fd4511
commit
087eacaf21
7 files changed
+250
-72
No files matched your search
+48
-21
@@ -2176,8 +2176,9 @@ delegate_publish_event() {
|
||||
# delegate_publish_cmd <job_id> [dispatch_token] [attempt_id]
|
||||
# The publish command an agent is told to run, shell-quoted, with absolute
|
||||
# script/registry/audit-log paths (Issue #21: an agent whose cwd is a worktree
|
||||
# runs it as written). ONE builder for create_session's onboarding and the
|
||||
# delegate-job wrapper's dispatch, so both hand the agent the same shape:
|
||||
# runs it as written). create_session's onboarding uses this; the delegate-job
|
||||
# wrapper still builds its own (_pub_cmd), and S7l pins the two equal
|
||||
# argument-for-argument, so both hand the agent the same shape:
|
||||
# `--expect-dispatch-token T --attempt-id A` make publish_event write
|
||||
# data.dispatch_token / data.attempt_id into the agent's own events, which is
|
||||
# the only way a `started` can name the dispatch it answers (the trial-2
|
||||
@@ -2219,33 +2220,59 @@ except Exception:
|
||||
' 2>/dev/null || true
|
||||
}
|
||||
|
||||
# delegate_started_recorded <job_id> [dispatch_token]
|
||||
# delegate_started_recorded <job_id> [dispatch_token] [logs_dir]
|
||||
# THE definition of "this dispatch's started is on disk", shared by
|
||||
# delegate_wait_started (create_session's onboarding ack) and the delegate-job
|
||||
# wrapper's _wait_started: a `started` line in the job's events.log whose
|
||||
# data.dispatch_token equals <dispatch_token> when one is given (a loop job
|
||||
# keeps one events.log across attempts, and a stale attempt's started must not
|
||||
# vouch for this one - review R1). The line is parsed as JSON, never
|
||||
# wrapper's _wait_started. Two sources, one rule:
|
||||
#
|
||||
# * <registry>/<job>.events.log - the local publisher's own line, whose
|
||||
# data.dispatch_token must equal <dispatch_token> when one is given (a loop
|
||||
# job keeps one events.log across attempts, and a stale attempt's started
|
||||
# must not vouch for this one - review R1);
|
||||
# * <logs_dir>/<job>/events.ndjson - the subscriber's audit record of a
|
||||
# `started` it received, the only evidence a REMOTE agent (no shared disk)
|
||||
# can leave. The subscriber verifies the HMAC and drops any event whose
|
||||
# data.dispatch_token differs from the job's expected token before it
|
||||
# writes this record, and marks it dispatch_verified when that check ran;
|
||||
# with <dispatch_token> given, only a dispatch_verified record counts.
|
||||
#
|
||||
# Never the registry status: a record leaves `pending` when another writer
|
||||
# closes it or another dispatch claims it. Lines are parsed as JSON, never
|
||||
# pattern-matched. rc 0 = recorded, 1 = not (yet).
|
||||
delegate_started_recorded() {
|
||||
local job_id="$1" tok="${2:-}"
|
||||
local job_id="$1" tok="${2:-}" logs_dir="${3:-}"
|
||||
local reg_dir="${MAM_JOBS_DIR:-${WORKSPACE_ROOT:-$PWD}/.mam/jobs}"
|
||||
[ -n "$logs_dir" ] || logs_dir="${DELEGATE_JOB_LOGS_DIR:-${WORKSPACE_ROOT:-$PWD}/.mam/delegate_job_logs}"
|
||||
local ev_log="$reg_dir/$job_id.events.log"
|
||||
[ -f "$ev_log" ] || return 1
|
||||
EV_LOG="$ev_log" EV_TOK="$tok" python3 -c '
|
||||
local audit="$logs_dir/$job_id/events.ndjson"
|
||||
[ -f "$ev_log" ] || [ -f "$audit" ] || return 1
|
||||
EV_LOG="$ev_log" EV_AUDIT="$audit" EV_TOK="$tok" python3 -c '
|
||||
import json, os, sys
|
||||
tok = os.environ.get("EV_TOK", "")
|
||||
with open(os.environ["EV_LOG"], encoding="utf-8", errors="replace") as f:
|
||||
for line in f:
|
||||
try:
|
||||
e = json.loads(line)
|
||||
except Exception:
|
||||
continue
|
||||
if not isinstance(e, dict) or e.get("event") != "started":
|
||||
continue
|
||||
data = e.get("data") if isinstance(e.get("data"), dict) else {}
|
||||
if not tok or data.get("dispatch_token") == tok:
|
||||
sys.exit(0)
|
||||
|
||||
def lines(path):
|
||||
if not os.path.isfile(path):
|
||||
return
|
||||
with open(path, encoding="utf-8", errors="replace") as f:
|
||||
for line in f:
|
||||
try:
|
||||
e = json.loads(line)
|
||||
except Exception:
|
||||
continue
|
||||
if isinstance(e, dict):
|
||||
yield e
|
||||
|
||||
for e in lines(os.environ["EV_LOG"]):
|
||||
if e.get("event") != "started":
|
||||
continue
|
||||
data = e.get("data") if isinstance(e.get("data"), dict) else {}
|
||||
if not tok or data.get("dispatch_token") == tok:
|
||||
sys.exit(0)
|
||||
for e in lines(os.environ["EV_AUDIT"]):
|
||||
if e.get("event") != "received" or e.get("source_event") != "started":
|
||||
continue
|
||||
if not tok or e.get("dispatch_verified") is True:
|
||||
sys.exit(0)
|
||||
sys.exit(1)
|
||||
' 2>/dev/null
|
||||
}
|
||||
|
||||
@@ -279,6 +279,13 @@ class _Watcher:
|
||||
# that survives defensive parsing is recorded here, including ones a
|
||||
# different host published. This is the external-observer record that
|
||||
# backstops the publisher's own "published" line if it never wrote one.
|
||||
# `dispatch_verified` says this record is attributable to the job's
|
||||
# current dispatch: the token check above ran (the registry record had
|
||||
# a dispatch_token and the event's data.dispatch_token matched it) and
|
||||
# HMAC verification passed before that. The token itself is never
|
||||
# written - the audit redactor masks token-named keys - so this flag is
|
||||
# what a local reader (delegate_started_recorded) can trust for a
|
||||
# remote agent's `started`, which has no line in events.log.
|
||||
mqtt_common.append_event(jid, {
|
||||
"event": "received",
|
||||
"source_event": payload.get("event"),
|
||||
@@ -286,6 +293,7 @@ class _Watcher:
|
||||
"topic": msg.topic,
|
||||
"timestamp": payload.get("timestamp"),
|
||||
"detail": payload.get("detail", ""),
|
||||
"dispatch_verified": expected_disp_tok is not None,
|
||||
})
|
||||
self.events.put((msg.topic, payload))
|
||||
|
||||
|
||||
@@ -633,40 +633,33 @@ _disk_status() {
|
||||
# ISSUE-18: delivery is confirmed by the protocol, not the screen. After the
|
||||
# prompt was handed over (herdr accepted the submission, or the dispatch was
|
||||
# published for a remote session) the job's own `started` must arrive within
|
||||
# the ack timeout T: the registry leaves `pending` (the agent's publish_event
|
||||
# syncs it on this host), a started line lands in events.log, or the
|
||||
# subscriber prints a started line (a remote agent - no shared disk). No
|
||||
# `started` within T is a delivery failure: fail closed, never re-inject
|
||||
# (single-injection contract). The status is not pinned to this dispatch and
|
||||
# may come from an external writer; the terminal handling after this still
|
||||
# keeps such an error/cancelled as it is rather than turning it into success.
|
||||
# the ack timeout T. The evidence is lib.sh's delegate_started_recorded - the
|
||||
# same predicate create_session's onboarding ack uses, so the two waits share
|
||||
# one contract (ack-finding req. 2, review R1): the local publisher's line in
|
||||
# events.log, or the subscriber's HMAC-verified, dispatch-attributed audit
|
||||
# record of a `started` it received (a remote agent has no shared disk). The
|
||||
# registry status is not evidence - a record leaves `pending` when another
|
||||
# writer closes it (error, cancelled) or another dispatch claims it, none of
|
||||
# which says this agent saw this prompt. No `started` within T is a delivery
|
||||
# failure: fail closed, never re-inject (single-injection contract). A job
|
||||
# that is already terminal is handled by the terminal logic after this, not
|
||||
# turned into a success here.
|
||||
_wait_started() {
|
||||
local job_id="$1" ack_s="${2:-$ACK_TIMEOUT}" sub_log="${3:-}" deadline wall st
|
||||
local job_id="$1" ack_s="${2:-$ACK_TIMEOUT}" sub_log="${3:-}" deadline wall
|
||||
# sub_log is retained for callers' signatures; the subscriber's evidence is
|
||||
# read from its audit record, which is attributable, not from its console.
|
||||
deadline=$(( $(date +%s) + ack_s ))
|
||||
# The ack wait never outlives this dispatch's wall clock (START_TIME + TIMEOUT).
|
||||
wall=$(( ${START_TIME:-$(date +%s)} + ${TIMEOUT:-3600} ))
|
||||
[ "$deadline" -gt "$wall" ] && deadline=$wall
|
||||
# A loop job keeps one events.log across iterations: only a `started` that
|
||||
# carries THIS attempt's dispatch token counts (review R1). The predicate is
|
||||
# lib.sh's delegate_started_recorded - the same one create_session's
|
||||
# onboarding ack uses, so the two waits share one definition of evidence.
|
||||
local budget=$(( deadline - $(date +%s) ))
|
||||
while :; do
|
||||
st="$(_disk_status "$job_id")"
|
||||
if [ -n "$st" ] && [ "$st" != "pending" ]; then
|
||||
echo "inject: job $job_id advanced to '$st' in the registry; treating the prompt as delivered and continuing." >&2
|
||||
return 0
|
||||
fi
|
||||
# publish_event records `started` here before its wire publish and before
|
||||
# the status sync, so a slow broker cannot delay the acknowledgement.
|
||||
if MAM_JOBS_DIR="$REGISTRY_DIR" delegate_started_recorded "$job_id" "${_disp_tok:-}"; then
|
||||
# publish_event records `started` locally before its wire publish and
|
||||
# before the status sync, so a slow broker cannot delay the acknowledgement.
|
||||
if MAM_JOBS_DIR="$REGISTRY_DIR" delegate_started_recorded "$job_id" "${_disp_tok:-}" "${LOGS_DIR_ABS:-}"; then
|
||||
echo "inject: job $job_id recorded 'started' in events.log; treating the prompt as delivered and continuing." >&2
|
||||
return 0
|
||||
fi
|
||||
if [ -n "$sub_log" ] && [ -f "$sub_log" ] && grep -Eq '^[^ ]+ job=[^ ]+ seq=[^ ]+ started( |$)' "$sub_log" 2>/dev/null; then
|
||||
echo "inject: subscriber received started for job $job_id; treating the prompt as delivered and continuing." >&2
|
||||
return 0
|
||||
fi
|
||||
if [ "$(date +%s)" -ge "$deadline" ]; then
|
||||
ACK_WAITED_S="$budget"
|
||||
return 1
|
||||
|
||||
Reference in new issue
Block a user