fix(issue-50): the waiter and its hold are pinned to one attempt and session; an unsignalled pending record lives by its ack budget (review 55219be1 R1, R2)

R1: wait-open-jobs takes --dispatch-token (the token the wrapper read for
this attempt) and keeps it, with the session it asked about, for its whole
life; every write (signal, admission, the hold's renewal) is a guarded
engine merge that re-reads the record under the registry lock and refuses
once the record is not that attempt's pending one in that session. A record
redispatched, moved or closed by someone else makes the waiter answer
`superseded`: the bash wrapper exits 5 without a close, the record is its
new owner's. hold-admission is given the waiter's pinned identity instead
of reading a token from disk, so a hold that starts late holds nothing.

R2: a pending record that never carried a waiter's signal (create --onboard
registers and injects directly and waits ack_timeout_sec for `started`) is
live for that budget from created_at, not for the 30 s signal window; the
window applies only once a dispatcher has signalled.

Tests: u1 rows for the two pending lifetimes, u3b asserts the guard, u3f/u3g/u3h
(cross-session redispatch, in-place attempt change, token changed before the
waiter started) with the real waiter subprocess, u4e/u4f (late hold, running
hold released by a redispatch), s1 pins the token argument and the no-close
branch; real herdr C10 (onboarding valid before and after 30 s blocks the
session), C11 (expired onboarding is stale), C12 (redispatched waiter leaves
its record alone, exit 5).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
GodopuandClaude Fable 5.1 committed 2026-10-11 02:58:16 +00:00
1 parent 9cbe08e8de
commit 30c25f193c
5 files changed
+394 -104

No files matched your search

@@ -104,7 +104,7 @@ Under fail-closed operation (`MAM_DELEGATE_FAIL_CLOSED=1`, default), the CLI ret
| `2` | Timeout | `error` (`deadline_timeout`) | Subscriber idle timeout (K x N without an event; the total budget is the wall-clock row below). **R1 liveness grace (Issue #18)**: after the subscriber idles out the wrapper keeps re-observing the disk while the job's events.log shows an event at most K x N (`--idle-timeout`) old - the automatic heartbeat counts - up to the wall-clock `--timeout`. herdr status and the pane are logged as a diagnostic only. |
| `2` | Wall clock | `error` (`wall_clock_timeout`) | The budget (`--timeout`) ran out, events or not (Issue #33; was `deadline_timeout` before v4.8.0). |
| `3` | Broker Error | `error` (`wall_clock_timeout` once the budget is spent, else `deadline_timeout`) | Broker unavailable (`MQTT_BROKER_UNAVAILABLE`) and terminal status not found on disk during deadline. |
| `5` | Session Busy | `cancelled` (`session_busy`) | Precondition failed / `session_busy`: Preflight check (`preflight_agent`) or TUI idle handshake wait failed, or the session's open jobs in the registry did not end within `MAM_DISPATCH_WAIT_S` (Issue #50) (prompt never delivered, retryable). |
| `5` | Session Busy | `cancelled` (`session_busy`) | Precondition failed / `session_busy`: Preflight check (`preflight_agent`) or TUI idle handshake wait failed, or the session's open jobs in the registry did not end within `MAM_DISPATCH_WAIT_S` (Issue #50) (prompt never delivered, retryable); or the record was redispatched or closed by someone else while it waited (then this wrapper leaves it untouched: no close, no `cancelled`). |
In all execution paths, the final line of standard output prints the persistent audit log directory path (`<logs_root>/<job_id>`), fulfilling the parsing contract for orchestrator runners.
@@ -358,7 +358,7 @@ EOF
# open jobs - an onboarding still running, a job sent by hand - end
# first, in order; herdr's agent_status is the screen, read after)
if [ "$AGENT" != "human" ] && ! _is_remote "$AGENT_SESSION"; then
_registry_wait "$JOB_ID" "$AGENT_SESSION"
_registry_wait "$JOB_ID" "$AGENT_SESSION" "$_disp_tok"
local idle_rc=0
wait_for_agent_idle "$AGENT_SESSION" || idle_rc=$?
export _SKS_IDLE_RC="$idle_rc"
@@ -540,7 +540,7 @@ EOF
local cur_agent="$AGENT"
[ "$_phase" = "reviewer" ] && cur_agent="$REVIEWER"
if [ "$cur_agent" != "human" ] && ! _is_remote "$current_session"; then
_registry_wait "$JOB_ID" "$current_session"
_registry_wait "$JOB_ID" "$current_session" "$_disp_tok"
local idle_rc=0
wait_for_agent_idle "$current_session" || idle_rc=$?
export _SKS_IDLE_RC="$idle_rc"
@@ -844,20 +844,29 @@ _within_deadline() {
wait "$pid"
}
# _registry_wait <job_id> <session> (Issue #50)
# _registry_wait <job_id> <session> <dispatch_token> (Issue #50)
# Waits, by the registry alone, until no live open job of <session> remains
# and this job is the earliest pending one (lib_py.delegation.wrapper
# wait-open-jobs: FIFO, the waiter signals on its own record, a dead record is
# ignored). A wait that runs out (MAM_DISPATCH_WAIT_S, default 600 s, bounded
# by this job's wall clock) closes this job as cancelled/session_busy and
# exits 5, like the preflight and the idle wait.
# ignored). The waiter is pinned to this attempt's token and session (review
# 55219be1 R1): a record redispatched, moved or closed by someone else while
# it waited is `superseded` - this wrapper exits 5 without touching it, the
# record is its new owner's. A wait that runs out (MAM_DISPATCH_WAIT_S,
# default 600 s, bounded by this job's wall clock) closes this job as
# cancelled/session_busy and exits 5, like the preflight and the idle wait.
_registry_wait() {
local job_id="$1" sess="$2" res
local job_id="$1" sess="$2" tok="$3" res
_decided res wait-open-jobs --job "$job_id" --agent-session "$sess" --registry-dir "$REGISTRY_DIR" \
${tok:+--dispatch-token "$tok"} \
--wait-s "$(mam_int_knob MAM_DISPATCH_WAIT_S 600 0)" ${START_TIME:+--start-time "$START_TIME"} \
--timeout "$TIMEOUT" --watch-pid "$$" --hold
case "$res" in
clear) return 0 ;;
superseded)
echo "ERROR: job $job_id was redispatched, moved or closed by someone else while it waited for session '$sess'; this attempt is not dispatched and the record is left as it is" >&2
local logs_root="${DELEGATE_JOB_LOGS_DIR:-$WORKDIR/delegate_job_logs}"
echo "$logs_root/$job_id"
exit 5 ;;
busy*)
echo "ERROR: session '$sess' still has open jobs in the registry (${res#busy }); job $job_id is not dispatched" >&2
finalize_unfinished "$job_id" "error" "$SUB_RC_ERROR" "registry_wait" "$sess"