fix(issue-30): review - the stall watcher's lifetime is bounded on both sides and it re-checks its dispatch before acting

Round-1 review of PR #41 (claude N1, codex R1/R2).

N1 - the wrapper stopped its stall watcher with `kill` followed by an
unbounded `wait`. A TERM that lands in the watcher's fork-exec window is
swallowed (bash defers terminating signals in the forked child until the
exec), the watcher then installs its own handler and lives until the job
closes, and the wrapper hung until someone killed the watcher by hand
(reproduced twice in test_issue14's ack test; the wrapper trace stops at
`kill` + `wait`).
R1 - a wrapper that is itself terminated left the watcher orphaned.
R2 - a `completed` or a re-dispatch that landed during a blocking herdr
wait (up to 60 s) was still followed by the idle notice and the nudge.

- multi-agent-mux-delegate-job: `stop_stall_watch` sends TERM, polls up
  to 5 s, then KILLs and reports "ignored TERM; killed"; `start_stall_watch`
  passes `--watch-pid $$`.
- lib_py/stall_watch.py: `--watch-pid` (rc 0 once that pid is gone, checked
  between herdr calls); `_dispatch_open()` re-reads the record after every
  wait and right before the nudge - terminal status, another dispatch token
  or attempt, or a missing record ends the run without a notice; a refused
  `stalled:idle` publish withholds the nudge (`_notify` returns False).
- delegate-job SKILL.md: the watcher's lifetime contract; exit-code rows 2
  and 3 name the idle clock and the wall-clock label; "before v4.7.5" ->
  v4.8.0 (M5, NB2).
- tests/test_issue30_stall_watch.py (M8): the module-wide Linux mark is
  narrowed to S6 (/proc); r4's stub stamps time.time() so its cross-process
  comparison holds on macOS too (47 passed, 1 skipped here).
- tests/test_issue30_stall_watch_stop.py (new, macOS and Linux): the stop
  routine against a TERM-ignoring stand-in returns in 4-12 s; the wrapper
  returns (rc 3) with a TERM-ignoring watcher shim; `--watch-pid` ends the
  watcher; a TERM to the real wrapper leaves no watcher; a job completed or
  re-dispatched during the wait is not nudged; a refused notice withholds
  the nudge. Every one of them hangs or fails on the round-1 code.

Not in this commit (follow-ups): M7 test hook in product code (no adapter
is unreliable, a seam is needed), M9/B3 public-broker latency (on this host
even `started` sometimes never reaches the subscriber), the leading-zero
budget parse (NB1) and M1/M2/M4/M6 go on the #17 item 4 list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
GodopuandClaude Fable 5.1 committed 2026-09-26 13:17:24 +09:00
1 parent 1673818bb5
commit af0a2a2557
5 files changed
+406 -12

No files matched your search

@@ -692,7 +692,7 @@ start_stall_watch() {
echo "stall watch: human job, no local pane - stall detection off" >&2; return 0
fi
local args=(--job "$JOB_ID" --registry-dir "$REGISTRY_DIR" --target "${session#herdr:}"
--agent "$agent" --herdr-session "${HERDR_SESSION_NAME:-}")
--agent "$agent" --herdr-session "${HERDR_SESSION_NAME:-}" --watch-pid "$$")
[ -n "$tok" ] && args+=(--expect-dispatch-token "$tok")
[ -n "$att" ] && args+=(--attempt-id "$att")
PYTHONPATH="$SCRIPT_DIR/..${PYTHONPATH:+:$PYTHONPATH}" "$PY" -m lib_py.stall_watch "${args[@]}" \
@@ -703,11 +703,23 @@ start_stall_watch() {
# A watcher that already ended on its own with an error (herdr unavailable)
# is not left silent: its last lines go to this wrapper's stderr.
# The stop is bounded: a TERM that lands in the watcher's fork-exec window is
# swallowed (bash defers terminating signals in the forked child until the
# exec), and an unbounded `wait` then hung this wrapper until someone killed
# the watcher by hand (PR #41 review N1). TERM, up to 5 s, then KILL.
stop_stall_watch() {
[ -n "${STALL_PID:-}" ] || return 0
local rc=0
local rc=0 i
if kill -0 "$STALL_PID" 2>/dev/null; then
kill "$STALL_PID" 2>/dev/null || true
for ((i = 0; i < 25; i++)); do
kill -0 "$STALL_PID" 2>/dev/null || break
sleep 0.2
done
if kill -0 "$STALL_PID" 2>/dev/null; then
echo "stall watch for $JOB_ID ignored TERM; killed (pid $STALL_PID)" >&2
kill -9 "$STALL_PID" 2>/dev/null || true
fi
wait "$STALL_PID" 2>/dev/null || true
else
wait "$STALL_PID" 2>/dev/null || rc=$?