fix(issue-30): review - the stall watcher's lifetime is bounded on both sides and it re-checks its dispatch before acting
Round-1 review of PR #41 (claude N1, codex R1/R2). N1 - the wrapper stopped its stall watcher with `kill` followed by an unbounded `wait`. A TERM that lands in the watcher's fork-exec window is swallowed (bash defers terminating signals in the forked child until the exec), the watcher then installs its own handler and lives until the job closes, and the wrapper hung until someone killed the watcher by hand (reproduced twice in test_issue14's ack test; the wrapper trace stops at `kill` + `wait`). R1 - a wrapper that is itself terminated left the watcher orphaned. R2 - a `completed` or a re-dispatch that landed during a blocking herdr wait (up to 60 s) was still followed by the idle notice and the nudge. - multi-agent-mux-delegate-job: `stop_stall_watch` sends TERM, polls up to 5 s, then KILLs and reports "ignored TERM; killed"; `start_stall_watch` passes `--watch-pid $$`. - lib_py/stall_watch.py: `--watch-pid` (rc 0 once that pid is gone, checked between herdr calls); `_dispatch_open()` re-reads the record after every wait and right before the nudge - terminal status, another dispatch token or attempt, or a missing record ends the run without a notice; a refused `stalled:idle` publish withholds the nudge (`_notify` returns False). - delegate-job SKILL.md: the watcher's lifetime contract; exit-code rows 2 and 3 name the idle clock and the wall-clock label; "before v4.7.5" -> v4.8.0 (M5, NB2). - tests/test_issue30_stall_watch.py (M8): the module-wide Linux mark is narrowed to S6 (/proc); r4's stub stamps time.time() so its cross-process comparison holds on macOS too (47 passed, 1 skipped here). - tests/test_issue30_stall_watch_stop.py (new, macOS and Linux): the stop routine against a TERM-ignoring stand-in returns in 4-12 s; the wrapper returns (rc 3) with a TERM-ignoring watcher shim; `--watch-pid` ends the watcher; a TERM to the real wrapper leaves no watcher; a job completed or re-dispatched during the wait is not nudged; a refused notice withholds the nudge. Every one of them hangs or fails on the round-1 code. Not in this commit (follow-ups): M7 test hook in product code (no adapter is unreliable, a seam is needed), M9/B3 public-broker latency (on this host even `started` sometimes never reaches the subscriber), the leading-zero budget parse (NB1) and M1/M2/M4/M6 go on the #17 item 4 list. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
This commit is contained in:
1 parent
1673818bb5
commit
af0a2a2557
5 files changed
+406
-12
No files matched your search
@@ -692,7 +692,7 @@ start_stall_watch() {
|
||||
echo "stall watch: human job, no local pane - stall detection off" >&2; return 0
|
||||
fi
|
||||
local args=(--job "$JOB_ID" --registry-dir "$REGISTRY_DIR" --target "${session#herdr:}"
|
||||
--agent "$agent" --herdr-session "${HERDR_SESSION_NAME:-}")
|
||||
--agent "$agent" --herdr-session "${HERDR_SESSION_NAME:-}" --watch-pid "$$")
|
||||
[ -n "$tok" ] && args+=(--expect-dispatch-token "$tok")
|
||||
[ -n "$att" ] && args+=(--attempt-id "$att")
|
||||
PYTHONPATH="$SCRIPT_DIR/..${PYTHONPATH:+:$PYTHONPATH}" "$PY" -m lib_py.stall_watch "${args[@]}" \
|
||||
@@ -703,11 +703,23 @@ start_stall_watch() {
|
||||
|
||||
# A watcher that already ended on its own with an error (herdr unavailable)
|
||||
# is not left silent: its last lines go to this wrapper's stderr.
|
||||
# The stop is bounded: a TERM that lands in the watcher's fork-exec window is
|
||||
# swallowed (bash defers terminating signals in the forked child until the
|
||||
# exec), and an unbounded `wait` then hung this wrapper until someone killed
|
||||
# the watcher by hand (PR #41 review N1). TERM, up to 5 s, then KILL.
|
||||
stop_stall_watch() {
|
||||
[ -n "${STALL_PID:-}" ] || return 0
|
||||
local rc=0
|
||||
local rc=0 i
|
||||
if kill -0 "$STALL_PID" 2>/dev/null; then
|
||||
kill "$STALL_PID" 2>/dev/null || true
|
||||
for ((i = 0; i < 25; i++)); do
|
||||
kill -0 "$STALL_PID" 2>/dev/null || break
|
||||
sleep 0.2
|
||||
done
|
||||
if kill -0 "$STALL_PID" 2>/dev/null; then
|
||||
echo "stall watch for $JOB_ID ignored TERM; killed (pid $STALL_PID)" >&2
|
||||
kill -9 "$STALL_PID" 2>/dev/null || true
|
||||
fi
|
||||
wait "$STALL_PID" 2>/dev/null || true
|
||||
else
|
||||
wait "$STALL_PID" 2>/dev/null || rc=$?
|
||||
|
||||
Reference in new issue
Block a user