- admission: a `submit --dry-run` record is marked dry_run and is not an
open job of its session (review 1e25cf2f F1)
- create: the monitor-row adoption requires the row's server to be the one
this create spawned on (review 1e25cf2f F2)
- pane lock: retry only on contention (BlockingIOError); the docs say the
flock lasts while any fd-9 holder (the claim shell and the shim) lives
(review 66680d9d)
- tests on macOS: real-herdr box HOMEs under /tmp (sun_path is 104 bytes),
resolved paths, the fake claude reads non-canonical input (MAX_CANON 1024)
and the pane is read unwrapped; lock holders exec their sleep so killing
them frees the lock; the adoption tests name the server
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
The live onboarding test asserted that the first job's wrapper registered in
the same wall-clock second as the onboarding it had just placed at a second
boundary. Process start-up has no sub-second contract, so the setup assertion
failed now and then before any dispatch behaviour was checked. The test now
stamps the onboarding record, which it owns and which nothing else depends on
yet, with the job's created_at once that record exists; the stamp is written
by atomic replace so a scanning wrapper never skips a half-written record.
The lifecycle assertions and the fixed-id tie coverage in test_u2h are unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
open_jobs reports a live pending record nobody signals on as `injected`:
its producer registered and injected it without a waiter (create
--onboard, a job sent by hand), so it holds the session like a running or
admitted job, and a waiter of the same second with a lower job id, or one
registered a moment earlier and still in its preflight, no longer passes
it in the FIFO. The waiter signals on its own record before its first
scan, so its record is never mistaken for one.
lib_py.pane_lock resolves the shim's ambient target (no -L,
HERDR_SESSION_NAME empty or `default`) to the server of the pane the
caller runs in - HERDR_SESSION, else HERDR_SOCKET_PATH's layout, as
lib.sh _herdr_caller_server reads them - so an ambient and a named call
into one server claim under one lock; outside a pane it stays the
default server's.
Tests: u2h (the tie, the later-registered onboarding, the signalled
control), u1/u4 labels, c10 forces the same-second registration, p1b,
s2 (real herdr: held lock stops both call forms; mixed concurrent calls
take distinct panes). Two FIFO fixtures now carry a waiter's signal.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The registry wait judged its record by one rule: not this attempt's
pending record in this session -> superseded (redispatched, moved or
closed by someone else). That also caught the record's own `started`
landing before the prompt went out - the Issue #14 registry ack winning
over an unreachable broker, as test_issue30_stall_watch_stop acknowledges
at registration - and the wrapper left on exit 5 instead of its
broker_unavailable path (rc 3).
A record that is running under this attempt's token in this session is
this attempt's own progress: the waiter answers `clear` at once, writes
no admission and starts no hold. Another token, another session or a
terminal status stay `superseded` (review 55219be1 R1 unchanged).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three `create --onboard` at once into one folder (the control root, so no
control-plane prefix reaches the pane and the shim reuses an idle pane):
the shim's new-session chooses a pane - an idle one in a workspace on the
same cwd, a split, a fresh workspace's root pane - and then claims it with
`agent start`; two shims could choose the pane one of them had just made,
and the later start failed with agent_pane_busy (one create in three).
mam_herdr now runs every new-session call holding one lock per server: a
flock on fd 9, opened for the block and taken by lib_py.pane_lock, which
also bounds the wait (MAM_PANE_CLAIM_WAIT_S, default 120). It is released
with the block or with the process, however it ends. The shim heredoc is
unchanged (S11). Other servers' creates are not held.
Serialising the claims exposed a second race: a create's end-of-run
`reconcile.sh --once` adopted the pane of another create that had not yet
written its row, and that create refused its own name as already running
(exit 4). Such a row - registered by the monitor, no job, for a pane
created after this create began - is now replaced, not refused.
Tests: real herdr - shim-level and create-level three-way races into one
folder (the real herdr's pane-creating replies held back so the window is
hit on every run; both fail on the base with the orchestrator's error),
the different-folder test kept; pane_lock unit tests (per-server file,
timeout, kill -9 holder, handover, knob); the adoption rule and its three
refusals; shim contract h26 (held lock: no pane calls). The herdr box gets
a ~/.zshrc: without it every first `agent start` typed into zsh's first-run
menu and timed out at 30 s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
R1: wait-open-jobs takes --dispatch-token (the token the wrapper read for
this attempt) and keeps it, with the session it asked about, for its whole
life; every write (signal, admission, the hold's renewal) is a guarded
engine merge that re-reads the record under the registry lock and refuses
once the record is not that attempt's pending one in that session. A record
redispatched, moved or closed by someone else makes the waiter answer
`superseded`: the bash wrapper exits 5 without a close, the record is its
new owner's. hold-admission is given the waiter's pinned identity instead
of reading a token from disk, so a hold that starts late holds nothing.
R2: a pending record that never carried a waiter's signal (create --onboard
registers and injects directly and waits ack_timeout_sec for `started`) is
live for that budget from created_at, not for the 30 s signal window; the
window applies only once a dispatcher has signalled.
Tests: u1 rows for the two pending lifetimes, u3b asserts the guard, u3f/u3g/u3h
(cross-session redispatch, in-place attempt change, token changed before the
waiter started) with the real waiter subprocess, u4e/u4f (late hold, running
hold released by a redispatch), s1 pins the token argument and the no-close
branch; real herdr C10 (onboarding valid before and after 30 s blocks the
session), C11 (expired onboarding is stale), C12 (redispatched waiter leaves
its record alone, exit 5).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
F1: `clear` is now an admission persisted on the released record - a merge
through the engine whose new `guard` re-reads the session's open jobs under
the registry lock, so two waiters cannot both be admitted. It is scoped to
the attempt by the dispatch token (a redispatch voids it) and reported as
`admitted` to every other waiter, which holds the session as a running job
does, whatever its FIFO place: a lower id in the same second or a
redispatched older loop job now waits.
F2: a pending record's liveness window is the dispatcher's signal cadence
times three misses (10 s x 3 = 30 s), no longer ack_timeout_sec, which the
agent's budget takes down to 1 s. After `clear` a detached hold renews the
admitted record's signal until the record leaves pending, its attempt
changes, the wrapper is gone, or the wall clock ends.
Tests: u1 boundaries, u1d/u2e/u2f/u2g/u3d/u3e for the admission, u4c/u4d for
the hold, s1 pins --hold; real herdr: C3/C7/C8 adjusted, C9 is C8's
live-owner counterpart with short ack windows.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The `--status pending` reset is a redispatch and issues a new dispatch
token, but the loop read `_disp_tok` only after the registry and idle waits.
A busy or idle_wait close from phase two on therefore carried the previous
phase's token and the engine refused it, leaving the record pending while
the caller saw session_busy. The read now follows the reset directly; s1
pins reset < token read < registry wait.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
F1: from phase two on, the loop's `_registry_wait` ran before the record was
reset to pending, so the waiter held a terminal record: the engine refused its
dispatch_wait_at merge and other waiters never saw it in the FIFO. The wait
(and the idle wait after it) now follows the `--status pending` update; s1
pins the order and u3c shows why a terminal self record cannot signal.
F2: the real-herdr box teardown survives a hung stop/delete, kills a server
bootstrapped by a late client call, and asserts the server and HOME are gone.
F3: the delegate-job SKILL.md exit 5 row names the registry wait.
Also drops the dead loop in s1.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The FIFO is (created_at, job_id) at second resolution; B and C registered
within the same second once in the full suite, C sorted first by id and
cleared while B's signal was still fresh, so the stale line never printed.
C7 already keeps the 1.2 s gap; C8 now does too and asserts the order.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merge-workspace review loop (--all-reviewer): round 1 (@a93c9bf) 4/4
PASS. Full suite at a93c9bf on macOS / Python 3.9 / herdr 0.9.3 with
WORKSPACE_ROOT set to an empty sentinel: 1752 passed / 0 failed / 38
skipped; the sentinel stayed empty, the review worktree had no .mam, and
the main checkout's shim was not touched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
test_b13_reexec_preserves_original_argv and test_b13_no_freeze_switch_
disables_reexec ran the checkout's run_loop.sh with --creator, which gets
past argument parsing to mam_acquire_loop_lock on the default marker,
$MAM_REAL_ROOT/.mam/loop-guard-active with MAM_REAL_ROOT = the script's own
checkout. In the main checkout that is the live loop's lock (taken and
released for a moment: a real loop refused, or the test refused); in a review
tree it left an empty .mam. Both now set MAM_LOOP_MARKER under their tmp_path.
A sweep of the 16 modules that use run_loop.sh (a per-test watcher for a new
.mam in the checkout, an empty .mam removed before each test) found only
these two; the others pin MAM_LOOP_MARKER already, end before the lock
(--help, argument errors), or run a copy of the tree.
Tests: test_loop_marker_isolation.py L1 runs every test that reaches the lock
through the checkout's run_loop.sh in a copy of the checkout and requires no
.mam there; W1 of the Issue #17 isolation test now also requires that no .mam
exists (it only looked for files, so the empty lock directory passed it). At
97c2946 both fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
Merge-workspace review loop (--all-reviewer): round 1 (@d198dd5) claude
agy PASS, grok codex NOT PASS (the same inherited WORKSPACE_ROOT in eleven
more modules; test_orc_onboard's import-time env); fixed in 8b47cc7.
Round 2 claude grok NOT PASS (test_o2_race_free_lock, test_o1_rebuttal
missed), agy codex PASS; fixed in ac48cb7. Round 3 (@ac48cb7) 4/4 PASS.
Full suite at ac48cb7 on macOS / Python 3.9 / herdr 0.9.3 with
WORKSPACE_ROOT set to an empty sentinel: 1751 passed / 0 failed / 38
skipped; the sentinel stayed empty and the main checkout's shim was not
touched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Two modules still ran the checkout's run_loop.sh - which sources lib.sh
before it parses an option - with the process environment and no root of
their own: test_o2_race_free_lock's test_o2_12 (review claude; it was in the
sweep's list and missed in 8b47cc7) and test_o1_rebuttal's
test_v3_rebut_option_parsing_and_budget, whose calls pass no env (review
grok; outside the env-copy grep). Both now use own_workspace_root, and W1
runs one node of each: at 8b47cc7 W1 fails with the copy's .mam/shim/herdr.
A last pass over the test files that run lib.sh, run_loop.sh, the wrapper or a
skill script: every one is marked, uses mam_sandbox or pins WORKSPACE_ROOT,
or does not source lib.sh (b7 diff_collect, deploy, issue42, issue45 config,
issue5_phase0, local_mqtt).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
d198dd5 fixed one helper; the same defect was in eleven more modules
(reviews grok and codex R1/R2): their shells inherited the process
environment, so each `source lib.sh` rewrote <WORKSPACE_ROOT>/.mam/shim/herdr
- the checkout's own when the variable is unset or names this checkout (which
pytest_configure keeps), another workspace's under --noconftest.
- conftest: own_workspace_root, an opt-in fixture setting WORKSPACE_ROOT to
<tmp_path>/.mam-test-state/workspace. Not autouse: runner.py and
mqtt_common read WORKSPACE_ROOT in-process, and some tests unset it on
purpose. mam_sandbox and explicit envs run later and still win.
- pytestmark usefixtures("own_workspace_root") in test_version_consistency,
test_b19_headless_reconcile_fixes, test_b8_send_keys_verification,
test_issue6_fail_closed, test_loop_cli, test_issue5_phase3_loop,
test_issue5_phase4_runtime, test_o3_scoped_guard, test_issue25_plan_reuse,
test_orc_onboard (their shells inherit os.environ, many with no env at all).
- test_orc_onboard (R2): SCRUBBED_ENV is taken at import and no longer
carries WORKSPACE_ROOT; every call builds _scrubbed_env(root) with the root
of the call (ws / cwd / mam_sandbox, else this test's).
- test_workspace_scope (unittest) and test_issue21_worktree_prereqs._mutate
pin the root they already create.
Tests (test_issue17_workspace_root_isolation.py): W1 copies the checkout and
runs one test of each of those modules there with WORKSPACE_ROOT naming the
copy - the copy gets no .mam; W2 imports test_orc_onboard under a sentinel
root, then calls it under another - the sentinel stays empty. At d198dd5 both
fail (the copy gets .mam/shim/herdr; the sentinel gets it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
_env kept an inherited WORKSPACE_ROOT, so every helper that sources the
wrapper (lib.sh's _init_herdr_isolation) rewrote <that workspace>/.mam/shim/
herdr - another workspace's during the PR #51 review (v4.8.7 review, claude).
Only dropping it is not enough: lib.sh then derives the checkout itself, and
conftest keeps a WORKSPACE_ROOT that names this checkout, so the run would
rewrite the checkout's own shim (the real one when this is the main
checkout). _env now sets WORKSPACE_ROOT to the test's tmp dir.
Test: with a sentinel WORKSPACE_ROOT in the environment, _run_both and a
direct _env run leave the sentinel empty and the checkout's shim untouched,
and the shell sees the tmp dir. With the old _env the sentinel gets
.mam/shim/herdr.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
Merge-workspace review loop (--all-reviewer): round 1 (@8241bd5) claude
grok PASS, agy NOT PASS (two static wrapper-literal tests did not follow
_decided/_exit_for), codex NOT PASS (R1: a yes/no decision that could not
run read as "no"); fixed in 541c3fe and 65b78b6. Round 2 (@65b78b6) 4/4
PASS. Full suite at 65b78b6 on macOS / Python 3.9 / herdr 0.9.3: 1730
passed / 0 failed / 56 skipped; 18 of the skips were the uninitialized
nats-docker submodule in the fresh worktree, and with it
tests/test_deploy_freshness.py is 31 passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
subscriber-up and stall-watch answered only by exit code, so a call that
could not run (module error rc 1, missing interpreter 127) read as "no":
with a subscriber that had ended 0, the error path mapped 0 to completed
and the submit ended with 0; a failed stall-watch silently skipped the
watcher.
Both now take --print: the answer is printed as yes/no and the exit code
is 0 for either. The wrapper asks with --print through _decided_yes, so a
nonzero exit, no answer or anything else is _decision_failed (ERROR line,
exit 1). Without --print the rc contract is unchanged (yes 0, no 1).
Tests: subscriber-up (direct and loop/discuss call sites) and stall-watch,
each failing by traceback, rc 127 or no answer with a subscriber that
ended 0, end the submit with 1 and the ERROR line - at 8241bd5 eight of the
nine ended with 0 (the loop/no-answer one ended with 1 through the loop's
own result and failed only on the ERROR line); yes and no keep their
branches in both paths; --print unit table; the static guard forbids an
rc-read yes/no decision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
a670ef5 routed the wrapper's sub-status and exit-code decisions through
_decided / _exit_for, but test_tier1_unit G-10 and test_issue5's
test_wrapper_exit_code_mapping_literals_are_frozen still looked for the raw
`_decide sub-status "$sub_rc"` and `_decide exit-code "$job_status"`. They now
pin `_decided job_status sub-status "$sub_rc"` and `_exit_for "$job_status"`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
Sourcing the wrapper sources lib.sh, which writes $WORKSPACE_ROOT/.mam/shim/
herdr; without WORKSPACE_ROOT that is the checkout. The new N2 wait-rc test
now runs with its tmp dir as WORKSPACE_ROOT, so the N2 tests leave the
checkout untouched. (The existing C tests in this file still write the
checkout's .mam/shim/herdr, as at 56d0bfd - not changed here.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
mam_int / mam_int_knob call `python3 -I` because the file is also sourced
without lib.sh and so without $PY. The header now says so, and that without
a python3 on PATH they print nothing (rc 127). The code is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
"Stdlib only" is true of the module, not of how the wrapper runs it:
`python -m lib_py.delegation.wrapper` imports lib_py/delegation/__init__.py
first, which loads the service and paho-mqtt. The docstring now says so; the
wrapper's $PY already needs paho. The run path is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
`exit "$(_decide exit-code ...)"` became `exit ""` when wrapper.py could
not answer (its module or $PY broken): rc 255 on bash 3.2. The other printed
decisions failed open in their own ways: a failed wait-rc returned "" (255),
a failed finalize-begin left finalize-end with an empty --final (argparse
rc 2), and a failed sub-status or finalize-end ended under set -e with the
interpreter's own code (1 for a traceback, 127 for one that cannot run).
Printed decisions now go through _decided <var> <subcommand>, and the
exits through _exit_for <status>: a failed call, an empty answer or a
non-numeric exit code / wait rc prints one ERROR line and exits 1. finalize-
end may legitimately answer nothing (its wrapper gone), so only its rc is
checked. The normal path is unchanged: same answers, exit codes and output.
Left as they are: subscriber-up and stall-watch are answered by rc, so a
failure reads as "no" - the subscriber is then treated as dead (the error
path, whose sub-status/exit-code are now fail-closed) and the notify-only
stall watcher is not started.
Tests (F): the real submit (human job, unreachable broker, 3 s budget) with
a $PY that fails one decision - exit-code, sub-status, finalize-begin,
finalize-end; traceback, rc 127 or no answer - ends with 1 and the ERROR
line; the same run without a failure still ends with 3. cmd_wait with a
failing wait-rc ends with 1. A static guard keeps printed decisions off
raw $(...). At 56d0bfd: exit-code 255/255, finalize-begin 2/2, wait-rc
255/255, sub-status and finalize-end 127 for "gone"; the traceback and
no-answer cases there already ended with 1 (by set -e or by the empty
status mapping to 1) and fail only on the missing ERROR line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
With the cwd premise fixed, test_b still failed on herdr 0.9.3: a pane whose
saved directory is gone is restored as a placeholder - `restore_error`
"Saved directory is unavailable...", no process - and `pane process-info`
answers pane_not_found. close_restored_shells raised on it and exited 1
before looking at the other panes, so the resumed session's removed-worktree
pane (the case #45 closes on herdr 0.9.1, where it is a shell in $HOME) was
left behind.
Such a pane has nothing running, so it now meets condition 4. Only that
exact answer counts - restore_error set AND pane_not_found; any other
process-info failure still stops the run with exit 1 as before.
Tests: test_issue45_restore.py passes on herdr 0.9.3 (5/5). Unit tests with a
stub herdr: the placeholder is closed (fails at 56d0bfd); pane_not_found
without restore_error, or restore_error with another error, still raises.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
test_b_only_the_resumed_sessions_restored_shell_is_closed asserted that a
pane whose directory was removed comes back in $HOME. That is herdr 0.9.1;
herdr 0.9.3 keeps the old path. The cwd check now takes either, with a note
on which version does what; closing never looked at the cwd.
herdr_restore.py's docstring said the same "$HOME when that directory is
gone" and now names both versions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
Merge-workspace review: round 1 (@518beda) claude agy grok PASS, codex
NOT PASS (finalize wait measured on the wall clock); fixed in 4c86107.
Round 2 delta (@4c86107) claude agy grok PASS; codex (24b21b9c, after
817d1650 closed deadline_timeout with no report) found R1 closed and no
new defect, but withheld PASS only because its sandbox could not bind
the suite's loopback broker. The same two-file suite run here at
4c86107: 278 passed. Full suite at 518beda on macOS / Python 3.9:
1711 passed / 1 failed / 38 skipped; the failure
(test_issue45_restore, herdr 0.9.3 cwd restore) also fails on main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
codex, PR #51 round 1. wrapper._sleep computed the span and the remainder
with time.time(). The broker re-observation (up to 3 s) and the disk-heartbeat
grace (1 s) use it, so a wall clock stepped back during the wait stretched it
by the step: on a mocked clock stepped back 10 s, an already completed job was
seen after 11 s / 13 s instead of 1 s / 3 s. The old bash ran `sleep 1` /
`sleep "$left"`, which do not follow the wall clock.
_sleep now measures on time.monotonic(). The deadline (START_TIME + TIMEOUT),
the evidence window and event timestamps stay on the wall clock (_now), as in
the bash.
test_u_a_wait_is_not_stretched_by_a_wall_clock_step_back (1 s, 3 s) swaps the
module's time for a clock stepped back 10 s during the first sleep; on the
previous code it measures 11.0 / 13.0 and fails. The comparison test and the
finalize-related behaviour tests (issue33, issue18, issue14) on macOS /
Python 3.9: 321 passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
kill(pid, 0) succeeds for a zombie. A caller that captures the wrapper's
output and calls communicate() after TERM leaves the wrapper unreaped while
the finalizer holds its pipes, so the owner check passed until the deadline
and the job was still closed (codex, round 2; reproduced with the real
registry). _alive now asks `ps -o stat=` (POSIX, no /proc) once kill(0)
succeeds and treats state Z as gone, as tests/conftest.py::_pid_alive does;
no ps answer keeps kill(0)'s verdict. The owner is looked at every 0.5 s.
Tests: TERM + communicate() with no wait() for both graces (stand-ins) and
for the timeout grace with the real registry, old vs new; a zombie is not
alive. All four fail on c5cebe6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
finalize-end ran the disk grace and the broker re-observation as a child
of the wrapper; TERM to the wrapper left it running, and at the deadline it
still closed the job and published the error (codex R1, reproduced with the
real registry). The bash loops it replaced ended with the wrapper.
The wrapper now passes its own pid (--watch-pid "$$", as for the stall
watcher). finalize-end checks it every 0.2 s through both waits and before
the close and the event, and leaves without either once it is gone.
A comparison test sends TERM to the old and the new wrapper inside each
grace: both exit at once, leave no process behind and make no call.
Also: no blank line at the end of the old-decisions fixture, and the #42
inventory follows main() one line down.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The wrapper keeps spawn / wait / kill, key injection and the herdr
diagnostic probe; what it decided in bash is now lib_py/delegation/wrapper.py,
reached through one `_decide` helper:
- finalize_unfinished: the clock choice, the disk grace and broker
re-observation, the close request / label, the close and the event
(finalize-begin / finalize-end; bash runs only the busy probe between them)
- start_stall_watch: whether to watch (MAM_STALL_WATCH, remote, human)
- _map_sub_rc, _rc_for (+ MAM_DELEGATE_FAIL_CLOSED), the subscriber
early-exit check and cmd_wait's rc normalisation
- mam_int / mam_int_knob run lib_py/ints.py, which is also the runner's
_positive_int: one implementation of the Issue #26 rule
No behaviour change: tests/test_issue17_item4_decisions.py runs the old
bash functions (copied verbatim from 7209750) and the new wrapper on the
same inputs and compares output, FINAL_JOB_STATUS and the registry /
publish calls. Three static tests that pinned the old bash literals now
pin the Python entry points instead.
Refs #17
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #48 already carries the 4.8.5 lockstep bump. Merge-workspace review:
round 1 (@35935f9) claude agy grok NOT PASS (test portability: tomllib on
Python 3.9, /proc and /tmp on macOS), codex no verdict (model at capacity,
budget spent); fixed in 36f0b8d and e8d9151. Round 2 (@e8d9151) codex NOT
PASS (unrunnable config check passed an unchecked merge, odd pids closed a
pane, nan dropped the user's settings); fixed in ab93776. Round 3
(@ab93776) 4/4 PASS. Full suite at ab93776 on macOS / Python 3.9: 1456
passed / 0 failed / 38 skipped. Reports and the round-3 identity sidecars
go to .agents/reports/<session>/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
codex round 2 (PR #48).
- R1: when `herdr config check` could not run (timeout, OSError), the
merged config was handed to the server as checked. A merge with a schema
error makes herdr drop the whole file, restore back on - the #45 failure.
The merged body is no longer used unchecked: _write_checked falls to the
fixed MINIMAL config with a warning. When the check cannot run on MINIMAL
either, MINIMAL is still used (with its own warning) rather than no
config: returning none would leave restore on, and MINIMAL is a fixed
two-line file the real-herdr test accepts. Unknown non-zero exits stay as
they were - herdr exits non-zero for harmless warnings (an unknown
section) that do not drop the file.
- R2: _shell_in_foreground compared shell_pid and the foreground group with
==, so 0/0, False/False, ""/"" or -1/-1 closed the pane. Both must now be
positive ints (bool excluded) before they are compared.
- R3: the re-parse check used ==, and nan != nan, so a valid TOML nan threw
away every other user setting even on Python 3.11. _same_doc compares
documents recursively with nan equal to nan.
Tests: R1 (timeout, OSError) and R3 (top-level, array, nested nan) in
test_issue45_herdr_config.py; R2 in the new tests/test_issue45_restore_unit.py
(a herdr stub: zero, false, empty, negative, null, string, true, float;
missing info; a real shell closed; a busy pane kept). On the previous code
12 of them fail. Python 3.11: 67 passed; Python 3.9: 33 passed / 34 skipped
(tomllib-only). Real-herdr restore tests and related files on 3.9: 197
passed / 1 skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
claude round 1, B2 and B3 (B1 was the previous commit).
- B2, tests/test_issue45_restore.py: the throwaway root is resolved, because
on macOS /tmp links to /private/tmp and herdr reports real paths (two B
tests compared /tmp/... with /private/tmp/...). The server's environment
is read from /proc on Linux and from `ps -E` elsewhere (macOS has no /proc;
two A tests failed with FileNotFoundError).
- B3, tests/test_issue9_server_env.py: the bootstrap builds the config with
the python3 on PATH, which here is 3.9 without tomllib, so the test's
merged-settings expectation could not hold and its own `import tomllib`
failed. The test now parses the result with that same python3; when it
has no tomllib, it expects the key-only config and the bootstrap's
"no tomllib" warning instead.
macOS, Python 3.9 venv: test_issue45_restore, test_issue9_server_env and
test_issue45_herdr_config 42 passed / 29 skipped (the tomllib-only checks).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
tests/test_issue45_herdr_config.py imported tomllib at module level. On
Python 3.9 - macOS's /usr/bin/python3, which the venvs and the bootstrap's
python3 use here - the import failed during collection and pytest ran no
test of the whole suite.
The import is optional now, and the 29 tests that parse a result with
tomllib or expect the user's settings kept are marked needs_tomllib: on
Python < 3.11 herdr_config deliberately writes the key-only config (restore
still off, the other settings dropped, a warning on stderr), which
test_without_tomllib_restore_is_still_off covers on every Python.
Python 3.9: 22 passed / 29 skipped; Python 3.11: 51 passed.
create SKILL.md (same line, so the #42 inventory is unchanged) and the
4.8.5 entry state that building the server config needs tomllib (Python
3.11+) in the python3 on PATH, and what an older python3 loses. Checked
with Python 3.9 in a throwaway HOME: the warning reaches stderr (the
bootstrap captures only stdout) and the user's file is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
The fallback wrote <.mam>/herdr-server-config.toml unchecked, so it could
replace a foreign file there, the source itself, or a link to it. It now goes
through output_path like the normal path (source identity incl. aliases,
generated-for-this-source ownership, then a per-source name) and writes
nothing when no name is safe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R1: tomllib accepting a file is not herdr accepting it (a schema error makes
herdr drop the whole file, restore included). The shim passes REAL_HERDR and
the written file must pass `herdr config check` ("parse error"/"using defaults"
reject; unknown keys and sound-file notices do not), else the key-only file is
written and checked. A non-table `session` and a source that is not UTF-8 reach
that fallback instead of raising.
R2: the line edit is replaced by parse -> set -> serialize; the result must
parse back to the source plus the key, so dotted keys and inline tables keep
the user's other settings.
R3: the output is never the source (aliases included) nor a file not generated
for the same source; then a per-source name beside it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue #45 A+B: MAM's isolated herdr servers start with restore-on-restart off,
and resume closes the session's own restored empty shell. Not reviewed yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shim runs a bare python3. Before 3.11 the module failed to import and the
server started with restore on; now it gets the key alone, with a warning.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a restart with restore off, the session's saved pane comes back as an
empty shell (in $HOME when its directory is gone) holding the agent_session it
reported. A resume with a CP_PREFIX cannot reuse it, so it stayed next to the
new pane. Just before spawning, resume closes panes whose agent_session is the
id being resumed, with no live agent and only the shell in the foreground; the
registry is not consulted, since a session whose server died stays running.
tests/test_issue45_restore.py drives a real herdr server under a throwaway HOME
and session with a fake claude: restore stays off through the MAM bootstrap
(with a restore-on control), the close conditions, and create -> crash ->
resume leaving one pane.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
S11 keeps the generated .mam/shim/herdr from changing by accident; #45 changes
it on purpose, so its base moves to ed73f67.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The headless bootstrap regenerates the server config on every start and hands
it to the server as HERDR_CONFIG_PATH. Without a file the server still starts,
with a warning. mam_sandbox unsets HERDR_CONFIG_PATH/XDG_CONFIG_HOME so the
file lands in the sandbox HOME.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lib_py/herdr_config.py builds the user's herdr config (HERDR_CONFIG_PATH, else
XDG_CONFIG_HOME, else ~/.config, as herdr reads it) with only
[session] resume_agents_on_restore forced to false, written next to the source
so relative paths resolve the same. The text merge is re-parsed; a merge that
does not leave the rest equal falls back to the key alone, since herdr drops a
config that does not parse and would restore again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The .claude/skills check accepts both local layouts and fails on a dangling
link. Review: round 1 (@e1ed270) claude agy grok PASS, codex NOT PASS (a
dangling top-level link was skipped); fixed in 150b493; round 2 4/4 PASS.
Reports and the round-2 identity sidecars go to .agents/reports/<session>/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Path.exists() follows the link, so a dangling top-level .claude/skills read
as "absent" and the test skipped: every skill's load path could be broken
with the suite still green (codex R1 on PR #47; the exists() branch predates
this PR).
The check moves into claude_skills_problems(link, skills_dir, names):
absence is "neither exists nor is a symlink"; a symlink must resolve to
.agents/skills; a directory must hold each skill resolving to the matching
.agents/skills/<skill>. test_claude_skills_layouts runs it on synthetic
layouts (absent, single link, per-skill links, a link to the wrong skill, a
dangling top-level link, a dangling per-skill link), so it is exercised in
a worktree too. Replaying the review's probe (a dangling .claude/skills in
the tree) now fails; the main checkout's per-skill layout reports nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
test_claude_skills_link_resolves_to_agents_skills assumed .claude/skills is
one symlink to .agents/skills. It is local and git-ignored, and a checkout
may instead hold a directory with one link per skill (this workspace's
layout), where every link is correct but the test failed. PR #46's reviews
and suites ran in worktrees, where the directory is absent and the test
skips, so the failure only showed in the main checkout after the merge.
The test now accepts a single link to .agents/skills, or a directory whose
entry for each of the eight skills resolves to the matching
.agents/skills/<skill>. Checked against no link (skip), per-skill links
(pass; the old test fails), one link pointing at the wrong skill (fail) and
a single link (pass).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Issue #42 stages 1-2: the SKILL.md code-block inventory as test data and
the removal of blocks the entry scripts already do. Merge-workspace review:
round 1 (@8d26106) claude agy grok PASS, codex NOT PASS (R1-R3); fixed in
17eba09 (docs, inventory data and the guard test only); round 2 4/4 PASS.
Full suite at 8d26106 1408 passed / 0 failed. Reports and the round-2
identity sidecars go to .agents/reports/<session>/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Merge-workspace review of PR #46 (codex R1-R3 and C1, claude O1 and O4).
- R1: resume never attaches, and the server it exports does not reach the
calling shell; the native herdr binary does not route on environment
variables. The attach advice and the Verification block now name the
server: `herdr --session <server> agent attach <name>`, with <server> the
row's herdr_session.
- R2: --dry-run in resume and create skips the spawn and the registry
update, but the checks before it still run and on macOS remove the
quarantine attribute from the agent binary. The option rows say so.
- R3: test_no_deleted_body_came_back compares every current block with the
deleted bodies by exact sha and by a whitespace-insensitive fingerprint
(stored as base_fingerprint), independently of the inventory, so a
deleted block cannot return by being listed again under class c.
Replaying the review's probe (the deleted resume Workflow re-added and
inventoried as c) fails the new test and passes the old one.
- C1 / O1: the base create call block at 263-266 (class c) folded into the
Workflow call is recorded under "superseded"; the 37 base blocks are all
accounted for.
- O4: create exit 42 also covers MAM_KILL_ON_READY_TIMEOUT=1; stop's no-op
exit 0 does not apply with --purge-conversation.
- The inventory's positions and shas are regenerated for the edited blocks;
every classification is unchanged.
No product code changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
raised == passthrough forced the declaration, but nothing checked the
declared code against the table: dropping create's exit 4 from both the
table and the data still passed. Assert passthrough <= table codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry update after the kill chain raises SystemExit(1) when the row
disappeared during the run (stop_session.sh:310), so "exit 1 changed nothing"
was false on that path. The stop table now says so, the code is declared as a
passthrough, and the test requires every integer SystemExit in an entry
script's heredocs to be declared - the check that would have caught it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 1: tests/data/skill_code_blocks.json lists all 37 fenced blocks of the
eight SKILL.md files at base 45d1621 (indented fences count), classed (a)
duplicate of a script, (b) procedure no script offers, (c) example/call/
diagram, with script file:line evidence and recorded mismatches.
tests/test_issue42_skill_blocks.py fails when a block changes without the
inventory, an (a) block comes back, a removed block's entry call is missing,
an exit-code table drifts from the script, or evidence points nowhere.
Stage 2: the five (a) blocks are gone - resume Workflow, stop pre-flight,
create pre-flight / Workflow / manual YAML append - replaced by the
entry-point call with option and exit-code tables. Each exit table has a
catch-all row: under set -e a failing sub-step's code passes through.
create's exit 4 is real (the YAML write raises SystemExit(4) for a duplicate
running row) and is listed as such. Doc-to-script mismatches are fixed in the
doc (resume Case 3 / no attach, stop --purge-conversation --yes, status
--json keys, create cmd_full and the nonexistent yaml-edit script). No
behaviour change, no version bump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #41/#43 review follow-ups F1-F6, implemented by creator-claude-01 in a
loop and reviewed by claude, agy, grok and codex: round 1 (@95eabc6)
codex NOT PASS (a failed env load cached as loaded), fixed in 75f364e;
round 2 4/4 PASS. Full suite at 75f364e 1380 passed / 0 failed. Reviewer
reports (round 2 with identity sidecars) and the creator's reports go to
.agents/reports/<session>/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
ensure_env_loaded() set _env_loaded before _load_dotenv() ran. With F2,
load_env_or_report() turns the EnvFileError into a return value, so a second
main() in the same process skipped the unreadable file and ran unconfigured
(codex R1: registry.main twice -> rc 1 then rc 0 with a job registered; a
repaired file was never read). The flag is now set only after a successful
load: a failure is retried on the next call, a success still loads once.
envfile parses the whole file before applying, so a failed load sets nothing.
MESSAGING.md: the one-line ERROR is the Python entry points'; the wrapper
adds its own "ERROR: cannot load <path>" line (codex, minor).
Test: in one process, registry.main twice on an undecodable file is rc 1
both times, prints the ERROR line twice and registers nothing; the repaired
file is applied on the next call; a later change is not re-read. At 95eabc6
it fails with rc [1, 0], one job registered and the value unset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
5edf913 gave job_subscriber.main a bare `return 1`; its exit codes are
returned by name (test_issue5 test_job_subscriber_cli_contract_snapshot).
The value stays 1 - what the EnvFileError traceback exited with before -
and is RC_ERROR. The wrapper reads the same file first and exits 1 itself,
so it never starts a subscriber that would report this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
delegate-job SKILL.md said a watcher outlives a killed wrapper by "one
wait, <= 60 s". That bound is the herdr wait's; a wrapper that goes while a
notice is being published is noticed when that publish ends. The text now
says both (PR #41 codex round 3).
"a wrapper gone during the wait" switched alive() by call count, so the
watcher saw the wrapper gone at the next loop head and never entered the
second wait (ticks == [1] with that band). The wrapper now vanishes inside
the second herdr wait (_during_notice's in_wait hook), and the test checks
that the second wait happened and that alive() was true until then. The
same test fails when the post-wait _wrapper_gone() check is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
stall_watch read MAM_TEST_UNRELIABLE_AGENT to pretend an adapter's
agent_status was unreliable (PR #41 M7). The branch is gone. S13 now runs
the real stall_watch.main() in a process where the adapter itself reports
agent_status_reliable = False (_UNRELIABLE_MAIN), and checks the same
things as before: no idle notice or nudge, the blocked notice, and the
"idle judgement off" log line. A static check keeps MAM_TEST_ out of the
watcher (fails at 0051fd6). Without the seam S13 fails on this tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
A non-numeric MAM_READY_EMPTY_GIVEUP, SKS_EMPTY_GIVEUP, SKS_IDLE_RETRIES,
SKS_IDLE_RETRY_BACKOFF_S, MAM_LOCK_STEAL_RETRIES, MAM_DIFF_MAX_BYTES,
MAM_DIFF_MAX_LINES or RECONCILE_POLL_INTERVAL was a `[` error or a sleep
error at the point of use (PR #43 review claude R3). They now go through
mam_int_knob: decimal, a warning and the default for a bad value. Minimums
are in test_issue26_ints.py INT_KNOBS: 1 for the give-up counts (0 acts as
1), the diff caps (0 truncates every diff) and the poll interval (0 is a
busy loop); 0 for the retry counts (one attempt) and the retry backoff.
SKS_QUIESCENT_INTERVAL stays raw: its default is 0.5 s and sleep's
fraction is its meaning; NOT_ARITHMETIC says so.
loop_lock.sh and diff_collect.sh also run without lib.sh (the Python runner
and the tests source them alone), so mam_int / mam_int_knob move verbatim
to lib_int.sh, which lib.sh sources and those two source when the function
is missing. remove.sh's no-manifest fallback list names the new file
(install copies every file under .agents/skills). test_issue21_loop_b l15,
which builds a control root holding only diff_collect.sh, copies it too.
Tests: the table guard now reads every shell file; scripts sourced alone
read their knobs as decimal; _pane_quiescent with SKS_EMPTY_GIVEUP=abc
gives up with a warning and no `[` error (14 of the new cases fail at
0051fd6).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
MAM_WAIT_GRACE_S silently fell back to 300 on anything but bare digits
(" 30 " too), MAM_CHILD_WAIT_S went through float() (abc raised ValueError,
0.5 and 1e3 were taken) and MAM_JOB_TREE_MAX through int() (abc and an
empty value raised). All three now go through runner._int_knob, which is
_positive_int's rule (PR #43 review grok): decimal, blanks around allowed,
an invalid value warns once and the default is used. Minimums:
MAM_WAIT_GRACE_S >= 0 (wait exactly the budget), MAM_CHILD_WAIT_S >= 0
(look once; test_issue34 uses 0), MAM_JOB_TREE_MAX >= 1 (a cap of 0 calls
every tree "over"; not waiting is MAM_CHILD_WAIT_S=0).
The loop SKILL.md now names MAM_WAIT_GRACE_S in the budget row and the
sub-job paragraph (PR #41 M6), with the knobs' integer rules.
Tests: RUNNER_INT_KNOBS in test_issue26_ints.py holds the minimums; the
real wait_for_descendants is driven per knob, and a static check ties the
call-site minimums to the table (15 of the new cases fail at 0051fd6; the
14 that pass are valid values whose result did not change).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
PR #43 review (grok):
(a) A UTF-8 byte-order mark at the start of the file made the first line an
"invalid key", so its setting was dropped. The one parser (envfile.py,
used by the wrapper and mqtt_common) now reads with utf-8-sig.
(b) A key given twice keeps its first value (a later line does not replace
it). The behaviour is unchanged; .mam.env.example's FORMAT section and
envfile.py's docstring now say so.
(c) MESSAGING.md said a MAM_ENV_FILE naming no file logs an error and makes
connections fail closed. Both loaders skip it silently and refuse the
ambient search; a file that exists but cannot be read is the error
(EnvFileError, one ERROR line and rc 1). The text now says that.
Tests A7: BOM in both loaders, first-value-wins in both loaders plus the
FORMAT text, and a missing explicit file loading neither it nor the ambient
.mam.env without an error plus the MESSAGING text (all three fail at 0051fd6).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
A .mam.env that exists but cannot be read or decoded stopped the Python
entry points with an EnvFileError traceback, while the wrapper printed one
`ERROR:` line and exited 1 (PR #43 review claude S1). The mains that load the
env file - publish_event, registry, job_subscriber, cli, bench, heartbeat and
remote - now call mqtt_common.load_env_or_report(), which prints
`ERROR: <path>: cannot read (<exception type>)` and lets the main return 1.
No value or file content is printed. ensure_env_loaded() still raises: no
library function exits the process.
Test: every entry point, for a 000 file and an undecodable one, ends with
exactly that line and rc 1 and creates nothing (14 cases; all 14 fail at
0051fd6 with a traceback).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
`ps -o lstart=` pads a one-digit day ("Oct 1"); loop_lock.sh's mam_lstart
squeezes the spaces before comparing, but A8 wrote the raw value, so the
live holder's marker looked stale and the second loop was not refused on
the 1st-9th of every month. The test now normalises like the product (the
product is unchanged) and also runs with a `ps` shim that always prints a
padded lstart, so the padded case is checked on every date.
The other tests that record a live lstart (test_o3_scoped_guard
_get_lstart) already normalise; test_o2's markers carry fixed stale values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
Issue #26: one .mam.env parser for the wrapper and the Python tools,
decimal integer knobs, tests on a local MQTT broker. Merge-workspace
review: round 1 (@b58edac) claude agy PASS, codex NOT PASS (R1-R4), grok
no verdict (its late report, archived as report-c9ed5b14.md, found R1 too);
round 2 (@0eec795) 4/4 PASS. Reports and round-2 identity sidecars go to
.agents/reports/<session>/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Merge-workspace review of PR #43 (codex R1-R4, claude R1/R2).
- R1: envfile.parse raises EnvFileError when the file exists but cannot be
read or decoded as a whole; `--shell` prints the cause and exits 1, so the
wrapper stops (rc 1, as before #26) instead of running on defaults, and
mqtt_common's entry points stop on the same error. A bad single line is
still a warning. The wrapper's message no longer claims a read failure
when the cause is elsewhere ("cannot load ... see the line above").
- R2: lib.sh mam_int spells out the ASCII blanks and digits instead of
[[:space:]]/[0-9], which follow the locale: U+2003 30 U+2003 was 30 in the
wrapper and invalid in the runner under en_US.UTF-8. The test oracle
_expected uses the same ASCII rule.
- R3: an integer flag given explicitly with an empty value is refused like
any other invalid value; only an absent flag keeps its default.
- R4: a line holding a NUL byte is skipped with a warning by both loaders;
before, the wrapper read "leftright" and the Python loader died.
- .mam.env.example states the format: no expansion, `export` and bash's own
variables skipped, a trailing `# ...` is part of the value, the environment
wins, an unreadable file is an error.
Tests: unreadable (0000) and invalid-UTF-8 files stop the wrapper and the
Python loader; a NUL line is skipped by both; Unicode blanks/digits are
invalid for mam_int, _positive_int, the wrapper and the runner under C and
en_US.UTF-8; explicit empty values for the five integer flags are refused.
All of them fail on b58edac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Review (fa45b0e6): paho's connect() does not wait for the CONNACK, so C0
could read protocol_levels before the broker had handled the second
CONNECT, and the `before` slice could take in a previous test's connection.
- _Client waits for CONNACK (and UNSUBACK, instead of sleeping 0.3 s);
C0 checks the levels of its own two client ids (stats() levels_by_client).
- The local broker now routes and stores a PUBLISH before its PUBACK, so a
publisher holding the ack may rely on the retained state it asked for
(late subscriber replay, retained delete).
- test_c0_holds_when_the_broker_is_slow runs C0 with CONNECT/UNSUBSCRIBE
delayed and PUBLISH routed late; the old C0 fails it ([] == [4, 4]), as do
an ack-before-route broker and a fixed-sleep unsubscribe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
34241d7 added a line near the top of send_keys_safe (SKS_QUIESCENT_TRIES
normalisation), which pushed `agent prompt` from inside the 2500-character
window to offset 2519. The order the test guards is unchanged
(_pane_quiescent < _pane_dialog_open < agent prompt); it now slices up to
the function's closing brace.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R1 (P1): a quoted export still ran code when the key was a bash special
variable (RANDOM=a[$(cmd)] is evaluated arithmetically on assignment).
envfile.RESERVED (bash's own variables, plus BASH_/COMP_/READLINE_
prefixes) is skipped with a line-numbered warning by both loaders; a test
keeps it covering every variable a clean bash defines. The wrapper also
hands SCRIPT_DIR/REPO_ROOT/TARGET_ENV to the parser as environment, so the
file cannot replace the variables it has already set.
R2 (P2): SKS_QUIESCENT_TRIES reached `for ((i < tries))` through a function
argument; it is normalised at the call site (minimum 0). B5 now requires
every integer-looking env default in the shell code to be normalised or
listed in NOT_ARITHMETIC with where it goes, and exercises _pane_quiescent.
R3 (P2): _positive_int strips leading zeros and rejects more than nine
significant digits before int(), like mam_int; 5000-digit inputs no longer
raise in the runner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tests/_local_mqtt.py: in-process MQTT 3.1.1 broker on 127.0.0.1 (what
the repo uses: protocol level 4, clean session, QoS 0/1, retained).
- conftest: one session broker named by MQTT_BROKER/MQTT_PORT/MQTT_TLS;
other redirecting/auth MQTT_* dropped and restored afterwards.
- Harnesses of the 22 broker-dependent tests (#18 acceptance1/2/3/5,
#6 t1_1/2/7/8/14/15/16/20, #19 MQTT loop x2, #33 b3/r3/b7, #30
s9/s10/r2) use it; the "broker unreachable" skips are gone (a local
broker that does not deliver is a failure). acceptance1/5 and #19 also
assert on the subscriber, runner and audit logs the real producers write.
- tests/test_local_mqtt.py: conformance with the real clients and the real
publish_event -> job_subscriber path (C0), an AST guard against public
broker literals and MQTT_* deleted without a replacement (C2), and the
session's broker config (C3).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bash read MAM_JOB_TIMEOUT_S=0700 as octal 448 (and failed on 0900) while
the loop runner read 700. One rule now, with a per-knob minimum:
lib.sh mam_int / mam_int_knob and runner._positive_int (blanks allowed,
digits only, decimal, min <= n <= 999999999).
- minimum 1: MAM_JOB_TIMEOUT_S, MAM_HEARTBEAT_SEC, MAM_HEARTBEAT_MISSES,
MAM_ACK_TIMEOUT_S (invalid env: warning + default) and the wrapper's
--timeout/--idle-timeout/--heartbeat/--heartbeat-misses/--ack-timeout
(invalid: rc 1); the runner's job_budget, --job-timeout, contract_args.
- minimum 0 (0 = check once): MAM_INJECT_ACK_GRACE_S,
delegate_wait_started's ack_s, SKS_DIALOG_TIMEOUT, SKS_BUSY_TIMEOUT,
SKS_DELIVERY_GRACE_S.
- tests/test_issue26_ints.py: INT_KNOBS is the one table of minimums; a
static guard keeps env defaults out of arithmetic and checks each
normalised knob against it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lib_py/delegation/envfile.py (stdlib only) holds the one set of rules:
comments and blank lines skipped, key and value stripped, one matching
quote pair removed, no shell interpretation, keys must match
^[A-Za-z_][A-Za-z0-9_]*$ (bad or `=`-less lines warn with the line
number, never the value).
- mqtt_common._parse_env_file delegates to envfile.apply.
- The delegate-job wrapper runs envfile.py --shell by path and evals the
shlex-quoted exports; values never reach a command line. The environment
now wins over the file in both loaders.
- .mam.env.example: commented assignments no longer use $HOME, which the
parser keeps literally.
The shell parsed the env file in one place only (wrapper :38), not three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue #33 (job budget, wall_clock_timeout vs deadline_timeout) and
Issue #30 (notify-only stall watcher). Reviews of PR #41 in the merge
workspace: round 1 (@1673818) claude N1 / codex R1 R2 NOT PASS, agy grok
PASS; round 2 (@af0a2a2) grok N1 / codex D1 D2 NOT PASS, claude agy PASS;
round 3 (@9200562) 4/4 PASS. Reports and the round-3 identity sidecars go
to .agents/reports/<session>/. The design consensus on items A-D is in the
VERSIONS.md entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
grok N1: the wrapper spent up to 5 s on a TERM budget before it finalized the
job, so when the watcher swallowed the TERM the silent job's close came ~5 s
late and a `completed` written in that gap won (test_issue14 idle-grace).
codex D1: `_notify` blocks on a publish for up to 120 s, and the prompt went
out right after it; a completed, a re-dispatch or an unreadable record in that
window still got the nudge. codex D2: the wrapper pid was checked only at the
loop head, so a watcher whose wrapper vanished during a wait could still
notify and nudge once.
- multi-agent-mux-delegate-job `stop_stall_watch`: KILL + wait, no budget.
The watcher holds nothing to flush (its TERM handler is a bare os._exit),
and KILL cannot be swallowed.
- lib_py/stall_watch.py: `_wrapper_gone()` after every wait (before any
notice) and at the loop head; `_may_prompt()` right before the keys - the
wrapper is alive and the record, read then, is open and this dispatch;
`_dispatch_open(strict=True)` treats an unreadable record as "no".
- SKILL.md: the lifetime contract states the one-wait lag and that nothing is
sent after the wrapper is seen gone.
- tests/test_issue30_stall_watch_stop.py: the stop returns at once against a
TERM-ignoring stand-in (took 6 s on the round-2 code); completed,
re-dispatch and a torn record during the notice publish withhold the nudge;
a wrapper gone during the wait gets no notice and no nudge. All five fail
on the round-2 code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
Round-1 review of PR #41 (claude N1, codex R1/R2).
N1 - the wrapper stopped its stall watcher with `kill` followed by an
unbounded `wait`. A TERM that lands in the watcher's fork-exec window is
swallowed (bash defers terminating signals in the forked child until the
exec), the watcher then installs its own handler and lives until the job
closes, and the wrapper hung until someone killed the watcher by hand
(reproduced twice in test_issue14's ack test; the wrapper trace stops at
`kill` + `wait`).
R1 - a wrapper that is itself terminated left the watcher orphaned.
R2 - a `completed` or a re-dispatch that landed during a blocking herdr
wait (up to 60 s) was still followed by the idle notice and the nudge.
- multi-agent-mux-delegate-job: `stop_stall_watch` sends TERM, polls up
to 5 s, then KILLs and reports "ignored TERM; killed"; `start_stall_watch`
passes `--watch-pid $$`.
- lib_py/stall_watch.py: `--watch-pid` (rc 0 once that pid is gone, checked
between herdr calls); `_dispatch_open()` re-reads the record after every
wait and right before the nudge - terminal status, another dispatch token
or attempt, or a missing record ends the run without a notice; a refused
`stalled:idle` publish withholds the nudge (`_notify` returns False).
- delegate-job SKILL.md: the watcher's lifetime contract; exit-code rows 2
and 3 name the idle clock and the wall-clock label; "before v4.7.5" ->
v4.8.0 (M5, NB2).
- tests/test_issue30_stall_watch.py (M8): the module-wide Linux mark is
narrowed to S6 (/proc); r4's stub stamps time.time() so its cross-process
comparison holds on macOS too (47 passed, 1 skipped here).
- tests/test_issue30_stall_watch_stop.py (new, macOS and Linux): the stop
routine against a TERM-ignoring stand-in returns in 4-12 s; the wrapper
returns (rc 3) with a TERM-ignoring watcher shim; `--watch-pid` ends the
watcher; a TERM to the real wrapper leaves no watcher; a job completed or
re-dispatched during the wait is not nudged; a refused notice withholds
the nudge. Every one of them hangs or fails on the round-1 code.
Not in this commit (follow-ups): M7 test hook in product code (no adapter
is unreliable, a seam is needed), M9/B3 public-broker latency (on this host
even `started` sometimes never reaches the subscriber), the leading-zero
budget parse (NB1) and M1/M2/M4/M6 go on the #17 item 4 list.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
- job_subscriber.py returns only named codes (RC_COMPLETED 0, RC_ERROR 1,
RC_NO_EVENT 2, RC_BROKER_UNAVAILABLE 3, RC_WALL_CLOCK 4). The wrapper
mirrors them as SUB_RC_* and compares subscriber codes only by name; a test
keeps the mirror equal to the Python constants and forbids numeric
comparisons of $sub_rc/$sub_exit and numeric `return` in the subscriber.
- `delegate-job wait` passed the subscriber's code straight through, so a
spent budget would have reached its callers as 4; it is 2 again (B7 fails
without the mapping).
- tier1 G-8 gave both clocks 0.5 s and raced; the wall clock now runs out
first by construction (idle 30 s) and the test asserts RC_WALL_CLOCK.
Other subscriber rc asserts use the names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- input_region: in the prompt-line shape the input box starts only at a
column-0 prompt (codex indents a draft's continuation lines), and the
prompt and the (whole-line) placeholder are stripped from the box's first
non-blank line only, so `› Explain this terminal prompt:\n ›` is not empty.
- test_r4 runs with S=40 again: the defective code nudged ~30 s after a
missed transition, which S=10 could not tell from the fixed ~10 s.
Both new checks fail against the defective versions (stall_watch.py at
47e4274 -> 30.24 s < 40; input_region.py at bf0f5c1 -> nudge sent) and pass
now, each with its passing twin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- R1 (P1): without rules the input region runs from the last prompt line to
the end of the pane; only a last line matching the adapter's footer pattern
(codex: ` <model> · ~/<cwd> · …`) is dropped, so a draft on a second line
keeps the nudge withheld.
- R2: while the synchronous submit runs, a relay thread finds the job it
registers (same session, created since the submit) and prints its
stalled:/stall_watch: notices; before, they appeared only after the job.
- R3: the broker-unavailable disk wait that runs into the deadline is the wall
clock: wall_clock_timeout (caller rc stays 3).
- R4: a state_change_seq change seen right before the judgement restarts the
idle window (decision 9: S seconds of continuous idle).
Regression tests are the reviewer's reproductions, each with its passing
twin; the wrapper-level #30 tests now use this environment's broker instead
of the #18 harness's public default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lib_py/stall_watch.py runs beside the wrapper's subscriber for a local herdr
job (started after delivery, stopped when the subscriber ends; MAM_STALL_WATCH=0
turns it off). It follows `herdr agent wait` transitions and publishes only
`progress` events through one publisher; it has no path that closes a job.
- blocked: `stalled:blocked <detection line>` once per episode, never answered.
- idle for MAM_STALL_IDLE_S (600) with no background descendants of the agent
pid (pane process-info + /proc; adapter resident_children and the heartbeat
timer excluded) and an empty input box: `stalled:idle` and one nudge.
- input box judged from adapter facts measured on herdr 0.9.1 (codex '›',
grok frame '│', opencode '┃' block + stop pattern); the whole-pane fallback
is gone. No facts: withheld + `stall_watch:input_check_unavailable` once;
never empty for 3 x S: `stall_watch:input_never_empty` once.
- herdr failures are not "no event": warning + `stall_watch:herdr_unavailable:
<code>` (or agent_not_found), backoff, rc 2 after MAM_STALL_HERDR_FAIL_S
(120); the wrapper prints the watcher's last lines.
- remote and human jobs are skipped with one log line.
- agent_status_reliable is an adapter field; false turns idle judgement off.
- the loop runner relays stalled:/stall_watch: notices once each.
- scoped_env drops a pane-inherited workspace scope for name lookups (#34 rule).
The module sits at the top of lib_py: delegation and loop may not import the
agent adapters (their sibling-import rules). Idle pane captures of the five
CLIs are fixtures; the mock herdr answers process-info in the real shape.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- wrapper: TIMEOUT falls back to MAM_JOB_TIMEOUT_S (--timeout > env > 3600);
an invalid value warns and keeps 3600.
- loop: --job-timeout <sec> (run_loop.sh passes it on; refused in the legacy
shell loop). The runner always hands the wrapper --timeout, so the budget
the wrapper registers is the one its wait caps come from.
- runner: wait_for_job and the MAM_CHILD_WAIT_S default are budget + 300 s
(MAM_WAIT_GRACE_S), today's 3900 with no option; an explicit
MAM_CHILD_WAIT_S still wins.
- subscriber: the wall-clock budget returns 4 (RC_WALL_CLOCK), silence keeps
2 (RC_NO_EVENT). The wrapper maps both to caller rc 2 and labels the close
from the rc: wall_clock_timeout vs deadline_timeout; the disk-heartbeat
grace that ends on the deadline is the wall clock too.
- registry close accepts wall_clock_timeout (engine: -> error); the remote
runner's deadline and command-timeout details lead with wall_clock_timeout;
the sub-job table files it CLOSED-UNCONFIRMED (first token, colon stripped).
Test expectations changed only where the contract changed: the loop's
contract_args now starts with --timeout, the subscriber's exit-code docstring
names rc 2/4, and the registry --help capture lists the new close kind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue #34: the loop waits for the creator job's sub-jobs, reviewers get
the sub-job table as evidence, a run with sub-jobs voids a round whose
source changed during review, and a pane caller reaches an agent of
another workspace on its own server. VERSIONS.md carries the entry; the
merge-side reports (round 1 @ed293dc and round 2 @e80186f with identity
sidecars, base ed293dc -> head e80186f, evidence review-tree) are archived
under .agents/reports/<session>/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
remote.py closes a job whose end it could not see with
`error --detail "deadline_timeout (remote runner)"`; the sub-job table
matched the reason exactly and filed it as a plain failure, losing the
unconfirmed count and the late-write warning (merge review, codex R2).
The kind is the first word of the reason now. Test T14 replays the
remote runner's close through the real registry and producer; it fails
without the fix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
- registry.get_feedback only returned the detail of the last completed or
error event, so a sub-job the wrapper cancelled before injection
(target_not_found, session_busy) reached the review table as a bare
`cancelled` and its reason was lost (merge review, grok N1). get_feedback
takes the event kinds to read (default unchanged); the table asks for
cancelled too and shows NOT-DISPATCHED with the reason. Test T13, both
reasons, fails without the fix.
- the [귀속되지 않은 열린 잡] line listed jobs registered without
--parent-job but gave the reviewer no rule for them - the one case the
table cannot show (claude K1). A rule sentence follows the line whenever
such a job exists; without one the prompt is unchanged. T6 pins it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
- mam_herdr: for name lookups (agent / has-session / kill-session) a
HERDR_WORKSPACE_ID that is the caller pane's own workspace (it matches
the HERDR_PANE_ID prefix) is dropped. Live agent names are unique per
server (herdr --skill), so this cannot address a same-named agent of
another workspace; an id the caller chose, or one set outside a pane,
still scopes (H-18/H-21/H-23). The generated shim is unchanged (S11).
- wrapper: a preflight that cannot see the target closes the job
cancelled with detail target_not_found instead of session_busy; the
loop's sub-job table counts it as NOT-DISPATCHED.
- tests: T10-T12 with the measured pane env (same-server socket, caller
w6, target w2); T8 follows the sub-job-armed guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without --worktree/--base the guard fingerprinted the source on every
loop, so a loop whose source holds untracked files that change on every
job voided each round: test_tier4_e2e scenario 5 keeps the mock herdr's
state file (rewritten on each herdr call) in its repository and failed
with rc 1 on 3fdbb62 (passed on 7a4a0c2). A loop without sub-delegation
now behaves as before (issue #34 requirement 3); with sub-jobs the guard
protects against a deadline-closed sub-job still writing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GodopuandClaude Opus 5.5