engine.transition is now the only writer of job status. It implements the target
table in state-transitions.md §3: claim, non-terminal metadata, explicit
terminals, idempotent re-receipt, refusal of conflicting or late signals with an
audit entry, stale-attempt rejection, and deadline timeout.
Two write paths become one. update_job_status merged fields unconditionally;
pick_pending hand-rolled a correct pending->running swap plus its own audit
mirror. Neither was wrong alone, but a rule added to one silently missed the
other, which is the shape that produced the phase 1 transport bugs. Both keep
their signatures and their preserved legacy bodies behind MAM_ENGINE_CAS=0.
A merged `status` value is routed to the request that owns it, so a raw field
write can no longer slip past the table. Getting that wrong first was caught by
the characterization tests: a metadata merge was overwriting the caller's status
with the pre-merge value.
Lock discipline is now enforced, not described. The lock covers a decision and
one record write; audit mirroring is queued and flushed after release, carrying a
monotonic transition_seq so ordering survives the move. A test watches for
subprocess, socket and sleep calls while the lock is held and asserts none.
Structural correction to my own plan, caught by a phase 1 guard: the plan put the
engine in lib_py/loop, but delegation would then have had to import a sibling,
which is exactly what mixes trees under a frozen loop. The engine belongs with
the registry writers, so it lives in lib_py/delegation and loop consumes it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
Fixes what the job-status writers do today, so routing them through a single
transition boundary has an oracle instead of a promise. Same ordering that paid
off in phase 1.
X-02 states the claim path's real contract rather than the one the plan first
assumed: pick_pending filters on pending under the lock, so it is already a
correct swap. It duplicates the chokepoint, it does not bypass its safety. The
test pins the properties that must survive routing -- only pending is claimed,
only one session wins, and a session never sees another's work.
Every test is parametrized over MAM_ENGINE_CAS so the legacy body and the engine
body must agree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
Three reviewers examined fc0bd96..8def235. Codex and Claude each reproduced a
defect; all three findings were re-verified here before being accepted, and none
were rebutted.
R1 (codex) — the shell helpers ran `python -m lib_py.delegation.cli`. For -m,
sys.path[0] is the working directory, which therefore beats PYTHONPATH: with cwd
inside a tree containing lib_py, the live package won even with the frozen path
listed first. The documented B-13 guarantee for this path did not hold.
Reproduced, then replaced with an explicit bootstrap that inserts SKILL_DIR
itself. cwd is left alone, because the registry and audit-log paths are relative
to it. The file-based shim path was never affected.
R2 (codex) — the persistent connection was reused on host and port alone. Two
jobs sharing an endpoint but carrying different credentials or TLS settings
silently published over the first connection. Reproduced (the client factory ran
once, under the first account, with TLS off). The guard now compares an
immutable snapshot of the whole connection identity and names only the differing
field, never its value.
F-1 (claude) — paho queues a publish issued while disconnected and raises
RuntimeError from wait_for_publish. RuntimeError is deliberately outside the
retry tuple, so the one state a persistent connection exists to survive, the gap
between a drop and paho's reconnect, failed on the first attempt. Confirmed
against paho 2.1.0. That single condition is now translated into a retryable
error; the tuple is unchanged, so genuine programming errors are still not
retried.
Two fixtures were hiding these: FakeInfo had no rc and never raised. It now
mirrors paho's contract, which is what makes the new tests meaningful.
Also folds in the duplication both Claude (O-1) and I had flagged: the CLI path
kept its own copy of connect-publish-disconnect plus its own timeout constants.
_publish_once now delegates to OneShotPublisher.publish_raw, so a transport fix
lands once. Having two copies is exactly why these bugs existed in one of them.
Documentation corrected rather than defended: the benchmark's rationale for
keeping three forks was imprecise (the || true already guarantees rc 0; the
lookup buys quietness and legacy parity), SKILL.md no longer claims PYTHONPATH
decides the frozen path, and the daemon's directory creation is recorded as a
deliberate difference. Trailing whitespace inherited by the relocated file is
stripped, so the commit range is now clean too.
Verified: each regression fails against the pre-fix code and passes after
(R1 1, R2 4, F-1 1). 66 phase-1 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
Found while self-reviewing phase 1. PersistentPublisher held the connect
handshake result on the instance, so a refused attempt left its reason code
behind for the next one. On retry the wait loop saw a non-None value straight
away and never waited for the new connection's CONNACK, which is wrong in both
directions:
* a healthy second connection was rejected using the first attempt's rc, and
* once a late callback had written a success rc, a subsequent connection that
never acknowledged could be accepted on that stale value, letting a publish
proceed on a socket that was not ready.
Both need an asynchronous CONNACK to reproduce, which is exactly how paho
delivers it; the earlier fake called the callback inline and hid it. The
handshake state is now a local the callback closes over, so it can only ever
describe the connection that produced it, and a client that fails the handshake
is torn down instead of leaked.
No production path is affected today: the CLI and shell helpers publish once
per process and use OneShotPublisher. The defect mattered because the phase 2a
engine is meant to build on this prototype, which is where a transient refusal
would have become a stuck publisher.
Also tightens PV-1 to check the bound tree against pytest's real basetemp
rather than sniffing for "pytest-" in the path.
Verified: the three new regressions fail against the previous implementation
(2 of 3, the third covers the healthy path) and pass against this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
Splits publish_event.main into three pieces with no behaviour change:
prepare_event load the job, consume the seq, build the signed payload,
decide retain. The seq is burned here, before any transport
runs, so a failed publish can never leave a number a replayed
packet could reuse.
record_outcome audit record, registry event log and status sync -- all three
best effort, all three regardless of the publish result. That
is exactly the contract G-1/G-2/G-4 pin down, now in one place.
main parse, prepare, publish with retry, record.
publisher.py holds the two transports behind one interface. OneShotPublisher is
the current behaviour. PersistentPublisher keeps a single connection and lets
paho reconnect underneath, raising on an unacknowledged publish so the caller's
retry policy stays the only thing deciding attempt counts. It refuses an event
for a different broker rather than publishing to the wrong place.
Nothing in production uses the persistent path: the CLI and shell helpers exit
after one publish. The consumer is the phase 2a engine, which pays the CONNACK
cost on every event today -- this job's own started event took ~12.5s because
of one CONNACK timeout plus reconnect.
bench.py measures both arms. The fake transport is deterministic and is what
the tests assert on; the real-broker arm is informational, since the PoC
default is a public broker whose latency is not ours to control.
Verified: N publishes cost N connections one-shot and 1 persistent; the two
transports leave field-for-field identical registry and audit records including
the HMAC signature and retain flag; an unacknowledged publish retries on the
same connection and records published=false; a second broker is refused. No
test opens a socket.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
Adds the python side of the delegation helpers and switches lib.sh over to it.
daemon.py port of start_watchdog. Same monitor path, same pgrep pattern, so
a daemon started by either implementation is found by the other;
same stderr wording and exit 1 when reconcile.sh is missing; same
"print the PID" contract, multiple pgrep matches included, since
the shell echoed them all and trimming would be a silent change.
start_new_session detaches it, and SIGHUP is ignored across the
exec so it outlives its parent exactly as nohup did.
cli.py thin router. submit/publish forward to the existing main() of the
module that already owned the behaviour, so exit codes, stdout
shape and side effects cannot drift between the shell path and
the script path. Only ensure-daemon carries logic of its own.
service.py DelegationService facade, with an optional persistent connection
for the phase 2a engine. CLI and shell always use one-shot.
lib.sh keeps the three legacy bodies verbatim as _*_legacy behind
MAM_DELEGATION_PY=0, which is the instant rollback switch. PYTHONPATH is passed
explicitly so `-m` resolves lib_py from the tree this lib.sh belongs to, which
is what makes a frozen loop run frozen delegation code. start_watchdog survives
as an alias for ensure_subscriber_daemon.
The wrapper and run_loop.sh are untouched; reconcile.sh gains only the
ensure_env_loaded line from the previous commit.
Verified: C-01..C-09 pass against both the legacy and the python bodies, the
daemon dedups and survives its parent, the cli reproduces the legacy failure
line exactly, and publish failure semantics (status synced, seq consumed,
audit records published=false) are identical through the shim and the cli.
tier1+tier2 109 passed; bash -n clean on all six shell files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
The four delegation scripts become a real package. Their old paths stay as
shims because those paths are a wire contract: agent instructions, the bash
wrapper, SKILL.md, BOOTSTRAP.md, deploy/install.sh and three test modules all
call them literally.
The shim binds the lib_py.delegation *package* to the tree beside it, once per
process, and never touches sys.path. Two cheaper designs were rejected with
evidence (the context matrix now lives in the test suite):
* inserting the sibling skills dir at sys.path[0] makes the first sandbox
copy imported under pytest shadow lib_py.agents/layout/state for the whole
session, and the identity assertion cannot see it -- `is` proves same
object, not same tree.
* loading only the named module by file path leaves a module that imports
siblings pulling them from whichever tree is on the path, so a frozen
publish_event ran against live dependencies and lost half of B-13.
Binding the package fixes both: one tree for all delegation modules, the parent
lib_py untouched, and a sys.path append only when no lib_py exists anywhere
(true standalone). Bind-once rather than nearest-wins, because rebinding would
split existing references and break monkeypatching.
Also moves the env-file load off import (_load_dotenv -> ensure_env_loaded,
called by each process entry point including reconcile.sh's embedded
subscriber), so importing the library no longer mutates os.environ.
Verified: PV-1 identity+provenance, PV-4's ten import scenarios plus K1/K3/K6
in fresh interpreters, PV-5 shim byte-identity, an AST check that the package
imports no sibling lib_py module, and --help output diffed against the
pre-move capture.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
The three lib.sh delegation helpers (delegate_submit_job,
delegate_publish_event, start_watchdog) had zero direct test coverage, so a
"full suite green" gate would have been vacuous for the Phase 1 extraction.
Add the regression oracle before the move:
- C-01..C-09 golden tests for the shell helpers: the non-fatal contract (empty
job id, missing script, non-zero interpreter exit), exact argv forwarding,
stdout-is-the-job-id-alone, and the daemon's PID/dedup/log-path behaviour.
All are parametrized over MAM_DELEGATION_PY so the legacy and python bodies
must agree.
- CLI contract snapshots taken from the argparse parsers rather than --help
text, so wording stays free to change but option names, choices and defaults
do not.
- Auto-completion invariant guards for the three surfaces that decide job
status: EVENT_TO_STATUS, the wrapper's exit-code mapping, and reconcile.sh's
terminal mutation literals.
- conftest: PV-2/PV-3 session guards asserting that every non-delegation
lib_py.* module resolves to the real skills tree and that no temp-tree
sys.path entry contributes a lib_py package. MAM_ENV_FILE is pointed at an
absent file inside the sandbox so delegation code can never inject the real
.mam.env into a test process.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
- Implement CodexAdapter in lib_py/agents/adapters/codex.py with safe flags and TUI readiness
- Extend create_session, resume_session, stop_session, and reconcile for codex CLI
- Support codex agent delegation and loop integration across all multi-agent skills
- Add unit, component, and contract test suites for codex integration
- Verify all 462 tests passing cleanly
- Branch on Class A agents (agy, hermes, opencode) in resume_session.sh to fall back to clean spawn (CMD_FULL) when stopped before first turn (empty UUID)
- Update update_yaml_resumed.sh to support empty UUID with --cmd-full and refresh herdr_session_epoch on fresh spawn resume to prevent stale transcript capture
- Strictly preserve Class B (claude, grok) assigned UUID escape hatch and test_t8 contract
- Add regression tests test_t14, test_t15, test_t16 (Closes#3 Item 2)
- Replace 'nohup ... server & disown' with portable Python 'Popen(..., start_new_session=True)' in lib.sh shim generator
- Isolate headless herdr server in its own SID/PGID to protect against parent process-group signal broadcast (SIGTERM/SIGINT)
- Add regression test test_h26_daemon_spawn_uses_process_group_detachment to verify child PGID detachment (Closes#3 Item 1)
- Add test_r2 and test_r3 for bare-kind fallback restriction and R-1 exact-match enforcement
- Add test_h24 and test_h25 for idle-pane reuse and env_flags preservation
- Add --yolo and --accept-hooks headless auto-approval flags to spawn_spec and resume_spec
- Define Hermes TUI input delimiters (input_prompt='❯', input_rule_pattern='─{10,}')
- Refactor verify_artifact and discover in hermes.py using session started_at timestamps
- Fix HERDR_EPOCH capture timing before spawn in create_session.sh to prevent epoch race
- Update reconcile.sh for hermes drift-C multi-candidate and sibling claimed exclusion
- Add Hermes contract, epoch filtering, and C-ambiguous unit tests (140 passed)
- Add agent evaluation report and final review reports for Hermes support
- Define immutable MAM_VERSION="3.1.0" runtime constant in lib.sh
- Bump YAML frontmatter version to 3.1.0 across all 8 skill packages
- Document v3.1.0 changelog, 2-tier TUI readiness architecture, and migration guide in VERSIONS.md
- Document new tuning knobs (MAM_KILL_ON_READY_TIMEOUT, MAM_DIALOG_TAIL_LINES, MAM_READY_EMPTY_GIVEUP) in .mam.env.example
- Add 3-way lockstep and immutability test suite in tests/test_version_consistency.py
- Separate TUI dialog detection into blocking gate layer (_MAM_MODAL_TOKENS) and recovery layer (_MAM_HINT_TOKENS) to eliminate idle dialog starvation
- Add modal_tokens contract across agent adapters (claude, cline) with dynamic scope facts resolution in send_keys_safe
- Implement fail-closed resolution for ambiguous same-kind panes (R-1) and enforce WORKSPACE_ROOT export (R-2)
- Preserve session on TUI readiness timeout when PID is alive for diagnostics
- Add comprehensive test suite in tests/test_c1_tui_readiness.py and adapter contract assertions in test_a4
- Accept agent_not_ready from herdr agent start to allow startup dialog handling without premature rollback, while preserving fail-closed behavior on dead process timeouts.
- Match Claude fullscreen renderer upsell modal via 'Yes, try it' and dismiss with Escape to avoid dropping permission flags or deadlocking on idle /tui tips.
- Add behavioral test suite in test_b19_headless_reconcile_fixes.py and cross-agent review reports.
- Change N=1->2 split policy from down to right to establish 2 full-height columns
- Share single decision table across GUI and headless modes
- Add _full_height_pane guard to prevent half-height column splits
- Quadruple-wire default max_columns=2 and max_rows=2 across layout.py, lib.sh, and .mam.env.example
- Update unit and integration tests to enforce deterministic 2x2 grid trajectory
- Replace '--target-agent' with mandatory '--creator <session>' flag
- Add explicit rejection error for legacy '--target-agent'
- Add '--planner <session>' flag with fail-fast check when '--plan' is missing
- Implement 2-branch session validation ('is not registered' vs 'is not running')
- Update in-repo references in hooks, SKILL.md, INSTALL.md, and tests
- Add tests/test_loop_cli.py with 9 unit/contract tests
- Standardize --herdr-session as primary flag with --herdr-server alias across create_session.sh, resume_session.sh, update_yaml_resumed.sh, and stop_session.sh
- Guard HERDR_SESSION_NAME in create_session.sh from being overwritten by workspace slug defaults when explicitly provided
- Forward explicit --herdr-session from resume_session.sh to update_yaml_resumed.sh and force-update row metadata
- Add 5 new Tier 2 component tests covering CLI dry-run parsing, usage matching, default preservation, YAML serialization, and resume propagation
- Update multi-agent-mux-create/SKILL.md documentation
- Verified by autonomous multi-agent loop with unanimous PASS verdicts from Claude and Cline
- In stop_session.sh and update_yaml_resumed.sh: standardize explicit --agent option and use resolve_agent_type_from_registry to read agent type from YAML/DB state rather than brittle suffix-only regex inference.
- Update multi-agent-mux-stop/SKILL.md, multi-agent-mux-resume/SKILL.md, multi-agent-mux-create/SKILL.md, and deploy/INSTALL.md to standardize passing --agent explicitly.
- Fix J-1 in layout.py: refactor _env_int(*names, default=None) to take an explicit default parameter, eliminating the falsy-zero trap so MAM_MIN_PANE_COLS=0 is respected.
- Add regression and contract tests: test_j1_env_zero_min_cols_matches_flag_zero, test_j1_env_zero_min_rows_matches_flag_zero, test_comp_stop_agent_fallback_*, test_comp_docs_stop_examples_pass_agent.
- Verified 100% UNANIMOUS PASS from Planner claude and Reviewers claude and cline.
- I-2: add 5.0s upper-bound execution time assertion in test_bug4_headless_unobservable_fast_path to contractually guard SKS_EMPTY_GIVEUP early-exit latency
- I-3: clean up PaneInfo.focused, wire MAM_MAX_PANE_COLS/MAM_MAX_COLS environment variables and CLI --max-cols flag
- C-1: apply max_columns growth guard to headless 0x0 layouts, mirroring GUI behavior
- Promoted Planner consensus plan Rev.2 (plan-fea5f1b2.md) and unanimous PASS reports from Reviewers Claude (report-55d1a1d9.md) and Cline (report-6f18ba0f.md)
- Verified all 333 test cases pass with exit code 0
- Replace import-time LOGS_DIR cwd binding with dynamic get_logs_dir() function
- Implement PEP 562 __getattr__ and __dir__ for transparent LOGS_DIR backward compatibility
- Update audit-log callers in mqtt_common.py and registry.py to use dynamic resolution
- Add 5 regression guards in tests/test_tier1_unit.py (276/276 PASS)
- Update IMPROVEMENTS.md, VERSIONS.md, registry.md, and include plan and peer review reports
- Remove dead agent_identities read path and PyYAML import from workspace_uuid.py
- Defer eager PyYAML import in verify_session.py to lazy YAML fallback branch
- Simplify UUID resolution to 2-tier model (tier-1 own row ID -> tier-2 adapter scan)
- Clean up unused Drift D and ghost cache clearing in reconcile.sh and stop_session.sh
- Add 3 regression guards in tests/test_tier1_unit.py (266/266 PASS)
- Update IMPROVEMENTS.md, VERSIONS.md, and SKILL.md files
GodopuandClaude Opus 5