Commit Graph
13 Commits
Author SHA1 Message Date
GodopuandClaude Opus 5.5 5edf913d22 fix(followups): F2 - Python entry points report an unreadable env file in one line
A .mam.env that exists but cannot be read or decoded stopped the Python
entry points with an EnvFileError traceback, while the wrapper printed one
`ERROR:` line and exited 1 (PR #43 review claude S1). The mains that load the
env file - publish_event, registry, job_subscriber, cli, bench, heartbeat and
remote - now call mqtt_common.load_env_or_report(), which prints
`ERROR: <path>: cannot read (<exception type>)` and lets the main return 1.
No value or file content is printed. ensure_env_loaded() still raises: no
library function exits the process.

Test: every entry point, for a 000 file and an undecodable one, ends with
exactly that line and rc 1 and creates nothing (14 cases; all 14 fail at
0051fd6 with a traceback).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
2026-10-02 08:09:46 +09:00
GodopuandClaude Opus 5 135d76f903 fix(issue-18): admit remote attempts atomically, end timed-out commands with their group, fence heartbeats off the terminal, renew liveness only on news
Review round 2 (planner-reviewer-codex-01, NOT PASS at d09c162):

- R1 remote.mirror_job: admission under the registry lock. The same
  dispatch_token is idempotent (a finished attempt never runs again), a
  lower attempt or the same attempt under another token is refused before
  any write, and only a strictly newer attempt replaces the record
  (register_job(supersede_older_attempt=True); events.log kept, seqs stay
  monotonic).
- R2 remote.handle_dispatch: the --exec command runs in its own session;
  at timeout_sec, or when the runner is interrupted (SIGTERM now unwinds
  it), the whole process group is terminated (TERM, then KILL) before the
  error is published.
- R3 terminal barrier: a terminal publish raises an attempt-scoped barrier
  under the lock that allocates seqs, stops the heartbeat timer, gives it
  MAM_HEARTBEAT_DRAIN_S (default 1 s) to exit and then kills it, and only
  then takes its own seq. Beats are admitted under that lock
  (next_seq(admit=...)); a beat still on the wire is cancelled with the
  timer rather than holding the terminal back; and a late non-terminal
  event never publishes a running state over a terminal record.
- R4 job_subscriber: a retained state renews the idle clock only when its
  seq is newer than everything already accepted; re-delivered or older
  snapshots still recover a terminal status.
- tests: five regressions, all failing on 77f2829; T1-14's wall clock
  15 -> 30 s against public-broker latency.
- docs: MESSAGING 2.5 (barrier, state renewal, detection vs final close),
  heartbeat docstring, loop SKILL.md (remote rows need the Python runner),
  VERSIONS v4.5.0 items 1/4/5 and the test count (25).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
2026-09-13 20:49:56 +09:00
GodopuandClaude Fable 5.1 0550320590 fix(issue-18): bind the heartbeat timer to the agent process; scope started to the attempt; ignore re-sent finished dispatches
Review round 1 (claude NOT PASS) corrections:
- B2: the timer now watches a process - `--watch-pid`, or by default the
  nearest agent-CLI ancestor of the publishing command - and stops within
  one period once it is gone, so a crashed agent stops heartbeating and is
  closed after K x N; a remote runner watches itself. Documented as process
  liveness (an alive-but-stuck agent keeps heartbeating).
- R1: `_wait_started` accepts an events.log `started` only when it carries
  the current dispatch token (loop jobs share one events.log).
- R2: the remote runner ignores a re-sent dispatch of an attempt that
  already reached its terminal.
- R3: registry.md EOF blank line (git diff --check clean).
- R4: the ack error states the effective budget when the wall clock cut it.
- P3: stop_timer verifies the pidfile pid is this job's timer before
  signalling; a retained state message keeps its own seq watermark so it
  can never make the monotonic guard drop its own event.
- MULTI_AGENT_RULES.md timeout section updated for the contract.
- tests: agent-process death stops the timer, recycled-pid safety,
  attempt-scoped started, re-sent dispatch; acceptance 3 now kills the
  agent process instead of the timer. Help captures refreshed
  (--watch-pid). The wrapper can be sourced with
  MAM_DELEGATE_JOB_NO_MAIN=1 for helper-level tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-09-13 19:48:03 +09:00
GodopuandClaude Fable 5.1 897b505266 feat(issue-18): decide liveness and delivery by the event protocol, not the screen
Contract (no new event types): heartbeat every N s (automatic timer),
unresponsive after K x N s without an event, `started` within T s of the
injection, retained per-job state, and a remote transport.

- registry: heartbeat_sec / heartbeat_misses / ack_timeout_sec / transport
  fields; idle_timeout_sec derives K x N unless explicit (defaults 30/4/60
  keep the historical 120 s idle window).
- heartbeat.py + publish_event --heartbeat N: detached fixed-cadence timer
  publishing signed `progress` heartbeats (threaded wire publish, local
  record first); stopped by the terminal event via pidfile, by re-dispatch
  and by lifetime. Terminal events refuse --heartbeat.
- publish_event: retained `<prefix>/state` (status, last_event, seq, HMAC)
  sent on the same connection after every event (--no-state to skip);
  publisher.publish_raw takes extra messages.
- job_subscriber: subscribes the state topic, recovers terminal status from
  retained state, resets idle on any accepted message, records seq_gap and
  reconnected in the audit log, idle budget before the first event is
  T + K x N (--ack-timeout), --remote mirrors started/terminal events into
  the registry through the transition boundary.
- lib.sh send_keys_safe: an accepted-but-unconfirmed submission is confirmed
  by agent_status working or the job's own `started` (_sks_job_started);
  pane change and busy token are no longer delivery signals.
- wrapper: --heartbeat/--heartbeat-misses/--ack-timeout (MAM_* env),
  injected start command arms the timer, brief states the contract,
  _wait_started (registry / events.log / subscriber log, bounded by T and
  the wall clock) replaces the 10 s inject grace, R1 grace uses only the
  on-disk event age (herdr/pane logged once as a diagnostic),
  remote:<name> sessions skip preflight/idle/pane and dispatch over the
  broker (remote_dispatch).
- remote.py (+ scripts shim): signed retained dispatch (MAM_DISPATCH_SECRET)
  and a runner that mirrors the job into its own registry with the
  orchestrator's identity, publishes started --heartbeat, runs --exec and
  publishes the terminal event.
- loop runner: passes MAM_HEARTBEAT_SEC / MAM_HEARTBEAT_MISSES /
  MAM_ACK_TIMEOUT_S through; YAML rows with transport: remote delegate as
  remote:<name>.
- docs: MESSAGING.md 2.5 contract + remote transport, delegate-job SKILL.md,
  registry.md, loop SKILL.md.
- tests: tests/test_issue18_liveness_contract.py (16); issue 14 pane-signal
  tests flipped to started-based; issue 6 working-agent tests acknowledge
  with started; CLI snapshot/help captures refreshed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-09-13 19:06:12 +09:00
GodopuandClaude Fable 5.1 c82ea75a15 fix(issue-10,11,13,15): complete agy purge, orchestrator purge, workspace-anchored registry
Issue #10 - agy purge_artifacts left resumable pieces behind: it now also
removes the SQLite sidecars (-wal/-shm/-journal), the title annotation and
the presence lock (the lock only once no live process references the
conversation), and residual_artifacts() reports anything under
~/.gemini/antigravity-cli still carrying the uuid.

Issue #13 - the orchestrator has no herdr_sessions row, so its conversation
could not be purged by any skill. `lib_py.agents purge <agent|auto> <uuid>`
resolves the adapter from what is on disk (orchestrator_uuids stores bare
uuids), refuses while a live process names the uuid (conversation_in_use:
process list minus the caller's own lineage, plus lsof), and prints what it
purged. `orc_onboard.sh --remove <uuid> --purge-conversation --yes [--agent]`
runs it before the registry update so a refusal changes nothing (exit 4
without --yes, 5 while in use).

Issue #11 - "Successfully removeed": conjugate add/remove explicitly.

Issue #15 - the job registry and audit log followed the caller's cwd, so the
pytest suite (create_session.sh with its default onboarding job, run from the
repo root) had left 593 test records in the real .mam/jobs. Defaults now
resolve MAM_JOBS_DIR / DELEGATE_JOB_LOGS_DIR, then $WORKSPACE_ROOT/.mam/...,
then the cwd (mqtt_common.default_registry_dir / get_logs_dir; registry.py,
publish_event.py, service.py, create_session.sh, stop_session.sh); the
sandbox fixture pins both. The tier suites that used to leak now leave the
real registry untouched (881 records before and after).

Tests: test_issue10_13_purge.py (13), test_issue15_registry_isolation.py (7);
T-O13 pins the corrected message; the R1 helper test and B-9 logs-dir test
now assert the workspace-anchored contract; publish_event snapshot updated.

Refs #10 #11 #13 #15

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-09-13 08:18:56 +09:00
GodopuandClaude Opus 5 ebe58f6be0 fix(issue-14): record every event on disk before the wire publish
Round-2 review of 952657e (reviewer-grok-01): R1's disk-heartbeat
evidence comes from events.log, but publish_event appended to it only
after with_retry(_publish_once) returned - after the broker ACK or all of
its retries. With the public broker at ~5 s per publish, an agent's
heartbeat reached the disk after the grace window and the live job was
closed as deadline_timeout (test_wrapper_idle_timeout_grace_accepts_disk_heartbeat
failed 2/2 on 952657e here, as it did for the reviewer).

- publish_event: new record_local_event appends the prepared payload to
  events.log right after prepare_event (validation and seq allocation
  first) and before the wire publish; record_outcome takes
  local_event_recorded=True so the line is not written twice, while the
  audit "published" entry and the status sync stay after the publish
- DelegationService.publish_event: the same local-first order
- wrapper comment and VERSIONS.md v4.4.2 item 6 now say the heartbeat is
  on disk before the publish; new-test count 21
- tests: the heartbeat is already in events.log when the publish starts
  (and is recorded once), and a publish failing on every retry leaves one
  local record with published=False in the audit; both fail on 952657e.
  The wrapper disk-heartbeat test keeps the real publish_event path and
  now passes 3/3 at the same broker latency

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe
2026-09-13 03:01:53 +09:00
Godopu ec61511699 fix(issue-6): fail-closed registry timeout and TUI turn handshake
- Introduce finalize_unfinished in multi-agent-mux-delegate-job for fail-closed atomic transitions
- Enforce CAS dispatch_token and attempt_id pinning across engine, CLI, and lifecycle scripts
- Integrate Herdr wait_for_agent_idle turn handshake to prevent prompt loss and race conditions
- Resolve bash 3.2 set -u empty array expansion compatibility in lib.sh
- Preserve Issue #5 R2 contract in CAS transition engine (engine.py, publish_event.py, service.py)
- Add comprehensive regression suites (test_issue6_fail_closed.py, test_issue6_turn_handshake.py)
- Bump lockstep framework and skills to v4.4.1 across lib.sh, eager.py, VERSIONS.md, 8 SKILL.md
- Synchronize documentation (IMPROVEMENTS.md, MESSAGING.md, state-transitions.md, task.md, phase5-e2e.md)
- Unanimous PASS achieved across 4 reviewer team leaders (Claude, Codex, OpenCode, Grok)

Closes #6
2026-09-09 14:02:47 +09:00
GodopuandClaude Opus 5 12cba25c18 fix(issue-5): keep finalize inside delegation and refresh the CLI golden
Two failures the full suite caught, both mine.

finalize.py started in lib_py/loop, which made publish_event import a sibling
package. That is the rule phase 1 wrote down and phase 2a already made me move
the engine for: under a frozen loop, delegation is bound to one tree while the
parent may be another, so reaching sideways mixes them. The manifest is written
into the job's own report directory and the CLI that declares it is a delegation
entry point, so finalize belongs there. Second time on the same rule, and the
guard caught it both times.

The publish_event --help golden no longer matched, because phase 2b
intentionally added the finalization options. The diff is exactly those three
options and nothing else, so the capture is refreshed rather than the assertion
weakened, and the test now states when a refresh is legitimate: an intended
extension, never drift nobody meant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 21:47:54 +09:00
GodopuandClaude Opus 5 3ea5757cfe feat(issue-5): report finalization manifest and side-effect evidence
The agent declares that its work is finished; the reader re-derives the hash
from the file rather than trusting the number in the declaration. A manifest the
watcher wrote itself would attest only that the watcher saw a stable file, which
is the self-certifying evidence shape rejected earlier in this issue.

Finalizing promotes a staged report by atomic rename before writing the
manifest, so a reader cannot observe a half-written report as final. Rewriting
the report afterwards invalidates the declaration immediately, because the
recomputed hash no longer matches -- a new body needs a new finalization.

A job that declared side effects must show evidence for every one of them.
Partial evidence is treated as none: the work is not known to be complete.

publish_event gains --finalize-report, --side-effect and --attempt-id, and does
the finalization before publishing. A `completed` event whose report is still
being written would invite exactly the partial adoption the contract forbids, so
a failure there fails the publish.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 21:27:41 +09:00
GodopuandClaude Opus 5 56d08efa95 fix(issue-5): address phase 2a cross-review findings (F-A, F-B, R1, R2, R3)
Two reviewers raised five defects. Every one was reproduced here before being
accepted; none were rebutted.

F-A (claude, blocking) -- the CAS table had no row for two transitions the
wrapper legitimately makes, so `submit --type loop|discuss` silently stopped
working by default: the per-iteration re-dispatch was refused and its metadata
discarded, the exhausted-budget finalisation was refused, and the CLI returned 0
both times. Reproduced against the real modules. Fixed by naming those
transitions rather than carving an exception: `redispatch` and `finalize` are
requests gated on operator authority, so the wrapper can move a finished record
while an agent still cannot touch its own terminal. A re-dispatch must advance
the iteration, so a replay cannot resurrect a job. The operator CLI now exits
non-zero with a reason when a write is refused -- the silence is what let this
hide.

F-B (claude) -- moving the audit mirror out of the lock made status.json
last-writer-wins, so a slow flush could leave it behind the registry. It now
carries transition_seq and skips a write that would go backwards, restoring the
sequential consistency the lock used to provide for free.

R1 (codex, blocking) -- the policy called attribution with only the dispatch
record, so it proved the record was well-formed and never confronted the
identifiers the evidence carried. A finalized scenario with a different session,
a different attempt and a manifest from another attempt was still approved.
Observed identifiers are now collected from the timeline and every one must
agree; a manifest belonging to another attempt holds. Collecting every
occurrence rather than the first matters, because taking the first would drop
exactly the contradicting value this is meant to catch.

R2 (codex, blocking) -- a supplied expected_attempt was ignored when the record
had no attempt field, so a compare-and-swap with nothing to compare committed.
It now fails closed, which matters because 2b plans to gate inferred completion
on precisely that precondition.

R3 (codex) -- a status could ride in as an ordinary field, so `progress` could
commit a terminal from inside the boundary that exists to prevent it. Status is
now settable only by the request that owns it, unknown requests are refused, and
pending has an owning request instead of being an untyped merge.

Each of the eleven new regressions was checked against the pre-fix code first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 19:33:24 +09:00
GodopuandClaude Opus 5 95f3e9a427 fix(issue-5): address phase 1 cross-review findings (R1, R2, F-1)
Three reviewers examined fc0bd96..8def235. Codex and Claude each reproduced a
defect; all three findings were re-verified here before being accepted, and none
were rebutted.

R1 (codex) — the shell helpers ran `python -m lib_py.delegation.cli`. For -m,
sys.path[0] is the working directory, which therefore beats PYTHONPATH: with cwd
inside a tree containing lib_py, the live package won even with the frozen path
listed first. The documented B-13 guarantee for this path did not hold.
Reproduced, then replaced with an explicit bootstrap that inserts SKILL_DIR
itself. cwd is left alone, because the registry and audit-log paths are relative
to it. The file-based shim path was never affected.

R2 (codex) — the persistent connection was reused on host and port alone. Two
jobs sharing an endpoint but carrying different credentials or TLS settings
silently published over the first connection. Reproduced (the client factory ran
once, under the first account, with TLS off). The guard now compares an
immutable snapshot of the whole connection identity and names only the differing
field, never its value.

F-1 (claude) — paho queues a publish issued while disconnected and raises
RuntimeError from wait_for_publish. RuntimeError is deliberately outside the
retry tuple, so the one state a persistent connection exists to survive, the gap
between a drop and paho's reconnect, failed on the first attempt. Confirmed
against paho 2.1.0. That single condition is now translated into a retryable
error; the tuple is unchanged, so genuine programming errors are still not
retried.

Two fixtures were hiding these: FakeInfo had no rc and never raised. It now
mirrors paho's contract, which is what makes the new tests meaningful.

Also folds in the duplication both Claude (O-1) and I had flagged: the CLI path
kept its own copy of connect-publish-disconnect plus its own timeout constants.
_publish_once now delegates to OneShotPublisher.publish_raw, so a transport fix
lands once. Having two copies is exactly why these bugs existed in one of them.

Documentation corrected rather than defended: the benchmark's rationale for
keeping three forks was imprecise (the || true already guarantees rc 0; the
lookup buys quietness and legacy parity), SKILL.md no longer claims PYTHONPATH
decides the frozen path, and the daemon's directory creation is recorded as a
deliberate difference. Trailing whitespace inherited by the relocated file is
stripped, so the commit range is now clean too.

Verified: each regression fails against the pre-fix code and passes after
(R1 1, R2 4, F-1 1). 66 phase-1 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 16:51:38 +09:00
GodopuandClaude Opus 5 a4b7b45e13 feat(issue-5): persistent MQTT publisher prototype and benchmark harness
Splits publish_event.main into three pieces with no behaviour change:

  prepare_event   load the job, consume the seq, build the signed payload,
                  decide retain. The seq is burned here, before any transport
                  runs, so a failed publish can never leave a number a replayed
                  packet could reuse.
  record_outcome  audit record, registry event log and status sync -- all three
                  best effort, all three regardless of the publish result. That
                  is exactly the contract G-1/G-2/G-4 pin down, now in one place.
  main            parse, prepare, publish with retry, record.

publisher.py holds the two transports behind one interface. OneShotPublisher is
the current behaviour. PersistentPublisher keeps a single connection and lets
paho reconnect underneath, raising on an unacknowledged publish so the caller's
retry policy stays the only thing deciding attempt counts. It refuses an event
for a different broker rather than publishing to the wrong place.

Nothing in production uses the persistent path: the CLI and shell helpers exit
after one publish. The consumer is the phase 2a engine, which pays the CONNACK
cost on every event today -- this job's own started event took ~12.5s because
of one CONNACK timeout plus reconnect.

bench.py measures both arms. The fake transport is deterministic and is what
the tests assert on; the real-broker arm is informational, since the PoC
default is a public broker whose latency is not ours to control.

Verified: N publishes cost N connections one-shot and 1 persistent; the two
transports leave field-for-field identical registry and audit records including
the HMAC signature and retain flag; an unacknowledged publish retries on the
same connection and records published=false; a second broker is refused. No
test opens a socket.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 15:44:33 +09:00
GodopuandClaude Opus 5 fb3498e23b refactor(issue-5): move delegation scripts into lib_py.delegation with package-bound shims
The four delegation scripts become a real package. Their old paths stay as
shims because those paths are a wire contract: agent instructions, the bash
wrapper, SKILL.md, BOOTSTRAP.md, deploy/install.sh and three test modules all
call them literally.

The shim binds the lib_py.delegation *package* to the tree beside it, once per
process, and never touches sys.path. Two cheaper designs were rejected with
evidence (the context matrix now lives in the test suite):

  * inserting the sibling skills dir at sys.path[0] makes the first sandbox
    copy imported under pytest shadow lib_py.agents/layout/state for the whole
    session, and the identity assertion cannot see it -- `is` proves same
    object, not same tree.
  * loading only the named module by file path leaves a module that imports
    siblings pulling them from whichever tree is on the path, so a frozen
    publish_event ran against live dependencies and lost half of B-13.

Binding the package fixes both: one tree for all delegation modules, the parent
lib_py untouched, and a sys.path append only when no lib_py exists anywhere
(true standalone). Bind-once rather than nearest-wins, because rebinding would
split existing references and break monkeypatching.

Also moves the env-file load off import (_load_dotenv -> ensure_env_loaded,
called by each process entry point including reconcile.sh's embedded
subscriber), so importing the library no longer mutates os.environ.

Verified: PV-1 identity+provenance, PV-4's ten import scenarios plus K1/K3/K6
in fresh interpreters, PV-5 shim byte-identity, an AST check that the package
imports no sibling lib_py module, and --help output diffed against the
pre-move capture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrDNJkHmm18uu7n7JwV5Qd
2026-09-06 15:41:43 +09:00