Compare commits

..
16 Commits
Author SHA1 Message Date
GodopuandClaude Opus 5.5 56e6713292 release(v4.12.0): update skill with in-place install and an update admission gate; resume starts the monitor (Issue #72) - archive the review reports, bump the lockstep version
The update skill joins the version matrix and the lockstep test's skill list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCgu9Fa9iT7TbJUSzdq4Ff
2026-10-28 07:49:53 +09:00
GodopuandClaude Opus 5.5 ec7f5c4e57 fix(update): the legacy start_watchdog starts the monitor in its own process group; tests clean up only the processes they started (Issue #72, review round 5)
With MAM_DELEGATION_PY=0 the monitor was left in the caller's process group
(decision 2), on every restart the update makes. The legacy body now spawns
it under job control (`set -m`, macOS has no setsid(1)) and restores the
caller's option. The update tests assert the restarted monitor's PGID in both
modes, and their teardown kills helper groups and this test's exact monitors
instead of `pkill -f <ws>`, then reports anything left that names the
workspace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:18:04 +09:00
GodopuandClaude Opus 5.5 1d63e882bc fix(update): the legacy start_watchdog matches the monitor exactly too; a restart is reported only when the monitor runs (Issue #72, review round 4)
With MAM_DELEGATION_PY=0 the legacy helper still found the monitor with an
unanchored pgrep -f, so a process that only named the monitor path was
reported as the restarted monitor and none was started. lib.sh _monitor_pids
is the shell twin of daemon.find_running; update.sh and post_update.sh check
the PID start_watchdog printed with mam_monitor_started before saying
"monitor: restarted".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 04:10:15 +09:00
GodopuandClaude Opus 5.5 cf1b693e85 fix(update): the monitor is this workspace's exact command line; create/resume take the pane workspace's admission; refusal snapshots compare the registry by rows (Issue #72, review round 3)
- daemon.find_running matches `<bash> <reconcile.sh> --subscribe[ ...]`
  exactly instead of an unanchored, unescaped pgrep regex: another
  workspace's daemon (ws.a vs wsXa) or a process naming the path in an
  argument is no longer signalled by the update. update.sh re-checks which
  PIDs are still the monitor before SIGKILL.
- create and resume (spawn path) take the update admission of every
  installed workspace at or above the pane cwd, not only the control
  root's (lib.sh mam_update_admit_at, lib_py.update target-workspaces --cwd);
  the ~/.local/bin wrapper child drops all of them.
- tests: snapshot() skips the SQLite -wal/-shm and compares the DB by its
  rows (macOS keeps the sidecars and a read rewrites -shm); regression tests
  for the monitor match and for create/resume in another installed
  workspace (marker and lock wait); evidence line refs shifted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 02:28:55 +09:00
GodopuandClaude Opus 5.5 454b77215f fix(update): a state write failure before the install restarts the monitor; the exit table names the conversion code (Issue #72, review notes)
install_phase.sh exited when it could not mark the backup `installing`,
before the installer ran, without starting the monitor phase 1 had stopped
(decision 2) and leaving the backup behind. Nothing was changed at that
point, so it now removes the backup, restarts the monitor if it ran,
releases the marker and exits 1.

SKILL.md's exit row said 1 for every failure; a failed registry conversion
after a good install passes the converter's nonzero code through, and a
cancelled prompt exits 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 14:35:35 +00:00
GodopuandClaude Opus 5.5 253a0aa56e fix(delegate): admission covers a rowless target and the installed parent of an incidental .mam (Issue #72, review round 2)
R1: a target with no registry row skipped the target gate. Its cwd is now
read from the live agent on the server delivery uses (resolve_herdr_session),
through the real herdr only when that server is running, so no server is
started; a herdr that cannot answer refuses before registration.

R2: any .mam counted as a workspace and stopped the walk, so B/sub/.mam hid
an installed B and the row's control_plane root was dropped. Workspaces are
now update.sh's installed trees (.mam_deploy/remove.sh or remove.sh): every
installed ancestor of the cwd, plus the recorded control_plane root.

lib_py.update target-workspaces prints them one per line; the wrapper holds
each on fds 20.. until the record is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 11:09:30 +00:00
GodopuandClaude Opus 5.5 ba75718ac3 fix(update): an unlistable jobs directory refuses; the rowless-agent hint names its server (Issue #72, review R2, R3)
R2: glob() hid a listing error, so an unreadable .mam/jobs (or MAM_JOBS_DIR,
or another row's jobs dir) counted as empty and the update installed over an
open job. Records are now listed with os.listdir; a directory that cannot be
listed, or stat'ed, is a blocker with its path.

R3: a discovered agent with no row was told to run stop with --session only;
stop then looked on the workspace's server, not the one the agent was found
on. The hint now carries --herdr-session <server>, quoted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 08:15:23 +00:00
GodopuandClaude Opus 5.5 e3394655b3 fix(delegate): submit also takes the target workspace's update admission (Issue #72, review R1)
Decision 8: delegate submit uses the admission lock of the workspace the
target session belongs to. The wrapper resolved only its own
WORKSPACE_ROOT, so a job could be registered and delivered to an agent of
another installed workspace while that workspace was being updated.

lib_py.update target-workspace resolves a herdr:<name> row's workspace (the
.mam at or above its pane.cwd, else its control_plane.workspace_root);
mam_update_admit takes an fd, and submit holds the caller root on fd 7 and
the target on fd 8 until the record is written. Rowless and remote targets
keep the caller-root gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 08:15:23 +00:00
GodopuandClaude Opus 5.5 595e34b2d1 fix(gate): no admission lock in a tree without .mam; MAM_UPDATE_MONITOR_STOP_S in the knob table (Issue #72)
mam_update_admit created .mam/ to hold its lock, so a checkout running its
own run_loop.sh got a .mam (test_loop_marker_isolation L1, test_issue17 W1).
A tree without .mam is not an installed workspace, the only kind the update
runs in, so the gate admits there without a lock. The update's monitor-stop
wait is a normalised integer knob and is listed in INT_KNOBS
(test_issue26_ints B5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:48:43 +00:00
GodopuandClaude Opus 5.5 8f325de979 docs(resume): the flow ends with the monitor daemon (Issue #72)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:16:38 +00:00
GodopuandClaude Opus 5.5 515b3206b9 chore(shellcheck): no new findings in the update skill, lib.sh and install.sh (Issue #72)
CI now shellchecks the update skill's scripts: SC1091 is disabled at their
`source` lines as in the other skills, and the two findings this branch added
(lib.sh's loop_lock.sh source, install.sh's one-item legacy loop) are
disabled at their lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:08:57 +00:00
GodopuandClaude Opus 5.5 fce9459e42 test(update): the first transition reaches the skill and the conversion (Issue #72, criterion 5)
A v4.11.0 workspace's .mam_deploy/update.sh installs the new version - the
skill present, no updater left - and the skill run after it converts the
registry. An update-backup in an unknown state now restarts the monitor it
stopped: nothing was changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:07:42 +00:00
GodopuandClaude Opus 5.5 47fe24e07c feat: multi-agent-mux-update skill and in-place install (Issue #72)
The update is a skill (decision 4); deploy/update.sh and the installed
.mam_deploy/update.sh are gone.

- Phase 1 (installed code, update.sh): publish the update marker under the
  exclusive admission lock; refuse before any change while a running row, a
  live herdr agent in the workspace on any running server, an open job
  (unreadable JSON included, dry_run not), a live loop marker or a
  monitor.lock held by a monitor the skills did not start (U3) is there - the
  calling agent (server + pane, its row and one running job) excepted
  (lib_py/update.py precheck, decisions 1 and 7). Stop the monitor (TERM, then
  KILL and restart it if it does not end; decision 2), handle a leftover
  backup by its state (U4), back up .agents, .mam_deploy, root files, .venv,
  the installer's .mam files and the registry (SQLite backup API under the
  registry lock) into .mam/update-backup/, then exec a copy of
  install_phase.sh outside the tree (decision 5).
- The copy (no lib.sh/lib_py, bash 3.2) runs the installer in place; on
  failure it restores the backup and restarts the monitor from the restored
  tree; on success it runs phase 2, post_update.sh of the new tree, which
  converts the registry with lib_py.registry_format and restarts the monitor
  if it ran. The backup and the marker are removed at the end.
- install.sh updates in place (decisions 3, 4, 9): install_state keys are set,
  one line each; no updater is installed and an old one (.mam_deploy/update.sh,
  or a root update.sh the manifest lists) is removed; framework files the new
  version no longer ships are deleted when unmodified, kept and listed when
  modified, and dropped from base/, the manifest and asset_hashes.txt; a local
  source's .mam is not staged. remove.sh knows the new skill; CI shellchecks
  its scripts; docs point at the skill.
- test_issue8_update_state and test_deploy_layout td6/td27 run the skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:57 +00:00
GodopuandClaude Opus 5.5 6359d65999 feat: update admission gate - entries refuse while an update runs (Issue #72, decision 8, U2)
The update skill publishes .mam/update-in-progress (pid + lstart, the
loop_lock.sh rules) while it holds .mam/update-admission.lock exclusively.
create, resume, run_loop.sh and delegate submit (decision 8) and stop and
orc-onboard (U2: a rollback would restore the registry without their write)
take that lock shared (lib.sh mam_update_admit, fd 7), refuse while the marker
is live or unknown, and hold the lock until their record is written. A stale
marker refuses nothing. lib_py.pane_lock acquire gains --shared and
--wait-env (MAM_UPDATE_ADMISSION_WAIT_S, default 120); create's wrapper path
closes fd 7 for the long-lived child. The exit-1 rows of create/resume/stop
name the refusal; skill_code_blocks.json pins follow the shifted lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:42 +00:00
GodopuandClaude Opus 5.5 179473e5d5 feat(resume): resume ensures the monitor daemon (Issue #72, decision 2)
Only create started the per-workspace monitor, and only on its job path, so
the stop -> update -> resume flow the update skill requires left no monitor.
resume now calls start_watchdog with the control root as workdir (as create
does since #21 A8, also for a worktree session) after the row is written, on
both the already-running and the spawn path. MAM_NO_SUBSCRIBER_DAEMON=1 starts
none; a failure warns and does not fail the resume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:32 +00:00
GodopuandClaude Opus 5.5 ec378e82c9 fix(attach): hints print the row's attach_command (Issue #72, comments 926/930, U1)
create's closing hint, the delegate-job wrapper's dispatch hint and stop's
stuck-TUI message printed the pre-#71 forms (`herdr session attach <name>`,
`HERDR_SESSION_NAME=... herdr agent attach <name>`), which miss an agent whose
name is over 32 characters or has upper case. They now print the session row's
attach_command, or lib_py.agents.sanitize.attach_command(server, name) when
there is no row (lib.sh session_attach_command). deploy/INSTALL.md and
install_mam.sh show the `herdr --session <server> agent attach <herdr name>`
form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:05:00 +00:00
55 changed files with 3724 additions and 387 deletions

No files matched your search

@@ -0,0 +1,14 @@
{
"loop_id": "loop-20261027T201245Z-63c71b",
"base_sha": "1d63e882bc1c46e9af8484647d5091b296d8fa3f",
"head_sha": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"tree_sha": "fbadb24de60ccdae684b23e2e0a41cdf23ab81eb",
"review_commit": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"reviewer": "planner-reviewer-claude-01",
"job_id": "1d2c21dc",
"verdict": "PASS",
"transport": "herdr",
"evidence": "review-tree",
"isolation": "worktree",
"verified_at": "2026-10-27T21:06:27Z"
}
@@ -0,0 +1,132 @@
# PR #75 후속 라운드 리뷰 — 1d63e88..ec7f5c4 (review round 5)
리뷰어: planner-reviewer-claude-01 (claude) · job 1d2c21dc
## 0. 리뷰 경로 확인
- 검증 경로: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1`
- HEAD `ec7f5c4e57286fb3c4238b8220f39094f4640391`, tree `fbadb24de60ccdae684b23e2e0a41cdf23ab81eb` 로 [리뷰 identity] 와 같다.
- `git status --porcelain --ignored` 는 비어 있다. `.mam` 은 없다.
- 리뷰 전후로 확인했다. 본 체크아웃 main 은 `0e1e2e8` 그대로다.
- 범위: 1d63e88..HEAD 의 커밋 `ec7f5c4` 하나다.
- 바뀐 파일: `.agents/skills/lib.sh`(+7/−1), `tests/test_issue72_update.py`(+32/−3)
- 브리프가 정한 범위(R2 와 테스트의 `pkill` 제거) 밖은 건드리지 않았다.
## 1. R2 — legacy `start_watchdog` 의 프로세스 그룹 분리
### 1.1 코드 (`lib.sh:2297-2310`)
- 모니터를 띄우는 `nohup bash … &` 직전에 `set -m` 을 켠다.
- 켜기 전에 호출자의 `$-` 에 `m` 이 있었는지 `had_m` 에 기억한다.
- `pid=$!` 를 읽은 뒤, 원래 꺼져 있었으면 `set +m` 으로 되돌린다.
- job control 이 켜진 상태에서 띄운 백그라운드 잡은 자기 프로세스 그룹의 리더가 된다(PGID = PID). 그래서 다음 효과가 있다.
- `$!` 는 그대로 그 PID 다. nohup 과 bash 가 exec 해도 PID 는 바뀌지 않는다.
- 따라서 정확한 PID 보고(`echo "$pid"`, `:2314-2316`)와 `mam_monitor_started` 확인이 바뀌지 않는다.
- 다음 기존 동작은 바뀌지 않았다.
- 중복 방지(`_monitor_pids`, `:2291`)
- `MAM_NO_SUBSCRIBER_DAEMON=1` 분기(`:2295-2296`)
- stdin `/dev/null`, 로그 append(`>> "$log_file"`), 작업 공간 cwd, `--idle-timeout 0`
- 바뀐 것은 분기 안의 세 줄뿐이다.
- 호출하는 곳은 모두 `$(…)` 안이다. 그래서 `set -m` 은 그 서브셸 안에서만 잠깐 켜진다.
- `update.sh:106`, `post_update.sh:41`, `install_phase.sh:64`
- `create_session.sh:748`, `resume_session.sh:162`
### 1.2 이 Mac 에서 직접 확인 (`/bin/bash` 3.2, tty 없음, `set -euo pipefail`)
- 같은 모양의 함수를 스크래치 스크립트로 만들어 돌렸다.
- 결과 1: `$(f)` 안에서 띄운 자식의 PGID 는 자기 PID 와 같다(53727 = 53727). 호출자의 PGID(53714)와 다르다.
- 결과 2: 직접 호출한 경우도 마찬가지다.
- 결과 3: 호출 뒤 `$-` 는 `ehuB` 로 그대로다. 곧 `m` 이 남지 않는다.
- 결과 4: "no job control", "[1] PID", "Done" 같은 잡 메시지가 stdout 에도 stderr 에도 나오지 않았다. 그래서 `$(…)` 로 받는 PID 출력이 오염되지 않는다.
- **고치기 전 코드에서 실패하는지도 확인했다.**
- 방법: `git archive HEAD` 로 TMPDIR 에 사본을 풀었다. 그 사본의 `lib.sh` 만 1d63e88 판으로 바꿨다.
- 그 사본에서 `-k test_every_restart` 를 실행했다.
- 결과: **mode=0 의 3개(installed, killed, rolled-back)가 `left in the caller's group` 로 실패했다. mode=1 의 3개는 통과했다(3 failed, 3 passed).**
- 즉 새 assertion(`tests/test_issue72_update.py:723-724`)이 R2 를 실제로 잡는다. 수정 뒤 6개가 모두 통과한다(§3).
- 사본은 확인 뒤 지웠고, 남은 프로세스도 없다.
- 세 경로와 두 모드: 정상 설치, 정지 시간 초과 뒤 복원, rollback 뒤 복원을 mode 0·1 로 돌리는 6 조합이 모두 같은 함수를 지나고, 모두 통과한다.
- rollback 경로는 되돌린 트리의 `lib.sh` 를 쓴다. 테스트 상자에서는 그 트리가 이 버전이다.
### 1.3 python 경로와의 차이 (비차단, 참고)
- 두 경로가 분리하는 정도가 다르다.
- python 경로(`daemon.py:95`): `start_new_session=True` 로 새 **세션**을 만든다. 그래서 제어 터미널에서도 떨어진다.
- legacy 경로: 새 **프로세스 그룹**만 만든다. 세션과 제어 터미널은 호출자와 같다.
- 결정 2 의 요구("부른 쪽 프로세스 그룹에서 분리")는 이것으로 충족된다.
- 프로세스 그룹 신호(테스트 러너·timeout 래퍼의 killpg, 터미널의 SIGINT)는 이제 모니터에 닿지 않는다. 터미널 신호는 전경 그룹에만 가는데, 모니터는 배경 그룹이다.
- 터미널이 닫힐 때의 SIGHUP 은 nohup 이 무시한다.
- 배경 그룹이 tty 에 읽고 쓸 때 생기는 SIGTTIN/SIGTTOU 도 없다. stdin 은 `/dev/null` 이고 출력은 로그 파일로 간다.
- `lib.sh:246-253`(shim 서버 기동)은 같은 문제에 `start_new_session=True` 를 쓰는 관례를 적어 두었다. legacy 경로만 `set -m` 을 쓴다.
- 브리프가 "가장 작은 수정" 을 요구했고, legacy 경로는 opt-in 이므로 결함으로 보지 않는다.
## 2. 테스트의 `pkill` 제거 (`tests/test_issue72_update.py:184-194`, `:218-240`)
### 2.1 새 teardown 이 정리하는 것
- 패턴 kill 이 없어졌다. `tests/test_issue72_*.py` 전체에 `pkill`·`killall` 이 남아 있지 않다(grep 으로 확인).
- 대신 다음 순서로 정리한다.
1. 테스트가 `Box.helper`(`:132-135`, `start_new_session=True`)로 띄우고 `b.helpers` 에 기록한 프로세스의 그룹을 `killpg` 하고 `wait` 한다.
2. 이 테스트의 작업 공간 `ws` 에 대해 `bash <ws>/…/reconcile.sh --subscribe` 형태로 정확히 맞는 모니터(`_exact_monitor_pids`)와 그 자손(`_descendants`, `pgrep -P` 재귀)만 SIGKILL 한다.
- 남은 프로세스 확인: 명령줄에 `ws` 가 들어 있는 살아 있는 프로세스가 10초 안에 없어지지 않으면 이름을 적어 실패한다(`_processes_naming`, `:236-240`).
- `ws` 는 basetemp 아래의 테스트별 경로다. 그래서 다른 작업 공간의 프로세스와 겹치지 않는다.
### 2.2 남은 프로세스 확인이 약해지지 않았는지
- 예전 `pkill -KILL -f <ws>` 는 `ws` 를 명령줄에 가진 것을 **조용히 죽였을** 뿐이다. 남은 프로세스를 확인하지는 않았다.
- 새 방식은 정확히 고른 것만 죽인다. 나머지 가운데 `ws` 를 가진 프로세스가 남으면 **실패로 드러낸다**.
- 그래서 확인은 더 강해졌다.
- 대가가 하나 있다. 그런 프로세스가 생기면 정리되지 않고 남는다. 다만 이름이 실패 메시지에 나오므로 사람이 정리할 수 있다.
- `ws` 를 명령줄에 갖지 않은 손자(예: stub 모니터의 `sleep 120`)는 두 방식 모두 패턴으로는 못 찾는다.
- 새 방식은 모니터의 자손으로 찾아서 정리한다. 따라서 이 부분도 약해지지 않았다.
- 이번 실행에서 teardown 실패는 없었다. 실행 뒤 `ps` 로 봐도 테스트가 남긴 프로세스가 없다(§3).
## 3. 실행한 테스트 (이 Mac: macOS, `/bin/bash` 3.2, `/usr/bin/python3` 3.9.6)
```
cd <검증 경로>
env -u HERDR_SOCKET_PATH -u HERDR_PANE_ID -u HERDR_TAB_ID -u HERDR_WORKSPACE_ID -u HERDR_SESSION_NAME \
-u HERDR_SESSION -u WORKSPACE_ROOT -u MAM_ENV_FILE -u MAM_JOBS_DIR -u DELEGATE_JOB_LOGS_DIR \
PYTHONDONTWRITEBYTECODE=1 /Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python \
-m pytest -p no:cacheprovider --basetemp="$TMPDIR/claude-j1d2c-bt" -q -rfs \
tests/test_issue72_update.py tests/test_issue19_subscriber_daemon_lifecycle.py \
tests/test_issue5_phase1_delegation.py
```
- 결과: **132 passed, 0 failed, 0 skipped** (191 s)
- `test_every_restart_starts_a_real_monitor_in_both_delegation_modes` 의 6 조합이 이 안에 들어 있다.
- `test_issue19`·`test_issue5_phase1` 의 watchdog 시험(중복 방지, `MAM_NO_SUBSCRIBER_DAEMON`, PID 보고, legacy·python 동등성)이 모두 통과한다.
- 고치기 전 대조 실행: TMPDIR 사본, `-k test_every_restart` → 3 failed(mode=0), 3 passed(mode=1). 자세한 것은 §1.2.
- 테스트 뒤 남은 프로세스: 테스트가 띄운 `reconcile.sh`, stub monitor, mock herdr, 구독자는 없다. 남은 것은 원래 있던 셋이다.
- 본 체크아웃 모니터 3714
- 이번 루프 7552
- 이번 잡을 디스패치할 때 띄운 job_subscriber 53086
## 4. 발견 사항
### 차단
- 없음.
### 비차단
- **N8 (이월, R2 에도 해당).** rollback 뒤 재기동은 결정 2 에 따라 **되돌린 옛 트리의** `start_watchdog` 을 쓴다.
- 그래서 실제로 이 PR 이전 버전으로 되돌리면, `MAM_DELEGATION_PY=0` 에서는 옛 `nohup … &` 그대로라 호출자 그룹에 남는다.
- 이 PR 이후 버전 사이의 rollback 에서는 해결된다.
- 결정 2 가 정한 방식("되돌린 트리의 함수")에서 생기는 한계라 결함으로 보지 않는다.
- **1.3 의 세션 차이.** legacy 경로는 프로세스 그룹만 분리하고, python 경로는 세션까지 분리한다. 결정 요구는 충족한다.
- N5·N6·N1·N3·N4 는 이번 범위 밖이라 그대로다.
## 5. 사용자 확인 필요
- N8: rollback 대상이 이 PR 이전 버전이고 `MAM_DELEGATION_PY=0` 인 경우를 받아들일지.
- 받아들이지 않는다면 결정 2 의 "되돌린 트리의 start_watchdog" 대신 `install_phase.sh` 복사본 쪽에서 분리해 띄우도록 결정을 바꿔야 한다.
- 이전 라운드에서 넘어온 N2, 그리고 결정 8 문구 보강 건은 그대로다.
## 6. 결론
- R2 는 정상 설치, 시간 초과 복원, rollback 복원의 세 경로와 두 모드 모두에서 해결됐다.
- 고치기 전 코드에서 mode=0 이 실패하고, 고친 뒤 6개가 모두 통과하는 것을 이 Mac 에서 확인했다.
- 분리 방식(`set -m`)은 중복 방지, opt-out, PID 보고, 로그를 바꾸지 않는다. bash 3.2 에서 잡 메시지 없이 동작한다.
- 테스트의 패턴 kill 은 기록한 PID 와 정확히 맞는 모니터·자손 정리로 바뀌었다. 남은 프로세스 확인은 오히려 강해졌다.
[VERDICT: PASS]
@@ -0,0 +1,14 @@
{
"loop_id": "loop-20261027T201245Z-63c71b",
"base_sha": "1d63e882bc1c46e9af8484647d5091b296d8fa3f",
"head_sha": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"tree_sha": "fbadb24de60ccdae684b23e2e0a41cdf23ab81eb",
"review_commit": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"reviewer": "planner-reviewer-codex-01",
"job_id": "028e98cb",
"verdict": "PASS",
"transport": "herdr",
"evidence": "review-tree",
"isolation": "worktree",
"verified_at": "2026-10-27T21:06:27Z"
}
@@ -0,0 +1,98 @@
# PR #75 후속 리뷰 — job 028e98cb
## 판정과 검증 대상
이번 범위의 차단 사항을 발견하지 않았다. R2의 PGID 분리와 테스트 teardown 수정이 요구 사항을 충족한다.
- base: 1d63e882bc1c46e9af8484647d5091b296d8fa3f
- head: ec7f5c4e57286fb3c4238b8220f39094f4640391
- tree: fbadb24de60ccdae684b23e2e0a41cdf23ab81eb
- 검증 경로: /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1
- HEAD/tree 일치, clean status, `git diff --check 1d63e88..HEAD` 통과. 실제 diff는 lib.sh와 test_issue72_update.py 두 파일이다.
- 새 브리프의 좁혀진 범위인 1d63e88..HEAD를 검토했다. 이전 비차단 권고에 해당하는 파일/기능을 추가 변경하지 않았음을 확인했다.
- `tea issues 72 --comments`, `tea pulls 75`를 다시 읽었다. 기준 이슈 내용은 앞선 검토와 같고, 이번 수정은 결정 2 및 기존 중복 방지/opt-out/PID/로그 계약과 정합적이다.
## 확인한 것
| 항목 | 파일:줄 근거 | 확인 결과 |
|---|---|---|
| R2: 호출자 PGID에서 분리 | `.agents/skills/lib.sh:2303–2309` | 기존 monitor 옵션을 기록하고 spawn 때만 `set -m` 사용. 비동기 monitor가 별도 PGID에서 뜨며, 기존 옵션은 복원된다. macOS에 없는 setsid(1) 없이 동작. |
| 기존 기동 계약 | `lib.sh:2289–2315` | 정확한 PID 검색과 중복 방지, opt-out 분기는 유지. cwd를 작업 공간으로 두고 원래 cwd로 복원, stdin /dev/null, 로그 append, --idle-timeout 0, nohup의 SIGHUP 처리와 PID stdout 계약 유지. |
| 0/1 × 세 복원 경로 회귀 | `tests/test_issue72_update.py:699–725`, 특히 `:724` | 정상 설치·정지 timeout·rollback의 두 모드 모두 실제 새 monitor와 PGID 분리를 검증한다. decoy 미신호 및 기동 횟수 검증도 유지. 여섯 조합 모두 통과. |
| pkill 제거·소유 프로세스 정리 | `test_issue72_update.py:184–193,220–240` | 기록된 helper 그룹을 정리하고 wait한다. monitor는 이 테스트의 정확한 절대 스크립트 명령행으로 고른 뒤, 미리 수집한 자손 PID와 함께 정리한다. 전역 패턴 kill은 없다. |
| 잔여 프로세스 검출 유지 | `test_issue72_update.py:237–240` | ws를 명령행에 남긴 프로세스가 있으면 최대 10초 기다린 뒤 실패한다. 이 문자열 검색은 보고/검증용이며 kill 대상으로 사용되지 않는다. 원본 테스트를 외부 정리 플러그인 없이 실행해 통과. |
## R2의 전후 재현 결과
base 1d63e88의 부정 대조는 직전 d14ae099 리뷰에서 실제로 실행한 동일 외부 탐침 로그를 사용했다. 당시 mode=0의 정상·timeout·rollback 세 경우는 PGID가 호출자와 같아 실패했고, mode=1의 세 대조군은 통과했다. 이번에도 같은 탐침 코드를 변경 없이 지정된 새 snapshot에서 실행했다.
| 경로 | mode=0의 새 monitor PGID / caller PGID | mode=1의 새 monitor PGID / caller PGID |
|---|---|---|
| 정상 설치 | 93359 / 88049 — 분리 | 96481 / 88049 — 분리 |
| 정지 timeout 복원 | 91992 / 88631 — 분리 | 94916 / 88631 — 분리 |
| rollback 복원 | 93376 / 88631 — 분리 | 96166 / 88631 — 분리 |
- 기존 `probe_monitor_group.py`: **2 passed / 11.95s** (base에서는 1 failed, 1 passed).
- 기존 `probe_monitor_restore_group.py`: **4 passed / 10.40s** (base에서는 2 failed, 2 passed).
- 별도 직접 호출 탐침으로 monitor 옵션 초기 off→off 및 on→on, cwd 복원을 확인: **2 passed / 3.24s**. 새 monitor의 PGID 분리도 확인했다.
- PGID 검증은 조회로 했으며 호출자의 프로세스 그룹에 신호를 보내지 않았다.
base 로그와 동일 탐침 코드, 이번 새 결과를 evidence/에 보존했다. base를 이번 허용 경로 밖에서 다시 실행한 것은 아니다.
## Mac에서 실행한 테스트
/bin/bash 3.2.57, Python 3.9.6에서 실행했다. 스냅샷에 .venv가 없어 브리프 앞부분에 명시된 제어 루트 .venv/bin/python을 사용했다.
작업 디렉터리는 위 snapshot이며, 주요 실행 명령은 다음과 같다.
```bash
env -u HERDR_SOCKET_PATH -u HERDR_PANE_ID -u HERDR_TAB_ID \
-u HERDR_WORKSPACE_ID -u HERDR_SESSION_NAME -u HERDR_SESSION \
-u WORKSPACE_ROOT -u MAM_ENV_FILE -u MAM_JOBS_DIR \
-u DELEGATE_JOB_LOGS_DIR -u AGENT_SESSIONS_YAML \
PYTHONDONTWRITEBYTECODE=1 \
/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python \
-m pytest -p no:cacheprovider \
--basetemp=/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review-028e98cb-pytest -ra \
tests/test_issue72_update.py \
tests/test_issue5_phase1_delegation.py::test_c07_watchdog_spawns_daemon_with_workdir_and_log \
tests/test_issue5_phase1_delegation.py::test_c08_watchdog_second_call_reuses_running_daemon \
tests/test_issue5_phase1_delegation.py::test_c09_start_watchdog_and_ensure_subscriber_daemon_agree \
tests/test_issue5_phase1_delegation.py::test_daemon_missing_monitor_raises_with_the_legacy_path \
tests/test_issue5_phase1_delegation.py::test_daemon_spawn_dedup_and_survives_parent_exit \
tests/test_issue19_subscriber_daemon_lifecycle.py::test_opt_out_still_reports_a_daemon_that_already_runs
```
**59 passed / 133.12s / exit 0**. update 파일 50건, phase1 watchdog 8건, 기존 daemon의 opt-out 보고 1건이다.
같은 환경 제거·Python·no-cache 설정으로 실행한 추가 검증:
| 선택 | 결과 |
|---|---|
| `test_issue19_subscriber_daemon_lifecycle.py::test_sandbox_default_never_spawns_a_daemon` (0/1) | 2 passed / 0.53s |
| `-p probe_monitor_group ... tests/test_issue72_update.py` | 2 passed / 11.95s |
| `-p probe_monitor_restore_group ... tests/test_issue72_update.py` | 4 passed / 10.40s |
| `-p probe_monitor_options ... tests/test_issue72_update.py` | 2 passed / 3.24s |
외부 탐침은 기존 Module의 Box fixture를 사용하고 항목만 외부 plugin으로 선택했다. source는 변경하지 않았다. 각 basetemp는 TMPDIR 아래 서로 다른 mam-review-028e98cb-{pytest,group,restore-group,options,optout}이었다. 이번에는 safe_cleanup 등의 외부 teardown 대체 없이 수정된 원본 cleanup을 실행했다.
- `/bin/bash -n .agents/skills/lib.sh`: 통과.
- `ast.parse(test_issue72_update.py, feature_version=(3,9))`: 통과.
- diff --check: 통과.
- ShellCheck는 PATH에 없어 실행하지 못했다. bash -n을 ShellCheck 통과로 표기하지 않는다.
- 이번 새 브리프가 정한 수정 파일·watchdog 범위로 실행했고 전체 스위트는 실행하지 않았다.
## 발견 사항 및 사용자 확인
차단/비차단 신규 지적 없음. R2와 pkill 제거 요구 모두 해결됐다. 새 정책 결정이나 재계획이 필요하지 않다.
## 자원 안전
- 검증은 지정된 snapshot에서만 했고 creator 작업 트리에서 실행·수정하지 않았다. installer/update는 TMPDIR 가짜 설치 및 로컬 source/file:// installer만 사용했다.
- 실제 herdr 서버·설정·에이전트 홈 DB를 조작하지 않았고, 실제 .mam.env 값을 테스트에 출력하거나 export하지 않았다. 이슈/PR 댓글·상태 변경, push/merge 없음.
- 다섯 basetemp의 종료 후 프로세스 점검 결과 `remaining_test_processes=[]`.
- 원래 제어 루트 monitor PID 3714는 실행 전후 같은 명령으로 살아 있었으며 신호를 보내지 않았다.
- 종료 시 HEAD/tree와 clean status가 그대로이며 diff --check도 다시 통과했다.
- 승인된 잡 이벤트·리포트 외에 실제 control-plane 상태를 수정하지 않았다. 원본 로그와 외부 탐침 코드는 이 리포트 옆 evidence/에 보존했다.
[VERDICT: PASS]
@@ -0,0 +1,14 @@
{
"loop_id": "loop-20261027T201245Z-63c71b",
"base_sha": "1d63e882bc1c46e9af8484647d5091b296d8fa3f",
"head_sha": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"tree_sha": "fbadb24de60ccdae684b23e2e0a41cdf23ab81eb",
"review_commit": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"reviewer": "reviewer-agy-01",
"job_id": "a967e100",
"verdict": "PASS",
"transport": "herdr",
"evidence": "review-tree",
"isolation": "worktree",
"verified_at": "2026-10-27T21:06:27Z"
}
@@ -0,0 +1,115 @@
# Code Review Report — PR #75 Follow-up Round (`1d63e88..ec7f5c4`, Issue #72)
- **Job ID**: `a967e100`
- **Reviewer**: `agy` (`herdr:reviewer-agy-01`)
- **Loop ID**: `loop-20261027T201245Z-63c71b` (Round 1)
- **Review Snapshot Path**: `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1`
- **Branch**: `fix/issue-72`
- **Base Commit**: `1d63e882bc1c46e9af8484647d5091b296d8fa3f` (cumulative base: `0e1e2e87dea7fcaab69d6ac388121c35b02b7dab`)
- **Head Commit**: `ec7f5c4e57286fb3c4238b8220f39094f4640391`
- **Tree Hash**: `fbadb24de60ccdae684b23e2e0a41cdf23ab81eb`
---
## 1. 확인한 것 (Files & Line Ranges Inspected in Review Snapshot)
- **`1d63e88..ec7f5c4` 변경분 집중 검증**:
- `.agents/skills/lib.sh:2278-2317` (`_start_watchdog_legacy`):
- `MAM_DELEGATION_PY=0` 경로에서 `case $- in *m*) had_m=1 ;; esac`, `set -m`으로 `nohup bash "$monitor_script" --subscribe --idle-timeout 0 </dev/null >> "$log_file" 2>&1 &`를 실행한 뒤 `[ "$had_m" = 1 ] || set +m`으로 호출자의 job-control 옵션을 복원(`lib.sh:2303-2309`).
- macOS(`/bin/bash` 3.2, `setsid(1)` 없음)에서도 `set -m` 아래 백그라운드 작업이 자체 프로세스 그룹(`pgid == pid`)으로 기동되어 호출자의 프로세스 그룹에서 분리됨을 확인.
- 기존 동작(`_monitor_pids` 중복 방지, `MAM_NO_SUBSCRIBER_DAEMON=1` opt-out, `echo "$pid"` 보고, `</dev/null`, `>> "$log_file" 2>&1`)이 그대로 보존됨을 확인.
- `tests/test_issue72_update.py:184-195, 221-241, 697-726`:
- `_descendants(pid)`(`184-186`) 및 읽기 전용 `_processes_naming(path)`(`189-193`) 추가.
- `box` fixture teardown(`221-241`)에서 패턴 기반 `pkill -KILL -f <ws>`를 완전히 제거하고, 테스트가 직접 띄운 `b.helpers` 프로세스 그룹(`os.killpg`)과 `_exact_monitor_pids(ws / MONITOR_REL)` + 그 자식(`_descendants(pid)`)만 `SIGKILL`로 정리한 뒤 `_until(lambda: not _processes_naming(ws), timeout=10)`으로 잔존 프로세스가 없음을 검증.
- `test_every_restart_starts_a_real_monitor_in_both_delegation_modes`(`697-726`): 6개 조합(`mode` `0`/`1` × `installed`/`killed`/`rolled-back`) 모두에서 `assert all(os.getpgid(int(p)) != os.getpgrp() for p in box.monitor_pids())`로 모니터의 PGID가 호출자의 PGID와 다름을 검증.
- **누적 변경분(`0e1e2e8..ec7f5c4`) 정합성 확인**:
- `.agents/skills/lib.sh:24-35, 2238-2317, 3299-3379`
- `.agents/skills/lib_py/delegation/daemon.py:40-79`
- `.agents/skills/lib_py/pane_lock.py:1-130`
- `.agents/skills/lib_py/update.py:1-428`
- `.agents/skills/multi-agent-mux-update/SKILL.md:1-30`
- `.agents/skills/multi-agent-mux-update/scripts/{update.sh,install_phase.sh,post_update.sh}`
- `.agents/skills/multi-agent-mux-{create,resume,stop,loop,orc-onboard,delegate-job}` 및 `deploy/{install.sh,install_mam.sh,remove.sh}`
---
## 2. 결정 1–10, U1–U4, Attach 댓글 및 완료 조건 대조표
| 항목 | 요구사항 요약 | 구현 위치 | 테스트 커버리지 | 상태 |
|---|---|---|---|---|
| 결정 1 | 활성 세션(레지스트리 `running` + 모든 실행 중 herdr 서버의 워크스페이스 내 라이브 에이전트)·루프·열린 잡(`pending`/`dispatched`/`running`, 타 제어 평면은 본 레지스트리 대상 잡만) 있으면 거부 및 해제 명령 안내 | `lib_py/update.py:150-267`, `update.sh:124-162` | `tests/test_issue72_update.py` (`test_every_refusal_condition_blocks_and_leaves_the_workspace_untouched` 외 9개) | 충족 |
| 결정 2 | 모니터 데몬(`reconcile.sh --subscribe`) 자동 정지(`SIGTERM`→`SIGKILL`)·설치/타임아웃/롤백 후 호출자 프로세스 그룹에서 분리하여 재기동(`MAM_DELEGATION_PY=0/1` 모두 정확 매칭 및 별도 PGID); `resume` 시 모니터 보장 | `update.sh:100-116,165-212`, `post_update.sh:37-46`, `install_phase.sh:57-69`, `resume_session.sh:215-225,250,299`, `daemon.py:40-79`, `lib.sh:2238-2317` | `tests/test_issue72_update.py` (`test_every_restart_starts_a_real_monitor_in_both_delegation_modes` 외), `tests/test_issue72_resume_monitor.py` | 충족 |
| 결정 3 | 설치 실패 시 `.agents`, `.mam_deploy`, 루트 문서, `.venv`, 설치기 관리 `.mam` 항목, 레지스트리 전체 되돌리기 | `update.sh:240-262`, `install_phase.sh:27-44,104-115` | `tests/test_issue72_update.py` (`test_a_failing_install_rolls_back_the_entire_workspace_and_restarts_the_monitor` 외) | 충족 |
| 결정 4 | 업데이트 완료 후 새 코드로 레지스트리 변환(`lib_py.registry_format`), 레지스트리 없으면 생략, 실패 시 설치 유지 + 0 아닌 종료 | `post_update.sh:27-35` | `tests/test_issue72_update.py` (`test_a_conversion_failure_keeps_the_new_install_and_exits_nonzero` 외) | 충족 |
| 결정 5 | 2단계 분리: 워크스페이스 밖 임시 복사본(`install_phase.sh`)을 `exec`(같은 PID 유지)한 뒤 새 트리의 `post_update.sh`를 별도 프로세스로 실행 | `update.sh:214-215,264-267`, `install_phase.sh:118-129` | `tests/test_issue72_update.py` (`test_the_clean_update_runs_both_phases_converts_the_registry_and_restarts_the_monitor`) | 충족 |
| 결정 6 | `deploy/update.sh` 제거, `multi-agent-mux-update` 스킬로 일원화, `deploy/install.sh` 제자리 재실행 정리(없어진 파일 삭제·수정 파일 보존, `install_state` 제자리 갱신) | `deploy/install.sh:191-196,403-448,632-653`, `deploy/update.sh`(삭제) | `tests/test_issue72_installer.py`, `tests/test_deploy_layout.py`, `tests/test_issue8_update_state.py` | 충족 |
| 결정 7 | 호출 에이전트 자신(`(caller_server, HERDR_PANE_ID)`)과 그 `running` 행·실행 중인 잡 1개는 차단 대상에서 제외 | `lib_py/update.py:192-219,229-230,244-250`, `lib_py/pane_lock.py:25-39` | `tests/test_issue72_update.py` (`test_the_calling_agent_its_row_and_its_running_job_are_excluded` 외 3개) | 충족 |
| 결정 8 | 업데이트 입장 잠금(`.mam/update-admission.lock` 공유/배타 + `.mam/update-in-progress` 표식): `create`·`resume`·`run_loop`·`delegate submit`·`stop`·`orc-onboard` 차단, 자식 프로세스 fd 누수 방지 | `lib.sh:3312-3379`, `lib_py/pane_lock.py:42-115`, `lib_py/update.py:277-322` | `tests/test_issue72_admission.py` (21개 케이스) | 충족 |
| 결정 9 | `update.sh [-y|--yes]`: TTY면 프롬프트, 비-TTY에서 `-y` 없으면 거부(`exit 1`) | `update.sh:30-60` | `tests/test_issue72_update.py` (`test_confirmation_prompt_and_non_tty_refusal`) | 충족 |
| 결정 10 | 되돌리기 범위에서 작업 기록(`jobs`, `delegate_job_logs`, `state`, `worktrees`, `locks`) 보존 및 레지스트리 백업(`sqlite3.backup` + YAML/bak) 복원 | `update.sh:241-256`, `lib_py/update.py:342-366`, `install_phase.sh:37-44` | `tests/test_issue72_update.py` (`test_rollback_restores_every_item_the_installer_can_touch_and_leaves_work_records`) | 충족 |
| U1 | attach 안내 헬퍼는 추가하지 않고 기존 행의 `attach_command`(`ATTACH_CMD`)를 그대로 사용 | `create_session.sh:654`, `multi-agent-mux-delegate-job:680-687` | `tests/test_issue72_attach_hints.py` | 충족 |
| U2 | `.mam` 없는 트리에서 입장 잠금 헬퍼가 `.mam`이나 잠금 파일을 만들지 않고 통과 | `lib.sh:3315` | `tests/test_issue72_admission.py` (`test_a_tree_without_mam_stays_clean`) | 충족 |
| U3 | `MAM_UPDATE_MONITOR_STOP_S`를 `mam_int_knob`으로 읽고 `test_issue26_ints.py` 표에 등록 | `update.sh:189`, `tests/test_issue26_ints.py:67` | `tests/test_issue26_ints.py` | 충족 |
| U4 | 이전 업데이트가 남긴 `.mam/update-backup/` 상태별 처리(`backing-up`/`installed`/빈 상태 삭제, `installing`은 `--skip-registry`로 파일만 복원, 알 수 없는 상태 거부) | `update.sh:218-237`, `install_phase.sh:36` | `tests/test_issue72_update.py` (`test_a_leftover_backup_from_a_killed_update_is_handled_by_state`) | 충족 |
| 댓글 #926, #930 | `multi-agent-mux-delegate-job` 및 `create_session.sh`의 attach 안내가 `--session <server>`를 포함한 `attach_command`를 출력 | `create_session.sh:654`, `multi-agent-mux-delegate-job:680-687` | `tests/test_issue72_attach_hints.py` (8개 케이스) | 충족 |
| 첫 전환 (완료 조건 5) | 옛 `update.sh`가 있는 설치본에서 첫 전환 시 새 `install.sh`를 거쳐 `multi-agent-mux-update` 스킬과 레지스트리 변환까지 도달 | `deploy/install.sh:191-196,403-405`, `tests/test_issue72_update.py:909-966` | `tests/test_issue72_update.py` (`test_the_first_transition_from_the_old_updater_reaches_the_skill_and_conversion`) | 충족 |
빠진 완료 조건: **없음**.
---
## 3. 실행한 테스트와 결과 (macOS 실측)
검증 경로(`/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1`)에서 브리프의 격리 실행 명령으로 새·바뀐 테스트 파일과 watchdog 관련 테스트(`test_issue5_phase1_delegation.py`, `test_issue19_subscriber_daemon_lifecycle.py`)를 모두 실행했습니다.
### 명령 1 (`tests/test_issue72_update.py`, `tests/test_issue72_resume_monitor.py`, `tests/test_issue42_skill_blocks.py`)
```bash
BT="$(mktemp -d "${TMPDIR:-/tmp}/mam-rev-a967e100-1.XXXXXX")"
trap 'rm -rf "$BT"' EXIT
cd /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1 && \
env -u HERDR_SOCKET_PATH -u HERDR_PANE_ID -u HERDR_TAB_ID -u HERDR_WORKSPACE_ID -u HERDR_SESSION_NAME -u HERDR_SESSION -u WORKSPACE_ROOT -u MAM_ENV_FILE -u MAM_JOBS_DIR -u DELEGATE_JOB_LOGS_DIR \
PYTHONDONTWRITEBYTECODE=1 /Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python -m pytest -p no:cacheprovider --basetemp="$BT/pytest" \
tests/test_issue72_update.py \
tests/test_issue72_resume_monitor.py \
tests/test_issue42_skill_blocks.py
```
- **결과**: `95 passed, 1 skipped in 379.90s (0:06:19)` (exit code `0`)
### 명령 2 (watchdog 관련 스위트 2종 + 나머지 변경·신규 테스트 7종)
```bash
BT="$(mktemp -d "${TMPDIR:-/tmp}/mam-rev-a967e100-2.XXXXXX")"
trap 'rm -rf "$BT"' EXIT
cd /var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1 && \
env -u HERDR_SOCKET_PATH -u HERDR_PANE_ID -u HERDR_TAB_ID -u HERDR_WORKSPACE_ID -u HERDR_SESSION_NAME -u HERDR_SESSION -u WORKSPACE_ROOT -u MAM_ENV_FILE -u MAM_JOBS_DIR -u DELEGATE_JOB_LOGS_DIR \
PYTHONDONTWRITEBYTECODE=1 /Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python -m pytest -p no:cacheprovider --basetemp="$BT/pytest" \
tests/test_issue5_phase1_delegation.py \
tests/test_issue19_subscriber_daemon_lifecycle.py \
tests/test_deploy_layout.py \
tests/test_issue26_ints.py \
tests/test_issue8_update_state.py \
tests/test_version_consistency.py \
tests/test_issue72_attach_hints.py \
tests/test_issue72_installer.py \
tests/test_issue72_admission.py
```
- **결과**: `219 passed in 509.96s (0:08:29)` (exit code `0`)
### 명령 3 (`/bin/bash -n` 구문 검사 및 프로세스·워크트리 무결성 확인)
- 변경된 모든 셸 스크립트(13개) `/bin/bash -n` 통과 (`exit 0`).
- `git status --short`: 출력 없음 (검증 스냅샷 깨끗함).
- `ps -ef`: 본 체크아웃의 기존 `reconcile.sh --subscribe` 데몬(PID `3714`)만 정상 유지 중이며, 테스트가 남긴 자식·더미 프로세스 없음.
---
## 4. 발견 사항 (차단 / 비차단)
- **차단 (Blocking)**: 없음.
- **비차단 (Non-Blocking)**: 없음.
---
## 5. 사용자 확인 필요
- 없음.
[VERDICT: PASS]
@@ -0,0 +1,14 @@
{
"loop_id": "loop-20261027T201245Z-63c71b",
"base_sha": "1d63e882bc1c46e9af8484647d5091b296d8fa3f",
"head_sha": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"tree_sha": "fbadb24de60ccdae684b23e2e0a41cdf23ab81eb",
"review_commit": "ec7f5c4e57286fb3c4238b8220f39094f4640391",
"reviewer": "reviewer-grok-01",
"job_id": "a43bca93",
"verdict": "PASS",
"transport": "herdr",
"evidence": "review-tree",
"isolation": "worktree",
"verified_at": "2026-10-27T21:06:27Z"
}
@@ -0,0 +1,71 @@
# PR #75 (Issue #72) 리뷰 — 라운드 5
검증 경로 `/var/folders/q_/68sxgw3j073dgjg8wl9s08vw0000gn/T/mam-review/loop-20261027T201245Z-63c71b/1` 만 봤다. HEAD `ec7f5c4e57286fb3c4238b8220f39094f4640391`, tree `fbadb24de60ccdae684b23e2e0a41cdf23ab81eb`, base `1d63e882bc1c46e9af8484647d5091b296d8fa3f` (HEAD 의 부모이자 조상), porcelain 0. 스냅샷에는 `.venv` 가 없다. 테스트 뒤에도 HEAD·tree·porcelain 은 같았다. 제어 평면의 입장 잠금과 업데이트 표식은 없었고, 체크아웃 모니터 pid 3714 (`reconcile.sh --subscribe --idle-timeout 0`) 는 그대로였다.
이번 커밋은 `ec7f5c4` 하나다. `1d63e88..ec7f5c4` 는 `lib.sh` 와 `tests/test_issue72_update.py` 이고 `git diff --check` 는 깨끗했다. diff 에 `mapfile`, `declare -A`, `local -n`, `[[ -v`, `sed -i`, `stat -c`, `readlink -f`, `date -d` 는 없다. `setsid(1)` 도 호출하지 않는다.
## 확인한 것
R2. `MAM_DELEGATION_PY=0` 의 기동도 호출자 프로세스 그룹 밖에 둔다.
- `lib.sh:2297-2310`. 이미 떠 있는 모니터가 없을 때만, 호출자가 이미 `-m` 이었는지 기억하고 `set -m` 으로 잡은 뒤 `nohup bash <reconcile.sh> --subscribe --idle-timeout 0` 을 백그라운드로 띄운다. stdin 은 `/dev/null`, 로그는 append, 그 다음 호출자의 `-m` 을 되돌린다. 중복 확인(`lib.sh:2291` `_monitor_pids`)과 `MAM_NO_SUBSCRIBER_DAEMON=1` (`lib.sh:2295-2296`) 은 그 앞에 그대로다. PID 출력은 `lib.sh:2314-2315` 이다.
- 이 함수를 쓰는 재시작은 세 곳이다. 설치 성공은 `post_update.sh:41`, 정지 시간 초과는 `update.sh:106`, 롤백과 상태 기록 실패는 `install_phase.sh:64` 의 `bash -c` 가 복구된 트리의 `start_watchdog` 를 부른다. `lib.sh:2567-2573` 에서 `MAM_DELEGATION_PY=0` 이면 레거시, 아니면 Python `start_new_session` 이다.
- tty 없이 `set -euo pipefail` 인 bash 3.2 에서 `set -m` 으로 띄운 `sleep` 의 PGID 는 그 PID 와 같고 부모 PGID 와 달랐다. 부모가 끝난 뒤에도 그 프로세스는 남아 있었고, 확인 뒤에 그 PID 만 종료했다.
- `test_issue72_update.py:697-725`. `MAM_DELEGATION_PY` 0 과 1, 설치 성공·TERM 무시 후 KILL·롤백 여섯 조합에서 새 모니터의 PGID 가 이 테스트 프로세스 그룹과 다른지 본다. `box.update` 는 프로세스 그룹을 바꾸지 않으므로 그 그룹이 호출자 그룹이다.
정리. 테스트는 패턴으로 죽이지 않는다.
- `tests/test_issue72_update.py` 에 `pkill` 과 `killall` 은 없다.
- `test_issue72_update.py:221-240`. 이 테스트가 `helper` 로 띄운 프로세스 그룹만 `os.killpg` 하고, 이 임시 작업공간의 `reconcile.sh --subscribe` 명령줄과 그 `pgrep -P` 자식만 `os.kill` 한다. 그 다음 10초 동안 그 작업공간 경로를 명령줄에 가진 프로세스가 남아 있으면 실패한다 (`_processes_naming`, `184-193`). 읽는 용도이고, 다른 작업공간의 모니터는 경로가 다르므로 대상이 아니다.
이전 라운드에서 맞던 중복 방지, opt-out, PID 보고, 로그 append 는 이 diff 가 그 순서를 바꾸지 않았다. Python 쪽 세션 분리는 `tests/test_issue5_phase1_delegation.py:697-699` 가 그대로 본다.
## 실행한 테스트
작업 디렉터리는 스냅샷이다. 인터프리터는 제어 평면 `/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python` (Python 3.9.6). `--basetemp=/tmp/mam-i72-a43bca93`.
```
env -u HERDR_SOCKET_PATH -u HERDR_PANE_ID -u HERDR_TAB_ID -u HERDR_WORKSPACE_ID \
-u HERDR_SESSION_NAME -u HERDR_SESSION -u WORKSPACE_ROOT -u MAM_ENV_FILE \
-u MAM_JOBS_DIR -u DELEGATE_JOB_LOGS_DIR \
PYTHONDONTWRITEBYTECODE=1 \
/Users/godopu16/PuKi/laa/canary_projects/multi-agent-mux/.venv/bin/python -m pytest \
-p no:cacheprovider --basetemp=/tmp/mam-i72-a43bca93 \
tests/test_issue72_admission.py tests/test_issue72_attach_hints.py \
tests/test_issue72_installer.py tests/test_issue72_resume_monitor.py \
tests/test_issue72_update.py tests/test_issue8_update_state.py \
tests/test_deploy_layout.py tests/test_issue42_skill_blocks.py \
tests/test_issue26_ints.py tests/test_version_consistency.py \
tests/test_issue5_phase1_delegation.py \
tests/test_issue19_subscriber_daemon_lifecycle.py \
-q --tb=line
```
결과: **314 passed, 1 skipped**, 888.47s, 종료 코드 0. PGID 여섯 조합과 watchdog 수명 주기가 이 안에 들어 있다.
건너뛴 한 건은 `tests/test_issue42_skill_blocks.py:266` (`.claude/skills` 가 이 체크아웃에 없음). 같은 파일만 `--basetemp=/tmp/mam-i72-a43bca93-skip -rs` 로 다시 돌리면 39 passed, 1 skipped, 0.07s 이다.
basetemp 경로를 가진 프로세스는 종료 뒤에 남아 있지 않았다.
## 발견 사항
차단: 없음. 설계를 다시 잡아야 하는 결함도 없다.
비차단:
- 레거시 경로는 새 프로세스 그룹이다 (`set -m`). Python 경로는 `start_new_session` 으로 새 세션이다. 이번 라운드가 요구한 것은 호출자 프로세스 그룹에서 벗어나는 것이고, 둘 다 그렇게 한다. `nohup` 이 SIGHUP 을 무시하는 것은 그대로다.
- `lib.sh:2310`. 모니터를 띄운 뒤 원래 디렉터리로 돌아가지 못하면 함수는 1 을 반환한다. 그 시점에는 이미 `set +m` 이 실행됐고 데몬은 떠 있다. 호출자는 PID 를 받지 못한다. 디렉터리가 사라진 경우에만 해당한다.
- 앞 라운드의 비차단 항목(롤백 재시작이 `mam_monitor_started` 를 호출하지 않음, SIGKILL 자식 PID 재확인, 이미 떠 있는 resume 의 잠금 범위, 호출자의 실행 중 잡이 둘일 때)은 이 커밋이 건드리지 않았고 이번 수정 범위 밖이다.
전용 테스트가 없는 조건:
| 조건 | 코드 | 테스트 |
| --- | --- | --- |
| 레거시 모니터가 호출자와 다른 세션인지 | `lib.sh:2306-2308` 은 프로세스 그룹만 분리 | 이번 테스트는 PGID 만 비교 (`test_issue72_update.py:724`) |
| 앞 라운드 표의 세 조건 | 변경 없음 | 이번 범위 밖 |
## 사용자 확인 필요
없음.
[VERDICT: PASS]
+149 -3
View File
@@ -29,7 +29,7 @@ export WORKSPACE_ROOT
# Framework semantic version. Single runtime source of truth; kept in lockstep
# with VERSIONS.md and the 8 SKILL.md frontmatters by tests/test_version_consistency.py.
# NOTE: unlike other MAM_* variables this one is intentionally NOT env-overridable.
MAM_VERSION="4.11.0"
MAM_VERSION="4.12.0"
export MAM_VERSION
AGENT_SESSIONS_YAML="${AGENT_SESSIONS_YAML:-$WORKSPACE_ROOT/.mam/agent-sessions.yaml}"
@@ -1671,6 +1671,22 @@ print(resolved)
"
}
# session_attach_command <session_name> [server]
#
# Issue #72: the attach hint a script prints - the session row's
# attach_command, else the one lib_py.agents.sanitize.attach_command builds for
# <server> (default: `default`).
session_attach_command() {
MAM_STATE_JSON="$(load_state_json)" SESSION_NAME="$1" ATTACH_SERVER="${2:-default}" python3 -c "
import os, json
from lib_py.agents.sanitize import attach_command
name = os.environ['SESSION_NAME']
d = json.loads(os.environ.get('MAM_STATE_JSON') or '{}')
row = next((s for s in d.get('herdr_sessions', []) if s.get('name') == name), {})
print(row.get('attach_command') or attach_command(os.environ['ATTACH_SERVER'], name))
"
}
# resolve_herdr_session <session_name> [workspace]
#
# returns the isolated herdr *session* name (socket/daemon) to use for this MAM session row,
@@ -2219,6 +2235,41 @@ _delegate_publish_event_legacy() {
"$py_bin" "$pub" --job "$job_id" --event "$event" --detail "$detail" ${extra[@]+"${extra[@]}"} || true
}
# _monitor_pids <monitor_script>
# Issue #72: PIDs whose command line is exactly `<bash> <monitor_script>
# --subscribe[ ...]` - the shell twin of lib_py.delegation.daemon.find_running,
# so both start_watchdog branches recognise the same daemons. Not a `pgrep -f`
# regex: that also matched another workspace's daemon (an unescaped `.`) and a
# process that only names the path in an argument.
_monitor_pids() {
ps -A -ww -o pid= -o args= | MAM_MONITOR_WANT="$1 --subscribe" awk '
{ line = $0; sub(/^ +/, "", line)
i = index(line, " "); if (!i) next
pid = substr(line, 1, i - 1); args = substr(line, i + 1)
j = index(args, " "); if (!j) next
n = split(substr(args, 1, j - 1), a0, "/"); rest = substr(args, j + 1)
want = ENVIRON["MAM_MONITOR_WANT"]
if (a0[n] == "bash" && (rest == want || index(rest, want " ") == 1)) print pid }'
}
# mam_monitor_started <workspace> <pid...>
# Issue #72: 0 once one of the PIDs start_watchdog printed is WORKSPACE's
# monitor (_monitor_pids), so a restart is only reported when it happened.
# Polled for up to 5 s: a daemon just started from the shell carries its
# parent's command line until it execs bash.
mam_monitor_started() {
local script="$1/.agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh" n=0 p
shift
while [ "$n" -lt 50 ]; do
for p in $(_monitor_pids "$script"); do
case " $* " in *" $p "*) return 0 ;; esac
done
sleep 0.1
n=$((n + 1))
done
return 1
}
# start_watchdog <job_id> [workdir]
# Spawns a watchdog process to monitor a delegate-job JOB in the background.
# The watchdog re-spawns the subscriber every 2 minutes (or whatever hard
@@ -2237,7 +2288,7 @@ _start_watchdog_legacy() {
# Check if reconcile.sh --subscribe is already running on this workspace
local pid
pid=$(pgrep -f "bash $monitor_script --subscribe" || true)
pid=$(_monitor_pids "$monitor_script" || true)
# Issue #19: MAM_NO_SUBSCRIBER_DAEMON=1 reports a running daemon but never
# starts one (test sandboxes; externally supervised monitors).
@@ -2246,10 +2297,16 @@ _start_watchdog_legacy() {
elif [ -z "$pid" ]; then
# Start the wildcard monitor subscriber daemon with --idle-timeout 0 (never idle out)
# and ensure it runs with $workdir as cwd to anchor relative log paths.
local orig_pwd="$PWD"
# Issue #72 (decision 2): job control (`set -m`) starts it in its own
# process group, out of the caller's, as the python body's new session
# does; macOS has no setsid(1). The caller's option is restored.
local orig_pwd="$PWD" had_m=0
case $- in *m*) had_m=1 ;; esac
cd "$workdir" || return 1
set -m
nohup bash "$monitor_script" --subscribe --idle-timeout 0 </dev/null >> "$log_file" 2>&1 &
pid=$!
[ "$had_m" = 1 ] || set +m
cd "$orig_pwd" || return 1
fi
@@ -3231,3 +3288,92 @@ mam_control_plane_prefix() {
[ -n "${1:-}" ] || return 0
PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" "$(_delegate_py_bin)" -m lib_py.control_plane prefix "$1"
}
# ---------------------------------------------------------------------------
# Issue #72 (decision 8): the update admission gate.
#
# The update skill publishes .mam/update-in-progress (pid + lstart, loop_lock.sh
# rules) while it holds .mam/update-admission.lock exclusively, then runs its
# pre-check. Every entry that starts something the pre-check must see - create,
# resume, run_loop.sh, delegate submit - or writes the registry - stop,
# orc-onboard - takes that lock shared on fd 7, refuses while the marker is
# `live` or `unknown`, and keeps fd 7 until its record is written
# (mam_update_admit_release). An update therefore either waits for that record
# and sees it, or publishes first and the entry refuses. The lock is a BSD
# flock taken by lib_py.pane_lock on the open fd: it ends when every process
# holding fd 7 has ended, however that ends, so nothing long-lived may inherit
# fd 7 (`7>&-` on such a child, as pane_lock's fd 9). The wait is bounded by
# MAM_UPDATE_ADMISSION_WAIT_S (default 120).
#
# mam_update_admit [workspace [fd]]: rc 0 admitted (fd 7, or the given one,
# held unless the workspace has no .mam), 1 refused (stderr). Delegate submit
# also takes the workspaces its target is in on fds 20 and up, create and
# resume those their pane's cwd is in (mam_update_admit_at).
mam_update_admit() {
local root="${1:-$WORKSPACE_ROOT}" fd="${2:-7}"
local lock="$root/.mam/update-admission.lock" marker="$root/.mam/update-in-progress" st
# No .mam: not an installed workspace, which is all the update runs in (and a
# checkout running its own scripts gets no .mam from this).
[ -d "$root/.mam" ] || return 0
if ! eval "exec $fd>>\"\$lock\""; then
echo "ERROR: cannot open the update admission lock $lock" >&2
return 1
fi
case "$(PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" python3 -m lib_py.pane_lock acquire --fd "$fd" --shared --wait-env MAM_UPDATE_ADMISSION_WAIT_S)" in
locked) ;;
timeout)
mam_update_admit_release "$fd"
echo "ERROR: an update of $root holds its admission lock (MAM_UPDATE_ADMISSION_WAIT_S passed); try again when it has finished" >&2
return 1 ;;
*)
mam_update_admit_release "$fd"
echo "ERROR: the update admission lock $lock could not be decided (lib_py.pane_lock)" >&2
return 1 ;;
esac
[ -f "$marker" ] || return 0
# shellcheck disable=SC1091
declare -F _mam_lock_holder_state >/dev/null || source "$SKILL_DIR/multi-agent-mux-loop/scripts/loop_lock.sh"
st="$(_mam_lock_holder_state "$marker")"
case "$st" in
stale)
echo "NOTE: stale update marker $marker (its update is gone); ignored" >&2
return 0 ;;
esac
mam_update_admit_release "$fd"
echo "ERROR: an update of $root is in progress (marker $marker, holder $st); try again when it has finished" >&2
return 1
}
mam_update_admit_release() {
eval "exec ${1:-7}>&-"
}
# mam_update_admit_at <cwd>: what create and resume take for an agent they
# start in <cwd> - mam_update_admit for the control root (fd 7) and for every
# other installed workspace at or above <cwd> (fds 20 and up), whose update
# counts that agent (decision 1). rc 1 refused, nothing held.
# mam_update_admit_release_all drops them all; a long-lived child started
# while they are held runs it first.
MAM_UPDATE_ADMIT_FDS=""
mam_update_admit_at() {
local roots root fd=20
roots="$(PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" "$(_delegate_py_bin)" -m lib_py.update target-workspaces \
--workspace "$WORKSPACE_ROOT" --cwd "$1")" \
|| { echo "ERROR: cannot resolve the installed workspaces at or above $1" >&2; return 1; }
mam_update_admit || return 1
while IFS= read -r root; do
[ -n "$root" ] || continue
if ! mam_update_admit "$root" "$fd"; then
mam_update_admit_release_all
return 1
fi
MAM_UPDATE_ADMIT_FDS="$MAM_UPDATE_ADMIT_FDS $fd"; fd=$((fd + 1))
done <<< "$roots"
}
mam_update_admit_release_all() {
local fd
mam_update_admit_release
for fd in $MAM_UPDATE_ADMIT_FDS; do mam_update_admit_release "$fd"; done
MAM_UPDATE_ADMIT_FDS=""
}
+24 -9
View File
@@ -2,7 +2,8 @@
Behaviour is byte-for-byte compatible with the shell original (issue #5 phase 1,
D5): same monitor path, same stderr wording and exit status when it is missing,
same `pgrep -f` pre-check pattern, same log file, same "print the PID" contract.
the same daemons found by the pre-check (matched exactly since Issue #72, see
find_running), same log file, same "print the PID" contract.
Callers capture stdout with $( ), so nothing else may be printed there.
"""
from __future__ import annotations
@@ -39,20 +40,34 @@ def monitor_script_path(workdir: str) -> str:
def find_running(monitor_script: str) -> List[str]:
"""PIDs of subscribers already running for this monitor script.
Uses the exact `pgrep -f` pattern the shell helper used, so a daemon started
by either implementation is found by the other. Multiple matches are returned
in full: the legacy helper echoed every line and callers only ever echo it
back, so trimming here would be a silent behaviour change.
A subscriber is a process whose command line is `<bash> <monitor_script>
--subscribe[ ...]`, argv0 a bash, as both this module and the shell helper
start it. Issue #72: matched exactly (lib.sh _monitor_pids is the shell
twin the legacy helper uses), not as a `pgrep -f` regex, because the update skill signals what this returns - a daemon of
another workspace the unescaped path also matched, or a process that only
names the path in an argument, is not this workspace's monitor. Multiple
matches are returned in full: the legacy helper echoed every line and
callers only ever echo it back, so trimming here would be a silent
behaviour change.
"""
try:
res = subprocess.run(
["pgrep", "-f", "bash {} --subscribe".format(monitor_script)],
["ps", "-A", "-ww", "-o", "pid=", "-o", "args="],
capture_output=True, text=True,
)
except OSError as exc: # pragma: no cover - pgrep missing
logger.warning("pgrep unavailable: %s", exc)
except OSError as exc: # pragma: no cover - ps missing
logger.warning("ps unavailable: %s", exc)
return []
return [line for line in res.stdout.split() if line.strip()]
want = "{} --subscribe".format(monitor_script)
pids = []
for line in res.stdout.splitlines():
parts = line.strip().split(" ", 1)
if len(parts) != 2:
continue
argv0, _, rest = parts[1].partition(" ")
if os.path.basename(argv0) == "bash" and (rest == want or rest.startswith(want + " ")):
pids.append(parts[0])
return pids
def _ignore_sighup(): # pragma: no cover - runs in the forked child
+1 -1
View File
@@ -13,7 +13,7 @@ import sys
from typing import List, Sequence
# Framework version locked with framework constants
FRAMEWORK_VERSION = "4.11.0"
FRAMEWORK_VERSION = "4.12.0"
CORE_MODULES: Sequence[str] = (
# Standard library modules
+10 -5
View File
@@ -18,6 +18,9 @@ one lock per server from the choice to the claim.
pane_lock.py acquire --fd N flock(LOCK_EX) on the inherited fd N; prints
`locked`, or `timeout` once MAM_PANE_CLAIM_WAIT_S
(default 120, minimum 1) has passed
[--shared] [--wait-env VAR] Issue #72: LOCK_SH instead, and the wait read
from VAR (same default and minimum) - the
update admission lock (lib.sh mam_update_admit)
The lock is a BSD flock, which belongs to the open file on fd 9: it outlives
this helper and ends when the last process holding that fd ends, however it
@@ -63,11 +66,11 @@ def lock_path(session: Optional[str], env: Mapping[str, str]) -> str:
return os.path.join(home, ".local", "state", "mam", "pane-lock", session + ".lock")
def wait_s(env: Mapping[str, str]) -> int:
return ints.positive_int(env.get("MAM_PANE_CLAIM_WAIT_S"), 1) or DEFAULT_WAIT_S
def wait_s(env: Mapping[str, str], var: str = "MAM_PANE_CLAIM_WAIT_S") -> int:
return ints.positive_int(env.get(var), 1) or DEFAULT_WAIT_S
def acquire(fd: int, wait: float) -> str:
def acquire(fd: int, wait: float, shared: bool = False) -> str:
"""`locked`, or `timeout` when the lock was not free within `wait` s. The
deadline is on the monotonic clock; a step of the wall clock does not
stretch or cut it. Only contention is waited out: any other error (a bad
@@ -75,7 +78,7 @@ def acquire(fd: int, wait: float) -> str:
deadline = time.monotonic() + wait
while True:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fd, (fcntl.LOCK_SH if shared else fcntl.LOCK_EX) | fcntl.LOCK_NB)
return "locked"
except BlockingIOError:
if time.monotonic() >= deadline:
@@ -90,11 +93,13 @@ def main(argv: Optional[List[str]] = None) -> int:
s.add_argument("--session", default="")
s = sub.add_parser("acquire")
s.add_argument("--fd", type=int, required=True)
s.add_argument("--shared", action="store_true")
s.add_argument("--wait-env", default="MAM_PANE_CLAIM_WAIT_S")
a = p.parse_args(argv)
if a.cmd == "path":
print(lock_path(a.session, os.environ))
else:
print(acquire(a.fd, wait_s(os.environ)))
print(acquire(a.fd, wait_s(os.environ, a.wait_env), a.shared))
return 0
+428
View File
@@ -0,0 +1,428 @@
"""Decisions behind the multi-agent-mux-update skill (Issue #72).
update.sh keeps the orchestration in shell (phase 1: this tree's code) and asks
this module what needs parsing. Standard library only, Python 3.9.
python -m lib_py.update precheck --workspace W --herdr H
What refuses the update (comment 921, decisions 1 and 7): the running
rows of the registry (MAM_STATE_JSON, from lib.sh `load_state_json
strict`), every live herdr agent whose cwd is inside W on any server
`herdr session list --json` reports running (read only: the real herdr H,
never the shim, so no server is started), and the open jobs. The agent
that called the update - by (server, pane): pane_lock.caller_server and
HERDR_PANE_ID - is not counted, nor its row and its one running job.
Prints one line per blocker with how to clear it; rc 0 nothing blocks,
1 something does (a herdr or registry that cannot be read included).
python -m lib_py.update monitor --workspace W
JSON {"pids": [...], "lock": "free"|"held"}: the monitor daemons
daemon.find_running recognises, and whether .mam/monitor.lock is held.
python -m lib_py.update target-workspaces --workspace W --agent-session S --server V --herdr H
The workspaces other than W whose update admission a delegate submit
to `herdr:<name>` must take (decision 8), one per line: every installed
workspace at or above the target's cwd (its row's pane.cwd; with no row
the live agent's on server V) and its row's control_plane
workspace_root. Nothing for a non-herdr target. rc 1: herdr cannot
answer for a rowless target.
python -m lib_py.update target-workspaces --workspace W --cwd C
The same for an agent create or resume starts in C: every installed
workspace at or above C other than W.
python -m lib_py.update backup-registry --workspace W --dest D
Copies the registry into D under the registry's own write lock
(agent-sessions.db.lock, as atomic_yaml takes it): the DB through the
SQLite backup API (commits still in -wal included), the YAML and its
.bak as files. Prints the names it copied.
"""
from __future__ import annotations
import argparse
import fcntl
import json
import os
import shlex
import shutil
import sqlite3
import subprocess
import sys
from typing import Dict, Iterable, List, Mapping, Optional, Tuple
from lib_py.agents.sanitize import sanitize_herdr_agent_name
from lib_py.delegation import daemon
from lib_py.delegation.engine import TERMINAL_STATUSES
from lib_py.pane_lock import caller_server
REGISTRY_FILES = ("agent-sessions.yaml", "agent-sessions.yaml.bak", "agent-sessions.db")
HERDR_TIMEOUT_S = 20
def _real(p: str) -> str:
return os.path.realpath(p)
def _inside(path: str, root: str) -> bool:
if not path:
return False
p, r = _real(path), _real(root).rstrip(os.sep)
return p == r or p.startswith(r + os.sep)
def row_server(row: dict) -> str:
return row.get("herdr_session") or row.get("herdr_server") or "default"
# --------------------------------------------------------------------------- herdr
class HerdrError(Exception):
pass
def _herdr_json(herdr: str, args: List[str], env: Mapping[str, str]) -> dict:
# A pane's own scope ids would narrow the answer to the caller's workspace.
cenv = {k: v for k, v in env.items()
if k not in ("HERDR_WORKSPACE_ID", "HERDR_PANE_ID", "HERDR_TAB_ID")}
try:
res = subprocess.run([herdr, *args], env=cenv, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=HERDR_TIMEOUT_S)
except (OSError, subprocess.SubprocessError) as exc:
raise HerdrError("`herdr %s` failed: %s" % (" ".join(args), exc))
if res.returncode != 0:
raise HerdrError("`herdr %s` exited %d: %s" % (" ".join(args), res.returncode,
(res.stderr or "").strip()[:200]))
try:
out = res.stdout
return json.loads(out[out.index("{"):])
except ValueError:
raise HerdrError("`herdr %s` printed no JSON" % " ".join(args))
def running_servers(herdr: str, env: Mapping[str, str]) -> List[str]:
d = _herdr_json(herdr, ["session", "list", "--json"], env)
sessions = d.get("sessions")
if not isinstance(sessions, list):
raise HerdrError("`herdr session list --json` has no sessions list")
return [s["name"] for s in sessions
if isinstance(s, dict) and s.get("running") and s.get("name")]
def server_agents(herdr: str, server: str, env: Mapping[str, str]) -> List[dict]:
d = _herdr_json(herdr, ["--session", server, "agent", "list"], env)
agents = (d.get("result") or {}).get("agents")
if not isinstance(agents, list):
raise HerdrError("`herdr --session %s agent list` has no agents list" % server)
return [a for a in agents if isinstance(a, dict)]
# --------------------------------------------------------------------------- jobs
def _job_records(d: str) -> Iterable[Tuple[str, Optional[dict]]]:
"""(path, record) per *.json; (path, None) when it cannot be read, and
(d, None) when d itself cannot be listed (not an empty registry)."""
try:
names = sorted(n for n in os.listdir(d) if n.endswith(".json") and not n.startswith("."))
except OSError:
yield d, None
return
for p in (os.path.join(d, n) for n in names):
try:
with open(p, encoding="utf-8") as fh:
rec = json.load(fh)
except (OSError, ValueError):
yield p, None
continue
yield p, rec if isinstance(rec, dict) else None
def _is_jobs_dir(d: str) -> bool:
"""d is a directory, or cannot be looked at (an unreadable parent): only an
absent d (or a file) is skipped."""
try:
os.stat(d)
except (FileNotFoundError, NotADirectoryError):
return False
except OSError:
return True
return os.path.isdir(d)
def _is_open(rec: dict) -> bool:
return rec.get("status") not in TERMINAL_STATUSES and rec.get("dry_run") is not True
def open_jobs(workspace: str, state: dict, env: Mapping[str, str]) -> List[dict]:
"""Every open record (not completed/error/cancelled, not dry_run) in this
workspace's .mam/jobs and MAM_JOBS_DIR; in another row's
control_plane.mam_jobs_dir only those addressed to a row of this registry.
A record, or a directory, that cannot be read is open (path only)."""
own, seen = [], set()
for c in (os.path.join(workspace, ".mam", "jobs"), env.get("MAM_JOBS_DIR") or ""):
if c and _is_jobs_dir(c) and _real(c) not in seen:
seen.add(_real(c))
own.append(c)
rows = state.get("herdr_sessions") or []
names = {"herdr:%s" % s.get("name") for s in rows if s.get("name")}
other = []
for s in rows:
c = (s.get("control_plane") or {}).get("mam_jobs_dir") or ""
if c and _is_jobs_dir(c) and _real(c) not in seen:
seen.add(_real(c))
other.append(c)
out = []
for dirs, addressed_only in ((own, False), (other, True)):
for d in dirs:
for path, rec in _job_records(d):
if rec is None:
out.append({"path": path, "unreadable": True, "dir": path == d})
elif _is_open(rec) and (not addressed_only or rec.get("agent_session") in names):
out.append({"path": path, "job_id": rec.get("job_id") or "",
"agent_session": rec.get("agent_session") or "",
"status": rec.get("status"), "registry_dir": d})
return out
# --------------------------------------------------------------------------- precheck
def precheck(workspace: str, state: dict, herdr: str, env: Mapping[str, str]) -> List[str]:
"""The lines that refuse the update; [] when nothing does."""
blockers: List[str] = []
stop = ("bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh --session %s"
" (resume it after the update with multi-agent-mux-resume)")
# A rowless agent: stop finds no row to take its server from (#71).
stop_rowless = ("bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh --session %s"
" --herdr-session %s")
# The caller (decision 7): its server and pane, both or nothing.
me_server, me_pane = caller_server(env), env.get("HERDR_PANE_ID") or ""
me_name: Optional[str] = None
me_found = False
agents_by_server: Dict[str, List[dict]] = {}
try:
for server in running_servers(herdr, env):
agents_by_server[server] = server_agents(herdr, server, env)
except HerdrError as exc:
blockers.append("herdr: cannot list the live agents (%s); fix herdr and try again" % exc)
if me_server and me_pane:
for a in agents_by_server.get(me_server, []):
if a.get("pane_id") == me_pane:
me_found = True
me_name = a.get("name") or None # a nameless pane is excluded by pane only
break
rows = state.get("herdr_sessions") or []
me_row = None
if me_name:
for s in rows:
if (s.get("status") == "running" and row_server(s) == me_server
and s.get("name") and sanitize_herdr_agent_name(s["name"]) == me_name):
me_row = s
break
for s in rows:
if s.get("status") == "running" and s is not me_row:
blockers.append("session %s is running (server %s); stop it: %s"
% (s.get("name"), row_server(s), stop % shlex.quote(s.get("name") or "")))
row_keys = {(row_server(s), sanitize_herdr_agent_name(s["name"]))
for s in rows if s.get("status") == "running" and s.get("name")}
for server, agents in agents_by_server.items():
for a in agents:
if me_found and server == me_server and a.get("pane_id") == me_pane:
continue
if not (_inside(a.get("cwd") or "", workspace)
or _inside(a.get("foreground_cwd") or "", workspace)):
continue
name = a.get("name") or ""
if name and (server, name) in row_keys:
continue # already listed as its running row
blockers.append("herdr agent %s (pane %s, server %s) runs in %s; stop it: %s"
% (name or "<unnamed>", a.get("pane_id"), server, a.get("cwd"),
stop_rowless % (shlex.quote(name), shlex.quote(server)) if name else
"herdr --session %s pane close %s"
% (shlex.quote(server), shlex.quote(a.get("pane_id") or ""))))
jobs = open_jobs(workspace, state, env)
if me_row is not None:
mine = [j for j in jobs if not j.get("unreadable")
and j.get("agent_session") == "herdr:%s" % me_row.get("name")
and j.get("status") == "running"]
if len(mine) == 1:
jobs = [j for j in jobs if j is not mine[0]]
for j in jobs:
if j.get("dir"):
blockers.append("jobs directory %s cannot be listed; fix its permissions, then try again"
% j["path"])
elif j.get("unreadable"):
blockers.append("job record %s cannot be read; check it, then move or delete it by hand"
% j["path"])
else:
blockers.append("job %s (%s, %s) is open; wait for it to end or close it: python "
".agents/skills/multi-agent-mux-delegate-job/scripts/registry.py"
" --registry-dir %s close --job %s --request cancelled"
% (j["job_id"], j["agent_session"] or "-", j["status"],
j["registry_dir"], j["job_id"]))
if blockers and not me_found:
blockers.append("note: no herdr agent was found at the caller's server %r and pane %r, so"
" nothing was excluded as the calling agent" % (me_server, me_pane))
return blockers
# --------------------------------------------------------------------------- delegate target
def _installed(d: str) -> bool:
"""update.sh's own test of an installed workspace."""
return os.path.isfile(os.path.join(d, ".mam_deploy", "remove.sh")) or os.path.isfile(os.path.join(d, "remove.sh"))
def target_workspaces(workspace: str, state: dict, agent_session: str, server: str,
herdr: str, env: Mapping[str, str]) -> List[str]:
"""Every installed workspace at or above the target's cwd - an update of
any of them counts that agent (decision 1) - and its row's
control_plane.workspace_root; the caller's own workspace left out. With
no row the cwd is the live agent's on <server> (the one delivery uses),
read with the real herdr. No herdr, a server that is not running, or no
such agent gives nothing (delivery then fails on its own); a herdr that
cannot answer raises HerdrError."""
if not agent_session.startswith("herdr:"):
return []
name = agent_session[len("herdr:"):]
row = next((s for s in state.get("herdr_sessions") or []
if isinstance(s, dict) and s.get("name") == name), None)
cwds, out = [], []
if row is not None:
cwds.append((row.get("pane") or {}).get("cwd") or "")
root = (row.get("control_plane") or {}).get("workspace_root") or ""
if root:
out.append(_real(root))
elif herdr and server in running_servers(herdr, env):
for a in server_agents(herdr, server, env):
if a.get("name") in (name, sanitize_herdr_agent_name(name)):
cwds += [a.get("cwd") or "", a.get("foreground_cwd") or ""]
break
return _others(workspace, out + installed_above(cwds))
def installed_above(cwds: Iterable[str]) -> List[str]:
"""Every installed workspace at or above each cwd."""
out = []
for cwd in cwds:
d = _real(cwd) if cwd else ""
while d:
if _installed(d):
out.append(d)
if os.path.dirname(d) == d:
break
d = os.path.dirname(d)
return out
def _others(workspace: str, dirs: List[str]) -> List[str]:
me = _real(workspace)
return [d for i, d in enumerate(dirs) if d != me and d not in dirs[:i]]
# --------------------------------------------------------------------------- monitor
def monitor_state(workspace: str) -> dict:
pids = daemon.find_running(daemon.monitor_script_path(workspace))
lock = os.path.join(workspace, ".mam", "monitor.lock")
held = False
if os.path.exists(lock):
with open(lock, "a") as fh:
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fh, fcntl.LOCK_UN)
except BlockingIOError:
held = True
return {"pids": pids, "lock": "held" if held else "free"}
# --------------------------------------------------------------------------- registry backup
def backup_registry(workspace: str, dest: str) -> List[str]:
mam = os.path.join(workspace, ".mam")
db = os.path.join(mam, "agent-sessions.db")
os.makedirs(dest, exist_ok=True)
copied = []
with open(db + ".lock", "w") as lk:
fcntl.flock(lk, fcntl.LOCK_EX)
try:
if os.path.exists(db):
src = sqlite3.connect(db, timeout=60.0)
dst = sqlite3.connect(os.path.join(dest, "agent-sessions.db"))
try:
src.backup(dst)
finally:
dst.close()
src.close()
copied.append("agent-sessions.db")
for name in REGISTRY_FILES:
p = os.path.join(mam, name)
if name != "agent-sessions.db" and os.path.exists(p):
shutil.copy2(p, os.path.join(dest, name))
copied.append(name)
finally:
fcntl.flock(lk, fcntl.LOCK_UN)
return copied
def _state() -> dict:
return json.loads(os.environ.get("MAM_STATE_JSON") or "{}") or {}
def main(argv: Optional[List[str]] = None) -> int:
p = argparse.ArgumentParser(prog="lib_py.update")
sub = p.add_subparsers(dest="cmd", required=True)
s = sub.add_parser("precheck")
s.add_argument("--workspace", required=True)
s.add_argument("--herdr", required=True)
s = sub.add_parser("monitor")
s.add_argument("--workspace", required=True)
s = sub.add_parser("target-workspaces")
s.add_argument("--workspace", required=True)
s.add_argument("--agent-session", default="")
s.add_argument("--cwd", default="")
s.add_argument("--server", default="default")
s.add_argument("--herdr", default="")
s = sub.add_parser("backup-registry")
s.add_argument("--workspace", required=True)
s.add_argument("--dest", required=True)
a = p.parse_args(argv)
ws = _real(a.workspace)
if a.cmd == "precheck":
try:
state = _state()
except ValueError as exc:
print("registry: cannot parse MAM_STATE_JSON (%s)" % exc)
return 1
lines = precheck(ws, state, a.herdr, os.environ)
for line in lines:
print(line)
return 1 if lines else 0
if a.cmd == "target-workspaces" and a.cwd:
for root in _others(ws, installed_above([a.cwd])):
print(root)
return 0
if a.cmd == "target-workspaces":
try:
state = _state()
except ValueError:
state = {}
try:
roots = target_workspaces(ws, state, a.agent_session, a.server, a.herdr, os.environ)
except HerdrError as exc:
print("cannot find %s live: %s" % (a.agent_session, exc), file=sys.stderr)
return 1
for root in roots:
print(root)
return 0
if a.cmd == "monitor":
print(json.dumps(monitor_state(ws)))
return 0
for name in backup_registry(ws, a.dest):
print(name)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-create
description: "Create a new agent session (claude, antigravity/agy) in a dedicated herdr session for context-preserving long-running work. Always creates a herdr session — never backgrounds with nohup/disown. Writes the new session to .mam/agent-sessions.yaml. Use when you want to start a fresh agent (no prior UUID) for a new project workspace."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -181,7 +181,7 @@ bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh --workspace
| Exit | Meaning |
|---|---|
| 0 | Created; or `--dry-run` |
| 1 | Pre-flight failure, nothing was spawned: workspace not a directory, herdr not installed, agent CLI not in `PATH`, agent not authenticated, or a same-cwd session of the same Class A agent did not record its conversation ID within `MAM_ID_ORDER_WAIT_S` (default 600). After spawn: the task brief could not be written or the instruction injection failed (the session is rolled back) |
| 1 | Pre-flight failure, nothing was spawned: workspace not a directory, herdr not installed, agent CLI not in `PATH`, agent not authenticated, or a same-cwd session of the same Class A agent did not record its conversation ID within `MAM_ID_ORDER_WAIT_S` (default 600), or an update of the workspace is in progress (`multi-agent-mux-update`). After spawn: the task brief could not be written or the instruction injection failed (the session is rolled back) |
| 2 | Bad arguments; control-plane binding refused for a worktree; whitespace in a control-plane path that must be forwarded |
| 3 | A herdr session of that name already exists (`multi-agent-mux-resume`, or `multi-agent-mux-stop` first) |
| 4 | The YAML already has a `running` row of that name. The YAML write raises it after spawn, and the herdr session is rolled back. A row the monitor registered for this very pane while it was being created is replaced instead |
@@ -260,7 +260,7 @@ spawn() {
SESSION_UUID=""
CMD_FULL="$("$(_delegate_py_bin)" -m lib_py.agents spawn-spec "$AGENT" "$RESOLVED_BIN" "" "true" 2>/dev/null || echo "${RESOLVED_BIN} --dangerously-skip-permissions")"
[ -z "$CP_PREFIX" ] || echo "WARNING: the ~/.local/bin wrapper path bypasses herdr; the control-plane binding is recorded but not forwarded" >&2
nohup "$WRAPPER" >/dev/null 2>&1 &
( mam_update_admit_release_all; exec nohup "$WRAPPER" >/dev/null 2>&1 ) &
disown
else
HERDR_SESSION_NAME="$HERDR_SESSION_NAME" _herdr new-session -d -s "$SESSION_NAME" -x 140 -y 40 -c "$WORKSPACE" "$CP_PREFIX$CMD_FULL"
@@ -285,6 +285,10 @@ if [ "$DRY_RUN" = "1" ]; then
fi
# Issue #63: a same-cwd session of this post-discovered agent records its id first (lib.sh mam_id_order_wait).
mam_id_order_wait "$WORKSPACE" "$AGENT" "$SESSION_NAME" || exit 1
# Issue #72: no new session while an update of the control root or of an
# installed workspace the pane's cwd is in runs; held until the row is written
# (lib.sh mam_update_admit_at).
mam_update_admit_at "$WORKSPACE" || exit 1
HERDR_EPOCH=$(date +%s)
NOW_ISO=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
spawn
@@ -638,6 +642,7 @@ snap['cwd'] = os.environ['PANE_CWD']
print(f"appended: {name}", flush=True)
PYEOF
ROW_APPENDED=1
mam_update_admit_release_all
echo
echo "=== created ==="
@@ -767,6 +772,6 @@ fi
trap - EXIT
echo "agent-sessions.yaml updated"
echo
echo "Attach: herdr session attach $SESSION_NAME"
echo "Attach: $(session_attach_command "$SESSION_NAME" "${HERDR_SESSION_NAME:-default}")"
echo "Delete: use multi-agent-mux-stop skill"
echo "Resume: use multi-agent-mux-resume skill (after first message creates a session id)"
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-delegate-job
description: "Delegate a unit of work to any autonomous agent (claude-code, hermes, agy, grok-build, codex, or a human) and observe it asynchronously over an MQTT event channel. Supported roles include orchestrator, worker, and reviewer."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos, windows]
@@ -301,6 +301,26 @@ cmd_submit() {
local reg_dry=()
if [[ "$DRY_RUN" == "1" ]]; then reg_dry=(--dry-run); fi
# Issue #72: no job is registered while an update of this registry's
# workspace or of one the target is in (lib_py.update target-workspaces:
# its row's, or with no row the live agent's on the server delivery uses)
# runs; each is held, fd 7 then 20.., until the record is written (lib.sh
# mam_update_admit). A dry-run record is never dispatched and is not
# counted by the update.
local admit_fds=() target_ws tws fd=20
if [[ "$DRY_RUN" != "1" ]]; then
target_ws="$(MAM_STATE_JSON="$(load_state_json)" PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" "$PY" -m lib_py.update \
target-workspaces --workspace "$WORKSPACE_ROOT" --agent-session "$AGENT_SESSION" \
--server "$(_is_remote "$AGENT_SESSION" || resolve_herdr_session "${AGENT_SESSION#herdr:}" "$WORKDIR" 2>/dev/null)" \
--herdr "$(_resolve_real_herdr_path || true)")" \
|| { echo "ERROR: cannot resolve the workspace of $AGENT_SESSION; nothing was registered" >&2; exit 1; }
mam_update_admit "$WORKSPACE_ROOT" || exit 1
while IFS= read -r tws; do
[[ -n "$tws" ]] || continue
mam_update_admit "$tws" "$fd" || exit 1
admit_fds+=("$fd"); fd=$((fd + 1))
done <<< "$target_ws"
fi
# 1) register job (prints the new job id)
JOB_ID="$("$PY" "$SCRIPT_DIR/scripts/registry.py" --registry-dir "$REGISTRY_DIR" register \
"${reg_prompt[@]}" --agent "$AGENT" --agent-session "$AGENT_SESSION" --role "$DELEGATE_ROLE" \
@@ -310,6 +330,10 @@ cmd_submit() {
--job-type "$TYPE" --reviewer "$REVIEWER" --reviewer-session "$REVIEWER_SESSION" \
--max-iterations "$MAX_ITERATIONS" ${reg_parent[@]+"${reg_parent[@]}"} ${reg_dry[@]+"${reg_dry[@]}"})"
echo "registered job: $JOB_ID"
if [[ "$DRY_RUN" != "1" ]]; then
mam_update_admit_release
for fd in ${admit_fds[@]+"${admit_fds[@]}"}; do mam_update_admit_release "$fd"; done
fi
# 1-1) Provision job directory and write direct brief.md (MAM Job Restructuring)
local job_dir="$REGISTRY_DIR/$JOB_ID"
@@ -1018,7 +1042,7 @@ inject_agent() {
return 1
fi
echo "작업이 세션 '$sess'에 전송되었습니다. (연결하려면: HERDR_SESSION_NAME=$HERDR_SESSION_NAME herdr agent attach $sess — lib.sh를 source한 셸에서 실행)"
echo "작업이 세션 '$sess'에 전송되었습니다. (연결하려면: $(session_attach_command "$sess" "${HERDR_SESSION_NAME:-default}"))"
trap - EXIT
return 0
}
+1 -1
View File
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-loop
description: "Run an autonomous planning-execution-review loop using multiple agents (Planner, Creator, Reviewers) in the workspace. Automatically orchestrates plan discussion, code changes, and peer reviews until a unanimous PASS is achieved or the maximum iteration limit is reached."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -244,6 +244,9 @@ log_error() {
echo -e "\033[1;31m[✗]\033[0m $1"
}
# Issue #72: no loop starts while an update runs; held until the loop marker
# is published (lib.sh mam_update_admit), released before the loop body runs.
mam_update_admit "$MAM_REAL_ROOT" || exit 1
_mam_acquire_rc=0
mam_acquire_loop_lock "$MAM_LOOP_MARKER" || _mam_acquire_rc=$?
case "$_mam_acquire_rc" in
@@ -262,6 +265,7 @@ case "$_mam_acquire_rc" in
;;
esac
trap _mam_release_guard EXIT INT TERM HUP
mam_update_admit_release
rm -f "$MAM_REAL_ROOT/.agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job".*.tmp 2>/dev/null || true
# --all-reviewer silently takes precedence over an explicit --reviewer list;
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-monitor
description: "Run a long-lived reconciler that watches .mam/agent-sessions.yaml against the actual herdr/agent runtime state and reconciles them. Use when you want live visibility into which agent sessions are running, which are dead, which have stale YAML entries, and which have new session ids that haven't been recorded yet. Runs as a persistent loop (`reconcile.sh --subscribe`) that keeps going until it times out, idles out, or is interrupted."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-orc-onboard
description: "Register current or specified orchestrator session UUID into agent-sessions.yaml orchestrator_uuids list to prevent sub-agent discovery capture."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -257,6 +257,11 @@ if [ "$DRY_RUN" = true ]; then
exit 0
fi
# Issue #72 (U2): no registry write while an update runs - a rollback would
# restore the registry without it. Held until the write is done (lib.sh
# mam_update_admit).
mam_update_admit || exit 1
# Issue #13: the orchestrator has no herdr_sessions row, so stop_session.sh
# --purge-conversation can never reach its conversation. Purge FIRST, through
# the adapter (`lib_py.agents purge`): a refusal because the conversation is
@@ -304,6 +309,7 @@ elif op == "remove":
uuids = [u for u in uuids if u != target]
d["orchestrator_uuids"] = uuids
PYEOF
mam_update_admit_release
case "$MODE" in
add) verb="added" ;;
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-resume
description: "Resume an existing agent (claude, antigravity/agy) conversation by UUID into a herdr session. Reads .mam/agent-sessions.yaml for the saved session/conversation id, spawns (or reuses) a herdr session of the matching name, and runs `claude -r <id>` or `agy --conversation <id>` inside. Use when you want to reattach to a previous session's context, or revive a session whose herdr died but the agent's conversation is still on disk."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -60,7 +60,7 @@ If the resolved id is empty → for Class A agents (`agy`, `hermes`, `opencode`,
bash .agents/skills/multi-agent-mux-resume/scripts/resume_session.sh --workspace <path> --agent <agent> --session <name>
```
`resume_session.sh` does the whole flow: original-cwd check, control-plane binding, the session-id decision (`lib_py/agents/session_ids.py`), the herdr-alive check, the agent command (adapter `resume-spec`, or `spawn-spec` for a Class A fresh spawn), spawn (just before it, the session's own empty shell left by a server restart is closed: same session id, no agent, nothing running; `lib_py/herdr_restore.py`), start-up dialogs, and the YAML update (`update_yaml_resumed.sh`, which also publishes a `progress "resumed"` event when the row has a `delegate_job_id`). It never attaches; attaching is a separate step that names the server, because the server the script exports does not reach the calling shell and the native `herdr` binary does not route on environment variables: the row's `attach_command`, `herdr --session <server> agent attach <name>` (`<server>` = the `--herdr-session` you passed, or the row's `herdr_session`; `<name>` = the name herdr registered).
`resume_session.sh` does the whole flow: original-cwd check, control-plane binding, the session-id decision (`lib_py/agents/session_ids.py`), the herdr-alive check, the agent command (adapter `resume-spec`, or `spawn-spec` for a Class A fresh spawn), spawn (just before it, the session's own empty shell left by a server restart is closed: same session id, no agent, nothing running; `lib_py/herdr_restore.py`), start-up dialogs, and the YAML update (`update_yaml_resumed.sh`, which also publishes a `progress "resumed"` event when the row has a `delegate_job_id`), then the control root's monitor daemon (`start_watchdog`, as create; not with `MAM_NO_SUBSCRIBER_DAEMON=1`). It never attaches; attaching is a separate step that names the server, because the server the script exports does not reach the calling shell and the native `herdr` binary does not route on environment variables: the row's `attach_command`, `herdr --session <server> agent attach <name>` (`<server>` = the `--herdr-session` you passed, or the row's `herdr_session`; `<name>` = the name herdr registered).
| Option | Meaning |
|---|---|
@@ -75,11 +75,11 @@ bash .agents/skills/multi-agent-mux-resume/scripts/resume_session.sh --workspace
| Exit | Meaning |
|---|---|
| 0 | Resumed; or already running, so only the row was updated (no attach); or `--dry-run` |
| 1 | From the checks before spawn, nothing was changed: no saved session for claude/grok (use `multi-agent-mux-create` first), the row's id is also claimed by another row, a same-cwd Class A session did not record its id within `MAM_ID_ORDER_WAIT_S` (default 600), or agent binary missing/not executable. **The YAML update after spawn (`update_yaml_resumed.sh`) also exits 1** (YAML not found) |
| 1 | From the checks before spawn, nothing was changed: no saved session for claude/grok (use `multi-agent-mux-create` first), the row's id is also claimed by another row, a same-cwd Class A session did not record its id within `MAM_ID_ORDER_WAIT_S` (default 600), agent binary missing/not executable, or an update of the workspace is in progress (`multi-agent-mux-update`). **The YAML update after spawn (`update_yaml_resumed.sh`) also exits 1** (YAML not found) |
| 2 | From the checks before spawn, nothing was changed: bad arguments, `--workspace` differs from the original cwd or it no longer exists, control-plane binding refused, whitespace in the control-plane path. **The YAML update after spawn also exits 2** (bad arguments) |
| other | A later step failed (herdr spawn, start-up dialogs, YAML update) and its code is passed through. resume does not roll back |
A non-zero exit after the spawn can leave the herdr session running while the row is still `stopped`. Before retrying or creating a new session, check with `multi-agent-mux-status`. The `ERROR:` line on stderr tells the two apart: the pre-spawn checks' messages are in `resume_session.sh` (lines 50-243).
A non-zero exit after the spawn can leave the herdr session running while the row is still `stopped`. Before retrying or creating a new session, check with `multi-agent-mux-status`. The `ERROR:` line on stderr tells the two apart: the pre-spawn checks' messages are in `resume_session.sh` (lines 50-259).
## Original cwd only (Issue #21 phase 2)
@@ -154,18 +154,34 @@ if [ -z "$UUID" ]; then
esac
fi
# Issue #72: a resumed session gets the monitor daemon create starts - the
# control root's (Issue #21 A8), also for a worktree --workspace. Called after
# the row is written; MAM_NO_SUBSCRIBER_DAEMON=1 starts none. Never fails the resume.
_ensure_monitor_daemon() {
local wd_pid
if wd_pid=$(start_watchdog "resume" "$WORKSPACE_ROOT"); then
echo "watchdog PID: $wd_pid"
else
echo "WARNING: could not start the monitor daemon of '$WORKSPACE_ROOT'; the session itself was resumed" >&2
fi
}
# 2. If herdr is alive, print warning or attach.
if [ "$HERDR_ALIVE" = "1" ]; then
if [ "${DRY_RUN:-0}" = "1" ]; then
echo "[dry-run] herdr '$SESSION_NAME' already running — nothing to validate"
exit 0
fi
# Issue #72: no registry write while an update runs (lib.sh mam_update_admit).
mam_update_admit || exit 1
echo "herdr '$SESSION_NAME' already running."
# Just update YAML to make sure it's set to running
CONTROL_PLANE_JSON="$CP_JSON" bash "$(dirname "${BASH_SOURCE[0]}")/update_yaml_resumed.sh" \
--session "$SESSION_NAME" --uuid "$UUID" --agent "$AGENT" --workspace "$WORKSPACE" \
--herdr-session "$HERDR_SESSION_NAME" --already-running \
${HERDR_WORKSPACE_OPT:+--herdr-workspace "$HERDR_WORKSPACE_OPT"}
mam_update_admit_release
_ensure_monitor_daemon
exit 0
fi
@@ -238,6 +254,11 @@ if [ "${DRY_RUN:-0}" = "1" ]; then
exit 0
fi
# Issue #72: no new session while an update of the control root or of an
# installed workspace the pane's cwd is in runs; held until the row is written
# (lib.sh mam_update_admit_at).
mam_update_admit_at "$WORKSPACE" || exit 1
# Issue #63: wait if another same-cwd session of this post-discovered agent
# is still being created or has not recorded its id yet (lib.sh mam_id_order_wait).
mam_id_order_wait "$WORKSPACE" "$AGENT" "$SESSION_NAME" "$UUID" || exit 1
@@ -274,5 +295,7 @@ CONTROL_PLANE_JSON="$CP_JSON" bash "$(dirname "${BASH_SOURCE[0]}")/update_yaml_r
--herdr-session "$HERDR_SESSION_NAME" --cmd-full "$CMD_FULL" \
${DROP_FLAG:+$DROP_FLAG} \
${HERDR_WORKSPACE_OPT:+--herdr-workspace "$HERDR_WORKSPACE_OPT"}
mam_update_admit_release_all
_ensure_monitor_daemon
echo "Successfully resumed $SESSION_NAME ($AGENT)"
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-status
description: "Read-only instant snapshot of all agent herdr sessions — name, YAML status, herdr alive, pane cmd/cwd, resume UUID on disk, and any drift. No mutation. Reuses reconcile.sh --dry-run for the diff logic. Use when you want to know 'what's running RIGHT NOW' without spinning up the monitor loop."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
+2 -2
View File
@@ -1,7 +1,7 @@
---
name: multi-agent-mux-stop
description: "Stop an agent herdr session (claude, antigravity/agy) and update .mam/agent-sessions.yaml. Default stops gracefully and marks status=stopped with conversation preserved for resume. Does NOT delete on-disk conversation artifacts (jsonl/db) — those are preserved unless --purge-conversation is passed. Use when ending a work session, switching to a different one, or cleaning up before a fresh start."
version: 4.11.0
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
@@ -58,7 +58,7 @@ bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh \
| Exit | Meaning |
|---|---|
| 0 | Stopped; or already `stopped` (no-op, except with `--purge-conversation`, which still purges) |
| 1 | From the checks before the stop, nothing was changed: YAML not found, its row could not be read, or the session is neither registered nor on the herdr server. **The registry update after the stop also exits 1** (the row disappeared from the YAML during the run) |
| 1 | From the checks before the stop, nothing was changed: YAML not found, its row could not be read, the session is neither registered nor on the herdr server, or an update of the workspace is in progress (`multi-agent-mux-update`). **The registry update after the stop also exits 1** (the row disappeared from the YAML during the run) |
| 2 | Bad arguments (including the retired `--mode` / `--capture-id` / `--graceful`), or `--agent` omitted and not resolvable |
| 3 | `--purge-conversation` without `--yes` |
| 4 | Purge aborted: the herdr session, or an agent process still holding the conversation, survived the kill chain. Records preserved: the registry row and the conversation files are untouched, though the session itself may already be down |
@@ -100,6 +100,11 @@ fi
[ -n "$SESSION_NAME" ] || { echo "ERROR: --session required" >&2; usage; exit 2; }
[ -f "$AGENT_SESSIONS_YAML" ] || { echo "ERROR: $AGENT_SESSIONS_YAML not found" >&2; exit 1; }
# Issue #72 (U2): no registry write while an update runs - a rollback would
# restore the registry without it. Held until the row is written (lib.sh
# mam_update_admit).
mam_update_admit || exit 1
# Implement the purging-<session> file lock mechanism
if [ "$PURGE" = "1" ]; then
touch "$WORKSPACE_ROOT/.mam/purging-$SESSION_NAME"
@@ -332,7 +337,7 @@ if [ "$PURGE" = "1" ] && [ "$HERDR_ALIVE" = "1" ]; then
if herdr has-session -t "$SESSION_NAME" 2>/dev/null; then
echo "ERROR: session '$SESSION_NAME' is still alive after the kill chain." >&2
echo " Refusing registry removal — records preserved (no state was modified)." >&2
echo " Diagnose the stuck TUI (herdr session attach '$SESSION_NAME'), then re-run" >&2
echo " Diagnose the stuck TUI ($(session_attach_command "$SESSION_NAME" "${HERDR_SESSION_NAME:-default}")), then re-run" >&2
echo " stop_session.sh --purge-conversation --yes (retry is safe/idempotent)." >&2
delegate_publish_event "$DELEGATE_JOB_ID" cancelled "purge aborted: session still alive; no state was modified" "$DELEGATE_JOB_TOKEN" "$DELEGATE_JOB_ATTEMPT"
exit 4
@@ -453,6 +458,7 @@ if purge:
else:
print(f"updated: {name} status={target['status']}", flush=True)
PYEOF
mam_update_admit_release
delegate_publish_event "$DELEGATE_JOB_ID" cancelled "session stopped by operator before job completion" "$DELEGATE_JOB_TOKEN" "$DELEGATE_JOB_ATTEMPT"
@@ -0,0 +1,30 @@
---
name: multi-agent-mux-update
description: "Update an installed MAM workspace in place to the latest version. Refuses while a session, an open job, a review loop or an unknown monitor is live; stops the monitor daemon and starts it again afterwards; rolls everything back if the install fails; converts the registry with the newly installed code. Use instead of the removed .mam_deploy/update.sh."
version: 4.12.0
author: godopu
license: MIT
platforms: [linux, macos]
environments: [terminal, herdr]
metadata:
hermes:
tags: [update, install, registry, migration]
related_skills: [multi-agent-mux-stop, multi-agent-mux-resume, multi-agent-mux-monitor]
---
# Multi-Agent Update
```bash
bash .agents/skills/multi-agent-mux-update/scripts/update.sh --yes
```
| Item | Behaviour |
|---|---|
| Where | An installed workspace only (`.mam_deploy/remove.sh` present). A git checkout updates with `git pull`. |
| Confirmation | `[y/N]` prompt; without a terminal `-y`/`--yes` is required. There is no force option. |
| Refused when | a running registry row, a live herdr agent whose cwd is in the workspace (any running server), an open job (an unreadable job JSON or jobs directory included, `dry_run` excluded), a live loop marker, `.mam/monitor.lock` held by a monitor the skills did not start, or another update. The calling agent (its herdr server + pane, row and one running job) is not counted. Each refusal lists the cleanup; nothing is changed. |
| During the update | create, resume, `run_loop.sh`, delegate submit, stop and orc-onboard refuse (`.mam/update-in-progress`, `.mam/update-admission.lock`). |
| Monitor | stopped first, started again afterwards only if it ran (`MAM_NO_SUBSCRIBER_DAEMON=1`: not started). |
| Install | the installer at `MAM_INSTALLER_URL` (default: Gitea `main`) runs in place; `.mam` is kept. On failure `.agents/`, `.mam_deploy/`, root files, `.venv`, the installer's `.mam` files and the registry are restored from `.mam/update-backup/`. |
| Registry | converted by the new code (`lib_py.registry_format`); a failed conversion keeps the install and exits nonzero. |
| Exit | `0` updated (or cancelled at the prompt), `1` refused or failed, `2` bad arguments; a failed registry conversion after a good install exits with the converter's nonzero code. |
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
# install_phase.sh — multi-agent-mux-update, the part that changes the tree (Issue #72, decision 5).
#
# install_phase.sh run <workspace> <monitor_was_running 0|1> <tmpdir>
# install_phase.sh restore <workspace> [--skip-registry]
#
# update.sh copies this file outside the workspace and execs the copy (same
# pid and start time, so its .mam/update-in-progress marker stays live). From
# here on the tree is being replaced, so this calls no lib.sh or lib_py: it runs
# the installer in place, and on failure restores .mam/update-backup/ and starts
# the monitor again with the restored (old, whole) tree's start_watchdog. On
# success phase 2 is the NEW tree's post_update.sh, in a child process. It ends
# by removing the backup and the marker (only its own: pid + lstart).
# Must run on macOS bash 3.2.
set -uo pipefail
MODE="${1:-}"
WS="${2:-}"
[ -n "$MODE" ] && [ -d "$WS" ] || { echo "usage: $0 run|restore <workspace> ..." >&2; exit 2; }
BK="$WS/.mam/update-backup"
set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
# Puts back every backed-up item, removes the ones that did not exist before,
# then (unless --skip-registry) the registry: the DB from the backup, its
# -wal/-shm removed (macOS keeps them after close).
restore() {
local rel n
[ -f "$BK/items" ] || { echo "ERROR: $BK/items is missing" >&2; return 1; }
while IFS= read -r rel; do
[ -n "$rel" ] || continue
rm -rf "${WS:?}/$rel" || return 1
grep -Fqx -- "$rel" "$BK/absent" 2>/dev/null && continue
mkdir -p "$(dirname "$WS/$rel")" && cp -pPR "$BK/files/$rel" "$WS/$rel" || return 1
done < "$BK/items"
[ "${1:-}" = "--skip-registry" ] && return 0
for n in agent-sessions.yaml agent-sessions.yaml.bak agent-sessions.db; do
rm -f "$WS/.mam/$n" || return 1
if [ -f "$BK/registry/$n" ]; then
cp -p "$BK/registry/$n" "$WS/.mam/$n" || return 1
fi
done
rm -f "$WS/.mam/agent-sessions.db-wal" "$WS/.mam/agent-sessions.db-shm"
}
release_marker() {
local m="$WS/.mam/update-in-progress" pid lstart cur
[ -f "$m" ] || return 0
pid="$(grep '^pid=' "$m" | head -n 1 | cut -d= -f2-)"
lstart="$(grep '^lstart=' "$m" | head -n 1 | cut -d= -f2-)"
cur="$(ps -p $$ -o lstart= 2>/dev/null | tr -s ' ' | sed 's/^ *//;s/ *$//')"
if [ "$pid" = "$$" ] && [ "$lstart" = "$cur" ]; then
rm -f "$m"
fi
}
restart_old_monitor() {
[ "$1" = "1" ] || return 0
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
return 0
fi
local pid
if pid="$(bash -c 'source "$1/.agents/skills/lib.sh" && start_watchdog update "$1"' _ "$WS")"; then
echo "monitor: restarted (PID $pid)"
else
echo "ERROR: could not restart the monitor daemon" >&2
fi
}
case "$MODE" in
restore)
restore "${3:-}"
exit $?
;;
run) ;;
*) echo "usage: $0 run|restore <workspace> ..." >&2; exit 2 ;;
esac
MON="${3:-0}"
TMPD="${4:-}"
trap '[ -z "$TMPD" ] || rm -rf "$TMPD"' EXIT
if ! set_state installing; then
# Nothing was installed yet: drop the backup and put the monitor back as phase 1 found it.
echo "ERROR: cannot write $BK/state; nothing was changed" >&2
rm -rf "$BK"
restart_old_monitor "$MON"
release_marker
exit 1
fi
URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
echo "install: $URL"
irc=0
if command -v curl >/dev/null 2>&1; then
curl -fsSL "$URL" | bash -s -- "$WS" || irc=$?
elif command -v wget >/dev/null 2>&1; then
wget -qO- "$URL" | bash -s -- "$WS" || irc=$?
else
echo "ERROR: neither curl nor wget is available to fetch the installer" >&2
irc=1
fi
if [ "$irc" -ne 0 ]; then
echo "ERROR: the install failed (rc $irc); rolling back" >&2
if restore; then
rm -rf "$BK"
echo "rolled back: the workspace is at the version it had before the update"
restart_old_monitor "$MON"
else
echo "ERROR: the rollback failed; the backup is kept at $BK (the next update restores it)" >&2
fi
release_marker
exit 1
fi
set_state installed
POST="$WS/.agents/skills/multi-agent-mux-update/scripts/post_update.sh"
prc=0
if [ -f "$POST" ]; then
bash "$POST" --monitor "$MON" || prc=$?
else
echo "ERROR: the installed version has no $POST; the registry converts on its next write and the monitor was not restarted" >&2
prc=1
fi
rm -rf "$BK"
release_marker
[ "$prc" -eq 0 ] && echo "Update complete."
exit "$prc"
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# post_update.sh — multi-agent-mux-update phase 2 (Issue #72, decision 5).
# post_update.sh --monitor <0|1>
#
# Run by install_phase.sh from the NEWLY installed tree, in its own process:
# converts the registry with the new code's entry point (lib_py.registry_format,
# #71) and starts the monitor again if it ran before the update. Exit 0, or
# nonzero when the conversion or the monitor restart failed (the install is
# kept either way: the registry converts again on its next write).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
export WORKSPACE_ROOT="$WS"
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
# shellcheck disable=SC1091
source "$WS/.agents/skills/lib.sh"
MON=0
while [ $# -gt 0 ]; do
case "$1" in
--monitor) MON="$2"; shift 2 ;;
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
esac
done
rc=0
if [ ! -f "$AGENT_SESSIONS_YAML" ] && [ ! -f "$WS/.mam/agent-sessions.db" ]; then
echo "registry: nothing to convert (no registry)"
else
"$(_delegate_py_bin)" -m lib_py.registry_format --yaml "$AGENT_SESSIONS_YAML" || rc=$?
if [ "$rc" -ne 0 ]; then
echo "ERROR: the registry was not converted (rc $rc); the install is kept and the next registry write converts it" >&2
fi
fi
if [ "$MON" = "1" ]; then
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
# shellcheck disable=SC2086
elif pid="$(start_watchdog update "$WS")" && mam_monitor_started "$WS" $pid; then
echo "monitor: restarted (PID $pid)"
else
echo "ERROR: could not restart the monitor daemon" >&2
rc=1
fi
fi
exit "$rc"
+267
View File
@@ -0,0 +1,267 @@
#!/usr/bin/env bash
# update.sh — multi-agent-mux-update (Issue #72): update an installed workspace in place.
# Usage:
# bash .agents/skills/multi-agent-mux-update/scripts/update.sh [-y|--yes]
#
# Phase 1 (this script, the installed code): publish .mam/update-in-progress
# under the admission lock (lib.sh mam_update_admit is the other side), refuse
# while anything live could see the tree change (lib_py/update.py precheck, a
# live loop marker, a monitor.lock held by a monitor the skills did not start),
# stop the monitor daemon, back up what the install can change into
# .mam/update-backup/, then exec a copy of install_phase.sh outside the tree.
# That copy runs the installer (MAM_INSTALLER_URL) in place, rolls back on
# failure, and runs phase 2 (post_update.sh of the NEW tree) on success.
#
# Exit codes:
# 0 = updated (registry converted or nothing to convert)
# 1 = refused or failed; what was changed and restored is printed
# 2 = invalid args
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
export WORKSPACE_ROOT="$WS"
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
# shellcheck disable=SC1091
source "$WS/.agents/skills/lib.sh"
# shellcheck disable=SC1091
source "$SKILL_DIR/multi-agent-mux-loop/scripts/loop_lock.sh"
YES=0
while [ $# -gt 0 ]; do
case "$1" in
-y|--yes) YES=1; shift ;;
-h|--help) sed -n '2,4p' "${BASH_SOURCE[0]}"; exit 0 ;;
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
esac
done
MAM="$WS/.mam"
MARKER="$MAM/update-in-progress"
BK="$MAM/update-backup"
PY="$(_delegate_py_bin)"
if [ ! -f "$WS/.mam_deploy/remove.sh" ] && [ ! -f "$WS/remove.sh" ]; then
echo "ERROR: $WS is not an installed MAM workspace (no .mam_deploy/remove.sh); a git checkout updates with git pull" >&2
exit 1
fi
if [ "$YES" != "1" ]; then
if [ ! -t 0 ]; then
echo "ERROR: not a terminal; run with -y/--yes" >&2
exit 1
fi
ans=""
read -r -p "Update MAM in $WS to the latest version? [y/N]: " ans || ans=""
case "$ans" in
y|Y|yes|Yes|YES) ;;
*) echo "Update cancelled."; exit 0 ;;
esac
fi
# --- the marker, published under the exclusive admission lock (decision 8) ---
mkdir -p "$MAM"
exec 7>>"$MAM/update-admission.lock"
case "$("$PY" -m lib_py.pane_lock acquire --fd 7 --wait-env MAM_UPDATE_ADMISSION_WAIT_S)" in
locked) ;;
timeout)
echo "ERROR: a create, resume, loop, delegate submit, stop or orc-onboard kept the admission lock past MAM_UPDATE_ADMISSION_WAIT_S; nothing was changed, try again" >&2
exit 1 ;;
*)
echo "ERROR: the admission lock $MAM/update-admission.lock could not be decided (lib_py.pane_lock); nothing was changed" >&2
exit 1 ;;
esac
if [ -f "$MARKER" ] && [ "$(_mam_lock_holder_state "$MARKER")" = "stale" ]; then
echo "NOTE: stale update marker (pid $(_mam_lock_field "$MARKER" pid)) taken over; its update is gone"
fi
marker_rc=0
mam_acquire_loop_lock "$MARKER" || marker_rc=$?
exec 7>&-
case "$marker_rc" in
0) ;;
3) echo "ERROR: another update is in progress (pid $(_mam_lock_field "$MARKER" pid)); nothing was changed" >&2; exit 1 ;;
*) echo "ERROR: cannot tell whether another update is in progress (ps unavailable?); nothing was changed" >&2; exit 1 ;;
esac
TMPD=""
MONITOR_WAS_RUNNING=0
RESTART_MONITOR=0
_phase1_exit() {
local rc=$?
if [ "$RESTART_MONITOR" = "1" ]; then
_restart_monitor || true
fi
mam_release_loop_lock "$MARKER" || true
[ -z "$TMPD" ] || rm -rf "$TMPD"
exit "$rc"
}
trap _phase1_exit EXIT
_restart_monitor() {
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
return 0
fi
local pid
pid=$(start_watchdog update "$WS") || { echo "ERROR: could not restart the monitor daemon" >&2; return 1; }
# shellcheck disable=SC2086
mam_monitor_started "$WS" $pid || { echo "ERROR: could not restart the monitor daemon (PID ${pid:-none} is not its monitor)" >&2; return 1; }
echo "monitor: restarted (PID $pid)"
}
_monitor_json() {
"$PY" -m lib_py.update monitor --workspace "$WS"
}
_json_field() {
"$PY" -c 'import json,sys; v=json.load(sys.stdin)[sys.argv[1]]; print(" ".join(v) if isinstance(v, list) else v)' "$1"
}
# --- pre-check: refuse before any change (decisions 1, 7; U3) ---
BLOCKERS=""
_block() { BLOCKERS="${BLOCKERS} - $1
"; }
if [ -f "$MAM/loop-guard-active" ]; then
loop_st="$(_mam_lock_holder_state "$MAM/loop-guard-active")"
case "$loop_st" in
stale) echo "NOTE: stale loop marker $MAM/loop-guard-active (its loop is gone); not a reason to refuse" ;;
*) _block "a review loop is running (pid $(_mam_lock_field "$MAM/loop-guard-active" pid), holder $loop_st); wait for it to end" ;;
esac
fi
STATE_JSON=""
if ! STATE_JSON="$(load_state_json strict)"; then
_block "the registry $AGENT_SESSIONS_YAML cannot be read; fix it first"
STATE_JSON='{}'
fi
REAL_HERDR="$(_resolve_real_herdr_path || true)"
if [ -z "$REAL_HERDR" ]; then
_block "no herdr binary found on PATH, so the live agents cannot be listed"
else
pre_out=""
pre_rc=0
pre_out="$(MAM_STATE_JSON="$STATE_JSON" "$PY" -m lib_py.update precheck --workspace "$WS" --herdr "$REAL_HERDR")" || pre_rc=$?
if [ "$pre_rc" -ne 0 ]; then
while IFS= read -r line; do
[ -n "$line" ] && _block "$line"
done <<< "${pre_out:-lib_py.update precheck failed (rc $pre_rc)}"
fi
fi
MON_JSON="$(_monitor_json)" || { _block "cannot inspect the monitor daemon (lib_py.update monitor)"; MON_JSON='{"pids": [], "lock": "free"}'; }
MON_PIDS="$(printf '%s' "$MON_JSON" | _json_field pids)"
MON_LOCK="$(printf '%s' "$MON_JSON" | _json_field lock)"
if [ -z "$MON_PIDS" ] && [ "$MON_LOCK" = "held" ]; then
_block "$MAM/monitor.lock is held by a monitor the skills did not start (not 'bash $WS/.agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh --subscribe'); stop it (find it with: pgrep -af reconcile.sh) and try again"
fi
if [ -n "$BLOCKERS" ]; then
echo "ERROR: update refused; nothing was changed:" >&2
printf '%s' "$BLOCKERS" >&2
exit 1
fi
# --- stop the monitor daemon (decision 2) ---
_monitor_gone() {
local p
for p in "$@"; do
kill -0 "$p" 2>/dev/null && return 1
done
[ "$(_monitor_json | _json_field lock)" = "free" ]
}
_wait_monitor_gone() {
local deadline=$(( $(date +%s) + $1 ))
shift
while :; do
_monitor_gone "$@" && return 0
[ "$(date +%s)" -lt "$deadline" ] || return 1
sleep 0.5
done
}
if [ -n "$MON_PIDS" ]; then
MONITOR_WAS_RUNNING=1
# shellcheck disable=SC2086
set -- $MON_PIDS
kids=""
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
echo "monitor: stopping the daemon (PID $MON_PIDS)"
kill -TERM "$@" 2>/dev/null || true
if ! _wait_monitor_gone "$(mam_int_knob MAM_UPDATE_MONITOR_STOP_S 30 1)" "$@"; then
# SIGKILL only what is still this workspace's monitor: a PID that ended
# during the wait may already be another process's.
still=""
for p in $(_monitor_json | _json_field pids); do
case " $* " in *" $p "*) still="$still $p" ;; esac
done
# shellcheck disable=SC2086
set -- $still
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
echo "WARNING: the monitor did not stop within MAM_UPDATE_MONITOR_STOP_S; sending SIGKILL to $* $kids" >&2
# shellcheck disable=SC2086
kill -KILL "$@" $kids 2>/dev/null || true
# shellcheck disable=SC2086
if _wait_monitor_gone 5 "$@" $kids; then
echo "ERROR: the monitor did not stop in time; it was killed and is started again. Nothing else was changed." >&2
_restart_monitor || true
exit 1
fi
echo "ERROR: could not restore the monitor state: PIDs still alive or monitor.lock still held ($*$kids; lock $(_monitor_json | _json_field lock)). Nothing else was changed." >&2
exit 1
fi
RESTART_MONITOR=1
fi
TMPD="$(mktemp -d "${TMPDIR:-/tmp}/mam-update.XXXXXX")"
cp "$SCRIPT_DIR/install_phase.sh" "$TMPD/install_phase.sh"
# --- a leftover backup of an update that died (U4) ---
if [ -e "$BK" ]; then
left_st="$(cat "$BK/state" 2>/dev/null || true)"
case "$left_st" in
""|backing-up|installed)
echo "NOTE: removing the backup a previous update left (state ${left_st:-none}); nothing to restore"
rm -rf "$BK" ;;
installing)
echo "NOTE: a previous update died during its install; restoring its backup (the registry is left as it is)"
if ! bash "$TMPD/install_phase.sh" restore "$WS" --skip-registry; then
RESTART_MONITOR=0
echo "ERROR: restoring the previous update's backup failed; it is kept at $BK. Nothing more was changed; the monitor stays stopped." >&2
exit 1
fi
rm -rf "$BK"
echo "the previous update was rolled back" ;;
*)
echo "ERROR: $BK has an unknown state '$left_st'; it is kept and nothing was changed. Check it and remove it by hand." >&2
exit 1 ;;
esac
fi
# --- the rollback backup (decisions 3, 10; U4) ---
_set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
ITEMS=".agents .mam_deploy AGENTS.md MESSAGING.md BOOTSTRAP.md BOOTSTRAP.ko.md .mam.env.example .mam.env .gitignore remove.sh update.sh .venv .mam/install_manifest.txt .mam/install_state .mam/asset_hashes.txt .mam/version.txt .mam/base .mam/skill-backups"
_backup() {
mkdir -p "$BK/files" || return 1
_set_state backing-up || return 1
: > "$BK/items" && : > "$BK/absent" || return 1
local rel
for rel in $ITEMS; do
printf '%s\n' "$rel" >> "$BK/items" || return 1
if [ -e "$WS/$rel" ] || [ -L "$WS/$rel" ]; then
mkdir -p "$BK/files/$(dirname "$rel")" && cp -pPR "$WS/$rel" "$BK/files/$rel" || return 1
else
printf '%s\n' "$rel" >> "$BK/absent" || return 1
fi
done
"$PY" -m lib_py.update backup-registry --workspace "$WS" --dest "$BK/registry" >/dev/null
}
echo "backup: $BK"
if ! _backup; then
rm -rf "$BK"
echo "ERROR: the backup could not be made; nothing was changed" >&2
exit 1
fi
# --- hand over to the copy (decision 5): same pid, so the marker stays live ---
RESTART_MONITOR=0
trap - EXIT
exec bash "$TMPD/install_phase.sh" run "$WS" "$MONITOR_WAS_RUNNING" "$TMPD"
+1 -1
View File
@@ -2,7 +2,7 @@
본 문서는 `herdr_agent_orchestration` 오케스트레이션 및 메시징 백플레인 워크플로우를 새로운 프로젝트에 도입하여 이식하고, 새로운 개발자/에이전트가 초기 가동을 시작할 때 수행해야 하는 환경 설정 및 구축 절차를 안내합니다.
[![Version: v4.11.0](https://img.shields.io/badge/version-v4.11.0-blue.svg)](VERSIONS.md)
[![Version: v4.12.0](https://img.shields.io/badge/version-v4.12.0-blue.svg)](VERSIONS.md)
새로운 에이전트는 이 안내서에 기술된 절차를 순차적으로 실행하여 초기 환경을 안정적으로 설정할 수 있습니다.
+1 -1
View File
@@ -2,7 +2,7 @@
This document guides you through the setup and initialization procedures required to adopt the `herdr_agent_orchestration` orchestration and messaging backplane workflow in a new project, enabling a new developer or agent to get up and running quickly.
[![Version: v4.11.0](https://img.shields.io/badge/version-v4.11.0-blue.svg)](VERSIONS.md)
[![Version: v4.12.0](https://img.shields.io/badge/version-v4.12.0-blue.svg)](VERSIONS.md)
A new agent can follow the steps in this guide sequentially to establish a stable and reliable initial environment.
+1 -1
View File
@@ -1,6 +1,6 @@
# Messaging System Technical Analysis & Architecture Report
Applies to: MAM v4.11.0
Applies to: MAM v4.12.0
This report provides a comprehensive, deep-dive analysis of the messaging system implemented in the `multi-agent-mux-delegate-job` skill. It covers the MQTT broker architecture, event protocols, job lifecycles, codebase internals, cross-system integration, and a list of known limitations along with production recommendations.
+4 -3
View File
@@ -2,7 +2,7 @@
[herdr](https://herdr.dev)와 MQTT 위에 만든 **다중 에이전트 오케스트레이션·메시징 백플레인**입니다. 장시간 실행되는 코딩 에이전트(`claude`, `agy`, `hermes`, `grok`, `opencode`, `codex`)를 격리된 herdr pane 에서 돌리고, 작업을 추적되는 잡으로 위임하며, 모든 리뷰어가 통과시킬 때까지 planner / creator / reviewer 루프를 돌립니다.
[![Version: v4.11.0](https://img.shields.io/badge/version-v4.11.0-blue.svg)](VERSIONS.md)
[![Version: v4.12.0](https://img.shields.io/badge/version-v4.12.0-blue.svg)](VERSIONS.md)
---
@@ -38,7 +38,7 @@ bash deploy/install.sh
- `MQTT_BROKER=localhost` 로 된 초기 `.mam.env` 를 씁니다. 이미 있는 `.mam.env` 는 그대로 둡니다. 예전 `.env` 만 있으면 그것을 `.mam.env` 로 복사하고 원본은 남깁니다.
- `.venv`, `.mam/`, `.mam.env` 를 `.gitignore` 에 추가합니다.
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트와 제거는 `deploy/update.sh`, `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트는 `multi-agent-mux-update` 스킬, 제거는 `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
---
@@ -56,6 +56,7 @@ bash deploy/install.sh
| **`multi-agent-mux-delegate-job`** | 세션(`herdr:<name>`)이나 다른 호스트의 러너(`remote:<name>`)에 잡을 위임하고 결과를 기다립니다. [위임 계약](#-위임-계약)을 보십시오. |
| **`multi-agent-mux-loop`** | 계획·구현·리뷰: 선택적인 planner 와 creator, 그다음 선택한 리뷰어(`--reviewer A,B`, 또는 `--all-reviewer` 로 creator 를 뺀 실행 중인 리뷰어 전원)가 모두 `PASS` 를 낼 때까지(또는 `--max-loop` 까지). 둘 다 없으면 creator 가 스스로 리뷰합니다. 루프 밖에서 끝난 작업은 `--review-only` 로 리뷰합니다. 통과한 리포트는 `.agents/reports/<session>/` 으로 옮겨 보관합니다. |
| **`multi-agent-mux-orc-onboard`** | 오케스트레이터 자신의 세션 ID 를 등록해, 세션 탐색이 그것을 하위 에이전트로 착각하지 않게 합니다. |
| **`multi-agent-mux-update`** | 설치된 작업 공간을 제자리에서 업데이트합니다. 실행 중인 세션·열린 잡·루프가 있으면 거부하고, 모니터를 멈췄다가 원래대로 되돌리며, 설치가 실패하면 되돌리고, 새 코드로 레지스트리를 변환합니다. |
---
@@ -195,7 +196,7 @@ sequenceDiagram
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
│ └── multi-agent-mux-*/ # 스킬마다 디렉터리 하나 (SKILL.md + scripts/)
├── .mam/ # 런타임 상태 (git 제외): 세션, 잡, 감사 로그, shim
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
├── docs/ # contracts/, design/, 에이전트 통합·브리지 가이드
├── nats-docker/ # 사설 NATS 브로커 배포 (서브모듈)
├── tests/ # pytest 스위트
+4 -3
View File
@@ -2,7 +2,7 @@
A **Multi-Agent Orchestration & Messaging Backplane** built on [herdr](https://herdr.dev) and MQTT. It runs long-lived coding agents (`claude`, `agy`, `hermes`, `grok`, `opencode`, `codex`) in isolated herdr panes, delegates work to them as tracked jobs, and drives planner / creator / reviewer loops until every reviewer passes the change.
[![Version: v4.11.0](https://img.shields.io/badge/version-v4.11.0-blue.svg)](VERSIONS.md)
[![Version: v4.12.0](https://img.shields.io/badge/version-v4.12.0-blue.svg)](VERSIONS.md)
---
@@ -38,7 +38,7 @@ bash deploy/install.sh
- writes a starter `.mam.env` with `MQTT_BROKER=localhost`. An existing `.mam.env` is kept. If only a legacy `.env` exists, it is copied to `.mam.env` and the original stays;
- adds `.venv`, `.mam/` and `.mam.env` to `.gitignore`.
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). `deploy/update.sh` and `deploy/remove.sh` update and uninstall. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). The `multi-agent-mux-update` skill updates an installed workspace, and `deploy/remove.sh` uninstalls. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
---
@@ -56,6 +56,7 @@ Every skill lives under `.agents/skills/` with a `SKILL.md` that documents its o
| **`multi-agent-mux-delegate-job`** | Delegates a job to a session (`herdr:<name>`) or to a runner on another host (`remote:<name>`) and waits for its result. See [Delegation contract](#-delegation-contract). |
| **`multi-agent-mux-loop`** | Plan, implement, review: an optional planner and a creator, then the selected reviewers (`--reviewer A,B`, or `--all-reviewer` for every running reviewer except the creator) until all of them `PASS` or `--max-loop` is reached. With neither option the creator reviews its own work. `--review-only` reviews work finished outside the loop. Passing reports are promoted to `.agents/reports/<session>/`. |
| **`multi-agent-mux-orc-onboard`** | Registers the orchestrator's own session id, so session discovery never mistakes it for a sub-agent. |
| **`multi-agent-mux-update`** | Updates an installed workspace in place: refuses while sessions, open jobs or a loop are live; stops and restores the monitor; rolls back on a failed install; converts the registry with the new code. |
---
@@ -195,7 +196,7 @@ sequenceDiagram
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
│ └── multi-agent-mux-*/ # One directory per skill (SKILL.md + scripts/)
├── .mam/ # Runtime state (git-ignored): sessions, jobs, audit logs, shim
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
├── docs/ # contracts/, design/, agent integration and bridge guides
├── nats-docker/ # Private NATS broker deployment (submodule)
├── tests/ # pytest suite
+51 -12
View File
@@ -6,8 +6,8 @@
## 📌 현재 버전 개요 (Current Release)
- **프레임워크 버전**: `v4.11.0`
- **최신 릴리스 일시**: 2026-10-27 (KST)
- **프레임워크 버전**: `v4.12.0`
- **최신 릴리스 일시**: 2026-10-28 (KST)
- **기준 브랜치**: `main`
- **핵심 아키텍처**:
- **Terminal 상태 단방향 불변성 (`engine.py`, Issue #6 Phase 1)**: `completed`, `error`, `cancelled` 터미널 상태 진입 후 후속 이벤트나 업데이트에 의한 덮어쓰기를 엄격히 차단(Terminal Immutability). `registry.py close` 서브커맨드 및 fail-closed 조기 종료 도입.
@@ -18,29 +18,68 @@
- **워크트리 운영 모델 2단계 (Issue #21 항목 4·5·7)**: 워크트리 준비·해체 헬퍼(변경 전 판정, untracked 포함, 검증된 단일 `--force`), 리뷰 identity(라운드별 스냅샷·stale 재리뷰·승격 직전 재확인·`identity.json`), 수용 사례 A1–A8.
- **잡 예산·라벨·멈춤 알림 (Issue #33·#30)**: 위임 잡 예산은 `--job-timeout`/`MAM_JOB_TIMEOUT_S`, 벽시계 초과는 `wall_clock_timeout`(무응답 `deadline_timeout` 과 구분), `lib_py/stall_watch.py` 가 `blocked`/`idle` 을 `progress stalled:*` 로 알리기만 한다(종결 근거는 #18 의 프로토콜 이벤트뿐).
- **설정 파일 로더 하나 (Issue #26)**: `.mam.env` 는 `lib_py/delegation/envfile.py` 한 곳이 읽는다(셸 해석 없음, 환경변수 우선, 읽을 수 없는 파일은 오류). bash 산술에 들어가는 정수 설정값은 `mam_int` 로 10진수.
- **4-Way 버전 락스텝**: `lib.sh`(`MAM_VERSION`) ↔ `lib_py/loop/eager.py`(`FRAMEWORK_VERSION`) ↔ `VERSIONS.md` ↔ 8개 `SKILL.md`.
- **4-Way 버전 락스텝**: `lib.sh`(`MAM_VERSION`) ↔ `lib_py/loop/eager.py`(`FRAMEWORK_VERSION`) ↔ `VERSIONS.md` ↔ 9개 `SKILL.md`.
---
## 🧭 스킬 패키지 버전 매트릭스 (Skills Version Matrix)
모든 8개 스킬은 YAML frontmatter 메타데이터(`author`, `version`, `platforms`, `environments`) 표준화를 통해 `v4.11.0`로 동기화되어 배포됩니다.
모든 9개 스킬은 YAML frontmatter 메타데이터(`author`, `version`, `platforms`, `environments`) 표준화를 통해 `v4.12.0`로 동기화되어 배포됩니다.
| 스킬명 | 버전 | 역할 및 주요 책임 | 상태 |
| :--- | :---: | :--- | :---: |
| **`multi-agent-mux-create`** | `4.11.0` | 에이전트 세션 신규 생성 및 Herdr 컨테이너 격리 스폰 | ✅ 배포 |
| **`multi-agent-mux-stop`** | `4.11.0` | 대화 UUID 원자적 캡처 및 세션 안전 종료 (Graceful Stop) | ✅ 배포 |
| **`multi-agent-mux-resume`** | `4.11.0` | 온디스크 대화 컨텍스트 기반 Tier-1 초고속 세션 복원 | ✅ 배포 |
| **`multi-agent-mux-status`** | `4.11.0` | 실시간 Herdr 세션 및 레지스트리 드리프트 스냅샷 조회 | ✅ 배포 |
| **`multi-agent-mux-monitor`** | `4.11.0` | YAML ↔ 런타임 상태 간 자율 조정자 (Reconciler Loop) | ✅ 배포 |
| **`multi-agent-mux-delegate-job`** | `4.11.0` | MQTT 이벤트 채널 기반 비동기 단위 작업 위임 | ✅ 배포 |
| **`multi-agent-mux-loop`** | `4.11.0` | Planner-Creator-Reviewer 3자 자율 계획·실행·피어리뷰 루프 | ✅ 배포 |
| **`multi-agent-mux-orc-onboard`** | `4.11.0` | 오케스트레이터 UUID 격리 등록 및 서브 세션 오염 방지 | ✅ 배포 |
| **`multi-agent-mux-create`** | `4.12.0` | 에이전트 세션 신규 생성 및 Herdr 컨테이너 격리 스폰 | ✅ 배포 |
| **`multi-agent-mux-stop`** | `4.12.0` | 대화 UUID 원자적 캡처 및 세션 안전 종료 (Graceful Stop) | ✅ 배포 |
| **`multi-agent-mux-resume`** | `4.12.0` | 온디스크 대화 컨텍스트 기반 Tier-1 초고속 세션 복원 | ✅ 배포 |
| **`multi-agent-mux-status`** | `4.12.0` | 실시간 Herdr 세션 및 레지스트리 드리프트 스냅샷 조회 | ✅ 배포 |
| **`multi-agent-mux-monitor`** | `4.12.0` | YAML ↔ 런타임 상태 간 자율 조정자 (Reconciler Loop) | ✅ 배포 |
| **`multi-agent-mux-delegate-job`** | `4.12.0` | MQTT 이벤트 채널 기반 비동기 단위 작업 위임 | ✅ 배포 |
| **`multi-agent-mux-loop`** | `4.12.0` | Planner-Creator-Reviewer 3자 자율 계획·실행·피어리뷰 루프 | ✅ 배포 |
| **`multi-agent-mux-orc-onboard`** | `4.12.0` | 오케스트레이터 UUID 격리 등록 및 서브 세션 오염 방지 | ✅ 배포 |
| **`multi-agent-mux-update`** | `4.12.0` | 설치된 작업 공간의 제자리 업데이트·되돌리기와 업데이트 중 입장 차단 | ✅ 배포 |
---
## 📋 버전별 상세 변경 내역 (Changelog)
### 🚀 `v4.12.0` — update 스킬로 제자리 업데이트하고, 업데이트 중에는 세션·잡·루프의 시작을 막는다 (Issue #72) (2026-10-28)
> **마이너 (MINOR Release)**: 설치된 작업 공간의 업데이트를 `.mam_deploy/update.sh` 대신 새 `multi-agent-mux-update` 스킬이 한다. 업데이트는 실행 중인 세션·열린 잡·루프가 있으면 거부하고, 모니터를 멈췄다가 다시 띄우며, 설치가 실패하면 되돌린다. 업데이트 중에는 create·resume·run_loop·delegate submit·stop·orc-onboard 가 거부한다. resume 이 모니터 데몬을 띄운다. 공개 진입점(`.mam_deploy/update.sh`)이 없어지고 기존 명령의 동작(업데이트 중 거부)이 늘어 MINOR 로 올린다. PR #75.
#### ⚠️ 호환성
- `.mam_deploy/update.sh` 와 `deploy/update.sh` 가 없어졌다. 업데이트는 `bash .agents/skills/multi-agent-mux-update/scripts/update.sh --yes` 로 한다. 설치기(`deploy/install.sh`)는 남아 있는 옛 `update.sh` 를 지운다.
- 업데이트 중(`.mam/update-in-progress` 표식이 live/unknown)에는 create·resume·run_loop·delegate submit·stop·orc-onboard 가 거부한다. 이 명령들은 `.mam/update-admission.lock` 을 공유로 잡고, 기다리는 시간은 `MAM_UPDATE_ADMISSION_WAIT_S`(기본 120 s)다. `.mam` 이 없는 트리(개발 체크아웃)에서는 잠금을 만들지 않는다.
- resume 이 끝나면 모니터 데몬(`reconcile.sh --subscribe`)을 띄운다. 이미 떠 있으면 새로 띄우지 않고, `MAM_NO_SUBSCRIBER_DAEMON=1` 이면 띄우지 않는다.
- create·delegate-job·stop 의 attach 안내는 행의 `attach_command`(`herdr --session <server> agent attach <herdr 이름>`)를 보여 준다.
- **첫 전환 (v4.11.0 이하 설치본, 옛 `update.sh` 로 한 번)**:
1. 루프가 끝나기를 기다리고, 열린 잡을 끝내거나 닫는다.
2. 세션을 모두 stop 한다.
3. 모니터(`reconcile.sh --subscribe`)를 멈춘다.
4. 옛 `.mam_deploy/update.sh` 를 실행한다.
5. 바로 update 스킬을 한 번 더 실행한다(레지스트리 변환).
- 4번의 옛 `update.sh` 는 `.mam` 을 지운 뒤 고정 목록만 되돌리므로 `herdr_workspace_id`, `diagnostics/`, 모니터 로그, `shim/`(lib.sh 가 다시 만듦), 있다면 `briefs/`·`resumelogs/`·`stoplogs/` 를 잃는다.
- 첫 전환 뒤에는 모니터가 없다. resume 하면 뜬다.
#### 🌟 주요 개선 사항
1. **update 스킬 (`multi-agent-mux-update`: `scripts/update.sh`, `install_phase.sh`, `post_update.sh`, `lib_py/update.py`)**:
- 1단계(설치된 옛 코드): 배타 입장 잠금 아래에서 표식을 걸고, 아무것도 바꾸기 전에 거부할 것을 확인한다(running 행, cwd 가 이 작업 공간 안인 살아 있는 herdr 에이전트, 읽을 수 없는 것을 포함한 열린 잡, live 루프, 스킬이 띄우지 않은 모니터가 쥔 `.mam/monitor.lock`, 다른 업데이트). 호출한 에이전트(서버+pane)는 세지 않는다.
- 이 작업 공간의 모니터만(정확한 명령줄로) 멈추고(TERM, 끝나지 않으면 KILL; `MAM_UPDATE_MONITOR_STOP_S`), `.agents/`·`.mam_deploy/`·루트 파일·`.venv`·설치기가 관리하는 `.mam` 파일·레지스트리를 `.mam/update-backup/` 에 백업한 뒤, 설치 단계의 복사본을 트리 밖에서 exec 한다.
- 설치(bash 3.2, lib.sh 를 부르지 않음): `MAM_INSTALLER_URL` 의 설치기를 제자리로 실행한다. 실패하면 백업에서 되돌리고 되돌린 트리로 모니터를 다시 띄운다.
- 2단계(새 코드): `lib_py.registry_format` 으로 레지스트리를 변환하고, 원래 떠 있던 모니터만 다시 띄운다. 변환 실패는 변환기의 코드로 끝난다.
- 남은 백업은 상태(`backing-up` → `installing` → `installed`)에 따라 지우거나 되돌린 뒤 진행한다. 되돌리기가 실패하면 백업을 두고 멈춘다.
2. **입장 잠금 (`lib.sh` `mam_update_admit`, `lib_py.pane_lock acquire --shared --wait-env`)**: 위 여섯 명령이 기록을 쓸 때까지 공유 잠금을 쥔다. 오래 사는 자식은 그 fd 를 물려받지 않는다. delegate submit 과 create·resume 은 대상 pane 이 속한 설치 작업 공간의 잠금도 잡는다.
3. **제자리 설치 (`deploy/install.sh`)**: `install_state` 를 키 단위로 덮어쓴다. 새 버전에 없는 프레임워크 파일은 사용자가 고치지 않았으면 지우고, 고쳤으면 남기고 목록을 출력한다.
4. **resume 의 모니터 (`resume_session.sh`)**: 행을 쓴 뒤 제어 루트를 workdir 로 `start_watchdog` 를 부른다(worktree 세션 포함). 실패는 경고만 한다. legacy `start_watchdog`(`MAM_DELEGATION_PY=0`)도 모니터를 자기 프로세스 그룹에서 띄운다.
5. **attach 안내**: create 의 마지막 안내, delegate-job 의 전달 안내, stop 의 TUI 멈춤 안내, `deploy/INSTALL.md`, `install_mam.sh`.
6. **버전 락스텝**: update 스킬이 매트릭스와 `test_version_consistency.py` 의 스킬 목록에 들어가 9개 `SKILL.md` 를 맞춘다.
#### ✅ 품질 검증
- **회귀**: `tests/test_issue72_{update,admission,attach_hints,installer,resume_monitor}.py`. base(v4.11.0)에서 78개 중 77개가 실패하고 1개는 건너뛴다(첫 전환 테스트, base 아카이브에 git 기록이 없어서).
- **리뷰 (`--all-reviewer`, 작성자 PR 의 리뷰 전용 루프)**: 1회차 4명 모두 NOT PASS(모니터를 정확한 명령줄로 고르기, pane 작업 공간의 입장, 레지스트리 스냅샷을 행 단위로 비교, macOS 의 `-shm`). agy 의 지적은 creator 의 반론으로 철회됐다. 2·3회차 3/4 PASS, codex 가 legacy `start_watchdog` 의 프로세스 그룹과 테스트의 패턴 kill 정리를 지적했다. 수정 루프(@ec7f5c4)에서 **4/4 PASS**. 리포트는 `.agents/reports/<session>/` 에 있다.
- **전체 스위트 (macOS, Python 3.9.6, @ec7f5c4)**: 2268 passed, 56 skipped, 실패 0 (100분). 실제 `~/.claude`·`/private/tmp/claude-<uid>`·herdr 서버 pane 목록·opencode 의 data·state·config·본 체크아웃의 `.mam` 에 이 실행이 남긴 것은 없었고, 본 체크아웃의 모니터는 그대로 떠 있었다.
- **작성자 확인 (Linux, @253a0aa)**: 2291 passed, 20 skipped, 실패 0.
- **알려진 비차단**: stale 표식 NOTE 의 출력 순서(다른 업데이트가 끼어들면 메시지가 둘). `MAM_STATE_JSON` 이 깨졌을 때 delegate submit 이 대상 해석을 건너뛴다(등록은 뒤의 레지스트리 접근에서 실패). 행 없는 대상에 submit 할 때마다 herdr 를 한 번 더 조회한다. 입장 경합을 실제 create 로 재현하는 테스트, 기존 `.venv` 내용의 복원 테스트, 되돌리기 뒤 DB 전체 비교 테스트는 없다.
### 🚀 `v4.11.0` — attach 명령은 herdr 에 등록된 이름을 쓰고, 세션 시작·종료는 스킬로만 한다 (Issue #71) (2026-10-27)
> **마이너 (MINOR Release)**: 레지스트리의 `attach_command` 가 원래 세션 이름으로 herdr 를 불러, 이름이 herdr 의 규칙(소문자 `[a-z0-9_-]`, 32자 이하)을 벗어나면 붙지 못했다. 이제 attach 명령은 한 함수가 만들고, 레지스트리 행에서 `kill_command`·`start_command` 를 지웠으며, 레지스트리에 형식 버전(`registry_format: 2`)을 더했다. `status --json` 의 키가 빠지고 stop 의 동작(행 없는 세션 닫기)이 늘어 MINOR 로 올린다. PR #73.
+1 -1
View File
@@ -60,7 +60,7 @@ $ bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh \
### 2) 세션 접속 (Attach)
백그라운드에서 구동된 에이전트 TUI 화면에 들어갑니다.
```bash
$ herdr session attach my-project-dev-claude
$ herdr --session multi-agent-mux agent attach my-project-dev-claude
```
* **화면 탈출**: 대화 중 세션을 유지한 채 터미널로 돌아오려면 `Ctrl + B`를 누른 뒤 `D` 키를 차례로 입력합니다.
+3 -3
View File
@@ -9,7 +9,7 @@ This directory contains packaging templates and installation scripts to deploy t
* **`install.sh`**: A self-contained, idempotent remote shell installer (via curl) that checks system requirements (`herdr`, `python3`), sets up a local python virtual environment (`.venv`), and performs 3-way refresh with key-level registry merge (Rev.2).
* **`install_mam.sh`**: A local-clone installer that copies rules/skills (`.agents/`), `AGENTS.md`, manifests, and asset hashes into target projects.
* **`lib_ownership.sh`**: Single source of truth for framework-owned files and key-level registry files.
* **`update.sh`**: In-place updater script installed into target `.mam_deploy/update.sh`.
* **Updates**: installed workspaces update with the `multi-agent-mux-update` skill (`bash .agents/skills/multi-agent-mux-update/scripts/update.sh`).
* **`remove.sh`**: Clean uninstaller script installed into target `.mam_deploy/remove.sh`.
* **`generate-env.sh`**: Environment configuration bootstrap helper.
* **`INSTALL.md`**: Detailed installation and quick-start user manual.
@@ -40,7 +40,7 @@ bash deploy/install_mam.sh --target /path/to/your/project
Refer to **`INSTALL.md`** inside this directory for the full instructions and workflows.
> [!NOTE]
> `install_mam.sh` automatically deploys `.mam_deploy/update.sh` and `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
> `install_mam.sh` automatically deploys `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
### 3. Custom Fork / Private Mirror Installations
If you run a private mirror or fork, you can override the source URLs during installation using environment variables:
@@ -53,7 +53,7 @@ curl -fsSL https://my-mirror.example.com/.../install.sh \
bash
# Updating an existing workspace against a mirror
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash deploy/update.sh
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash .agents/skills/multi-agent-mux-update/scripts/update.sh
```
### 4. Registering as a Workspace Plugin
+1 -1
View File
@@ -41,7 +41,7 @@ jobs:
shellcheck deploy/install.sh
shellcheck deploy/install_mam.sh
shellcheck deploy/generate-env.sh
shellcheck deploy/update.sh
shellcheck .agents/skills/multi-agent-mux-update/scripts/*.sh
shellcheck deploy/remove.sh
echo "✅ ShellCheck completed successfully."
+68 -18
View File
@@ -125,7 +125,9 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
FETCH_METHOD="archive"
if [ -d "$REPO_URL" ]; then
echo "🌐 Copying local working tree into a staging area..."
cp -R "$REPO_URL/." "$STAGE_DIR/"
# Everything but .mam/: runtime state is never staged, and it may hold the
# update skill's rollback copy (.mam/update-backup/, Issue #72).
find "$REPO_URL/." -mindepth 1 -maxdepth 1 ! -name .mam -exec cp -R {} "$STAGE_DIR/" \;
FETCH_METHOD="local"
elif command -v git &>/dev/null; then
echo "🌐 Cloning repository (shallow) into a staging area..."
@@ -157,9 +159,16 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
MANIFEST_FILE=".mam/install_manifest.txt"
touch "$MANIFEST_FILE"
# Migrate legacy root layout for remove.sh and update.sh into .mam_deploy/ if owned
# Drop every line naming this path from the manifest
manifest_drop() {
grep -Fvx -- "$1" "$MANIFEST_FILE" > "$MANIFEST_FILE.tmp" || true
mv -f "$MANIFEST_FILE.tmp" "$MANIFEST_FILE"
}
# Migrate legacy root layout for remove.sh into .mam_deploy/ if owned
mkdir -p .mam_deploy
for legacy_script in remove.sh update.sh; do
# shellcheck disable=SC2043 # one legacy script left (update.sh went with Issue #72)
for legacy_script in remove.sh; do
if [ -f "$legacy_script" ] && grep -Fqx "$legacy_script" "$MANIFEST_FILE" 2>/dev/null; then
mv -f "$legacy_script" ".mam_deploy/$legacy_script"
python3 -c '
@@ -179,6 +188,13 @@ with open(path, "w") as f:
fi
done
# update.sh is no longer shipped (Issue #72; the multi-agent-mux-update skill
# updates). A legacy root update.sh goes only if the manifest says we put it there.
if [ -f update.sh ] && grep -Fqx update.sh "$MANIFEST_FILE" 2>/dev/null; then
rm -f update.sh
manifest_drop update.sh
fi
# Safe refresh & fingerprint checking logic
TS=$(date -u +%Y%m%dT%H%M%SZ)
FRAMEWORK_LEDGER=".mam/.framework_ledger.tmp"
@@ -374,7 +390,7 @@ MERGEREG
fi
done
# Install remove.sh and update.sh into .mam_deploy/
# Install remove.sh into .mam_deploy/
mkdir -p .mam_deploy
if [ -f "$STAGE_DIR/deploy/remove.sh" ]; then
cp "$STAGE_DIR/deploy/remove.sh" .mam_deploy/remove.sh
@@ -384,13 +400,9 @@ MERGEREG
fi
fi
if [ -f "$STAGE_DIR/deploy/update.sh" ]; then
cp "$STAGE_DIR/deploy/update.sh" .mam_deploy/update.sh
chmod 0755 .mam_deploy/update.sh
if ! grep -Fqx ".mam_deploy/update.sh" "$MANIFEST_FILE" 2>/dev/null; then
echo ".mam_deploy/update.sh" >> "$MANIFEST_FILE"
fi
fi
# An older install's updater is not shipped any more (Issue #72)
rm -f .mam_deploy/update.sh
manifest_drop .mam_deploy/update.sh
if [ -f "$STAGE_DIR/.mam.env.example" ] && [ ! -e ".mam.env.example" ]; then
cp "$STAGE_DIR/.mam.env.example" .
@@ -407,6 +419,34 @@ MERGEREG
fi
fi
# Framework-owned files the previous install recorded (manifest and
# asset_hashes.txt) that this version no longer ships (Issue #72). Unmodified
# ones are deleted. Modified ones are kept and become the user's file: they
# leave the manifest, so remove.sh leaves them alone, and asset_hashes.txt
# (rebuilt below from the shipped files only).
if [ -f .mam/asset_hashes.txt ]; then
SHIPPED_LIST=".mam/.shipped_paths.tmp"
cut -f2 "$FRAMEWORK_LEDGER" > "$SHIPPED_LIST"
while read -r _old_sha _old_path; do
[ -n "${_old_path:-}" ] || continue
is_framework_owned "$_old_path" || continue
grep -Fqx -- "$_old_path" "$MANIFEST_FILE" || continue
if grep -Fqx -- "$_old_path" "$SHIPPED_LIST"; then continue; fi
if [ -f "$_old_path" ] && [ "$(python3 -c 'import hashlib, sys; print(hashlib.sha256(open(sys.argv[1], "rb").read()).hexdigest())' "$_old_path")" != "$_old_sha" ]; then
echo "ℹ️ Local modification detected (no longer shipped, kept as yours): $_old_path" >&2
else
rm -f "$_old_path"
_old_dir="$(dirname "$_old_path")"
while [ "$_old_dir" != ".agents" ] && rmdir "$_old_dir" 2>/dev/null; do
_old_dir="$(dirname "$_old_dir")"
done
fi
rm -f ".mam/base/$_old_path"
manifest_drop "$_old_path"
done < .mam/asset_hashes.txt
rm -f "$SHIPPED_LIST"
fi
# Re-build asset_hashes.txt from the UPSTREAM copy of every framework-owned file
python3 - .mam/asset_hashes.txt "$FRAMEWORK_LEDGER" <<'HASHDB'
import os, hashlib, sys
@@ -590,15 +630,25 @@ with open(gi_path, "w") as f:
PY
if [ "${MAM_SKIP_GITIGNORE:-0}" -eq 0 ]; then
# Set the key: overwrite it in place, one line per key, other keys kept
python3 -c '
import os
os.makedirs(".mam", exist_ok=True)
lines = []
if os.path.exists(".mam/install_state"):
with open(".mam/install_state", "a") as f:
f.write("gitignore_created=1\n")
else:
os.makedirs(".mam", exist_ok=True)
with open(".mam/install_state", "w") as f:
f.write("gitignore_created=1\n")
with open(".mam/install_state") as f:
lines = f.read().splitlines()
out, done = [], False
for line in lines:
if line.startswith("gitignore_created="):
if done:
continue
line, done = "gitignore_created=1", True
out.append(line)
if not done:
out.append("gitignore_created=1")
with open(".mam/install_state", "w") as f:
f.write("\n".join(out) + "\n")
'
fi
@@ -617,6 +667,6 @@ cat <<EOF
$ bash .agents/skills/multi-agent-mux-status/scripts/status.sh
3. Update installed skills in the future:
$ bash .mam_deploy/update.sh
$ bash .agents/skills/multi-agent-mux-update/scripts/update.sh
--------------------------------------------------------------------------------
EOF
+3 -7
View File
@@ -132,16 +132,12 @@ if [ -f "$SRC_DIR/.mam.env.example" ]; then
log_ok "Copied .mam.env.example configuration template"
fi
# Deploy remove.sh and update.sh into .mam_deploy/ (R-3)
# Deploy remove.sh into .mam_deploy/ (R-3)
mkdir -p "$TARGET_DIR/.mam_deploy"
if [ -f "$SRC_DIR/deploy/remove.sh" ]; then
cp "$SRC_DIR/deploy/remove.sh" "$TARGET_DIR/.mam_deploy/remove.sh"
chmod 0755 "$TARGET_DIR/.mam_deploy/remove.sh"
fi
if [ -f "$SRC_DIR/deploy/update.sh" ]; then
cp "$SRC_DIR/deploy/update.sh" "$TARGET_DIR/.mam_deploy/update.sh"
chmod 0755 "$TARGET_DIR/.mam_deploy/update.sh"
fi
if [ -f "$SRC_DIR/deploy/generate-env.sh" ]; then
mkdir -p "$TARGET_DIR/scripts"
@@ -161,7 +157,7 @@ mkdir -p "$TARGET_DIR/.mam"
MANIFEST_FILE="$TARGET_DIR/.mam/install_manifest.txt"
: > "$MANIFEST_FILE"
( cd "$TARGET_DIR" && find .agents -type f -print ) >> "$MANIFEST_FILE"
for extra in ".mam.env.example" ".mam_deploy/remove.sh" ".mam_deploy/update.sh" \
for extra in ".mam.env.example" ".mam_deploy/remove.sh" \
"scripts/generate-env.sh"; do
if [ -e "$TARGET_DIR/$extra" ]; then
echo "$extra" >> "$MANIFEST_FILE"
@@ -327,7 +323,7 @@ cat <<EOF
--herdr-server multi-agent-mux
2. Attach to the running session:
$ HERDR_SERVER_NAME=multi-agent-mux herdr agent attach <session_name>
$ herdr --session multi-agent-mux agent attach <herdr name>
3. Gracefully stop the session:
$ bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh \\
+1
View File
@@ -92,6 +92,7 @@ else
".agents/skills/multi-agent-mux-resume"
".agents/skills/multi-agent-mux-status"
".agents/skills/multi-agent-mux-stop"
".agents/skills/multi-agent-mux-update"
".agents/hooks"
".agents/hooks.json"
".agents/MULTI_AGENT_RULES.md"
-269
View File
@@ -1,269 +0,0 @@
#!/usr/bin/env bash
# ==============================================================================
# update.sh — Multi-Agent Mux (MAM) Orchestration Updater
# ==============================================================================
# Safely updates MAM skills, virtual environment, and docs to the latest version.
# Preserves user configuration (.env) and local metadata/jobs database (.mam).
# ==============================================================================
set -euo pipefail
TARGET_DIR=""
FORCE=0
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Parse arguments
while [[ $# -gt 0 ]]; do
case "$1" in
-y|--yes|--force)
FORCE=1
shift
;;
*)
TARGET_DIR="$1"
shift
;;
esac
done
if [ -z "$TARGET_DIR" ]; then
if [ "$(basename "$SCRIPT_DIR")" = ".mam_deploy" ]; then
TARGET_DIR="$(dirname "$SCRIPT_DIR")"
else
TARGET_DIR="$(pwd)"
fi
else
if [ "$(basename "$TARGET_DIR")" = ".mam_deploy" ]; then
TARGET_DIR="$(dirname "$TARGET_DIR")"
fi
if [ ! -d "$TARGET_DIR" ]; then
echo "❌ Error: Target directory '$TARGET_DIR' does not exist." >&2
exit 1
fi
TARGET_DIR="$(cd "$TARGET_DIR" && pwd)"
fi
echo "===================================================================="
echo "⚡ Starting Multi-Agent Mux (MAM) Update"
echo "📂 Target Workspace: $TARGET_DIR"
echo "===================================================================="
cd "$TARGET_DIR"
# 1. Verification of existing install (B-2: dual resolution)
REMOVER=""
for cand in ".mam_deploy/remove.sh" "remove.sh"; do
if [ -f "$cand" ]; then
REMOVER="$cand"
break
fi
done
if [ -z "$REMOVER" ]; then
echo "❌ Error: No MAM installation (remove.sh) found in '$TARGET_DIR'." >&2
echo " Please run install.sh first to set up the workspace." >&2
exit 1
fi
# Request confirmation if not forced
if [ $FORCE -eq 0 ]; then
echo "⚠️ WARNING: This will update MAM orchestration skills, virtual environment, "
echo " and docs to the latest version."
echo " (Your configuration, job history, and custom skills will be preserved)."
if [ ! -t 0 ]; then
echo "❌ Error: Non-interactive terminal detected. Please run with -y/--yes/--force." >&2
exit 1
fi
if ! read -p "❓ Proceed with update? [y/N]: " -r response; then
response="n"
fi
if [[ ! "$response" =~ ^[yY](es)?$ ]]; then
echo "❌ Update cancelled by user."
exit 0
fi
fi
# 1.5 Pre-capture legacy ownership before remove.sh deletes the manifest (M-4)
MAM_LEGACY_ENV_OWNED=0
if [ -f ".mam/install_manifest.txt" ] && grep -Fqx ".env" ".mam/install_manifest.txt" 2>/dev/null; then
MAM_LEGACY_ENV_OWNED=1
fi
export MAM_LEGACY_ENV_OWNED
# 2. Stage backups of user configurations and metadata to prevent deletion
echo "💾 Backing up configuration and database..."
HAS_ENV=0
ENV_BACKUP_SRC=""
ENV_BACKUP_TMP=""
if [ -f ".mam.env" ]; then
HAS_ENV=1
ENV_BACKUP_SRC=".mam.env"
ENV_BACKUP_TMP=".mam.env.update-tmp"
mv ".mam.env" "$ENV_BACKUP_TMP"
elif [ -f ".env" ]; then
HAS_ENV=1
ENV_BACKUP_SRC=".env"
ENV_BACKUP_TMP=".env.update-tmp"
mv ".env" "$ENV_BACKUP_TMP"
fi
HAS_MAM=0
if [ -d ".mam" ]; then
HAS_MAM=1
# Copy database and jobs to temporary backup outside of .mam.
# We do NOT move the .mam folder away so that remove.sh can still read .mam/install_manifest.txt!
mkdir -p .mam.update-tmp
# Copy SQLite databases and session files
for db in .mam/agent-sessions.*; do
if [ -f "$db" ]; then
cp -f "$db" .mam.update-tmp/
fi
done
# Copy jobs history
if [ -d ".mam/jobs" ] && [ "$(ls -A .mam/jobs 2>/dev/null)" ]; then
mkdir -p .mam.update-tmp/jobs
cp -rf .mam/jobs/* .mam.update-tmp/jobs/
fi
# Copy delegate logs
if [ -d ".mam/delegate_job_logs" ] && [ "$(ls -A .mam/delegate_job_logs 2>/dev/null)" ]; then
mkdir -p .mam.update-tmp/delegate_job_logs
cp -rf .mam/delegate_job_logs/* .mam.update-tmp/delegate_job_logs/
fi
# Copy manifest so we have a backup
if [ -f ".mam/install_manifest.txt" ]; then
cp -f .mam/install_manifest.txt .mam.update-tmp/
fi
# C12: Copy MAM state files across update cycle
for st in install_state asset_hashes.txt version.txt; do
if [ -f ".mam/$st" ]; then
cp -f ".mam/$st" .mam.update-tmp/
fi
done
if [ -d ".mam/skill-backups" ]; then
mkdir -p .mam.update-tmp/skill-backups
cp -rf .mam/skill-backups/* .mam.update-tmp/skill-backups/ 2>/dev/null || true
fi
fi
# Define trap to restore backup files on failure
restore_on_failure() {
echo "❌ Update failed. Reverting configuration and database to previous state..."
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC" 2>/dev/null || true
fi
if [ $HAS_MAM -eq 1 ] && [ -d ".mam.update-tmp" ]; then
# Revert to old database/jobs backup by restoring .mam directory
rm -rf .mam 2>/dev/null || true
mkdir -p .mam
cp -f .mam.update-tmp/agent-sessions.* .mam/ 2>/dev/null || true
if [ -d ".mam.update-tmp/jobs" ]; then
cp -rf .mam.update-tmp/jobs .mam/ 2>/dev/null || true
fi
if [ -d ".mam.update-tmp/delegate_job_logs" ]; then
cp -rf .mam.update-tmp/delegate_job_logs .mam/ 2>/dev/null || true
fi
if [ -f ".mam.update-tmp/install_manifest.txt" ]; then
cp -f .mam.update-tmp/install_manifest.txt .mam/ 2>/dev/null || true
fi
for st in install_state asset_hashes.txt version.txt; do
if [ -f ".mam.update-tmp/$st" ]; then
cp -f ".mam.update-tmp/$st" .mam/ 2>/dev/null || true
fi
done
if [ -d ".mam.update-tmp/skill-backups" ]; then
mkdir -p .mam/skill-backups
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
fi
rm -rf .mam.update-tmp 2>/dev/null || true
fi
}
trap restore_on_failure EXIT
# 3. Perform uninstallation of existing files
echo "🗑️ Removing existing installation..."
# remove.sh will run in manifest mode because .mam/install_manifest.txt is still present.
# It will delete .agents/, documents, scripts, .venv, and .mam folder.
bash "$REMOVER" --force "$TARGET_DIR"
# 4. Fetch and run the latest installer from Gitea
echo "📥 Fetching and running the latest installer..."
# Note: MAM_*_URL environment variables are automatically inherited by the child installer bash process.
INSTALLER_URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
if command -v curl &>/dev/null; then
curl -fsSL "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
elif command -v wget &>/dev/null; then
wget -qO- "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
else
echo "❌ Error: Neither 'curl' nor 'wget' is available to fetch the installer." >&2
exit 1
fi
# Disable failure trap since installation succeeded
trap - EXIT
# 5. Restore backups of configuration and database
echo "🔄 Restoring configuration and database..."
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
if [ "$MAM_LEGACY_ENV_OWNED" = "1" ] && [ "$ENV_BACKUP_SRC" = ".env" ] && [ ! -f ".mam.env" ]; then
mv -f "$ENV_BACKUP_TMP" ".mam.env"
chmod 0600 ".mam.env" 2>/dev/null || true
else
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC"
fi
fi
if [ $HAS_MAM -eq 1 ]; then
if [ -d ".mam.update-tmp" ]; then
# The installer created a new .mam directory with a new manifest.
# We want to merge the old .mam database/jobs back while keeping the new manifest.
for db in .mam.update-tmp/agent-sessions.*; do
if [ -f "$db" ]; then
cp -f "$db" .mam/
fi
done
if [ -d ".mam.update-tmp/jobs" ] && [ "$(ls -A .mam.update-tmp/jobs 2>/dev/null)" ]; then
mkdir -p .mam/jobs
cp -rf .mam.update-tmp/jobs/* .mam/jobs/
fi
if [ -d ".mam.update-tmp/delegate_job_logs" ] && [ "$(ls -A .mam.update-tmp/delegate_job_logs 2>/dev/null)" ]; then
mkdir -p .mam/delegate_job_logs
cp -rf .mam.update-tmp/delegate_job_logs/* .mam/delegate_job_logs/
fi
# Issue #8: asset_hashes.txt and version.txt describe the tree the
# installer just wrote, so the pre-update copies must NOT come back here -
# restoring them made version.txt name the previous commit and remove.sh
# treat every updated file as locally modified (spurious backups). Only
# install_state carries history worth keeping: merge in any key the fresh
# install did not write (e.g. gitignore_created from the original install).
# The failure path (restore_on_failure) still puts all three back verbatim.
if [ -f ".mam.update-tmp/install_state" ]; then
touch .mam/install_state
# A fresh install_state without a final newline would glue the first
# merged key onto its last line.
if [ -s .mam/install_state ] && [ -n "$(tail -c1 .mam/install_state)" ]; then
printf '\n' >> .mam/install_state
fi
while IFS= read -r line || [ -n "$line" ]; do
case "$line" in
*=*) ;;
*) continue ;; # not a key=value line
esac
key="${line%%=*}"
[ -n "$key" ] || continue
# exact key match, no regex interpretation of the key
if ! awk -F= -v k="$key" '$1 == k { found = 1 } END { exit !found }' .mam/install_state; then
printf '%s\n' "$line" >> .mam/install_state
fi
done < .mam.update-tmp/install_state
fi
if [ -d ".mam.update-tmp/skill-backups" ]; then
mkdir -p .mam/skill-backups
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
fi
rm -rf ".mam.update-tmp"
fi
fi
echo "===================================================================="
echo "🎉 Update complete!"
echo "===================================================================="
+31 -17
View File
@@ -11,7 +11,7 @@
"kind": "example",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:99",
"multi-agent-mux-create/scripts/create_session.sh:356"
"multi-agent-mux-create/scripts/create_session.sh:360"
],
"mismatch": null,
"note": "HERDR_SESSION_NAME environment example"
@@ -67,7 +67,7 @@
"class": "c",
"kind": "example",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:514-541"
"multi-agent-mux-create/scripts/create_session.sh:518-545"
],
"mismatch": null,
"note": "Output format: the row create_session.sh writes"
@@ -122,7 +122,7 @@
"class": "c",
"kind": "invocation",
"evidence": [
"multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:1046-1057"
"multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:1051-1062"
],
"mismatch": null,
"note": "CLI reference; subcommands submit/status/list/verify/wait/logs match main()"
@@ -433,7 +433,7 @@
"class": "c",
"kind": "diagram",
"evidence": [
"multi-agent-mux-stop/scripts/stop_session.sh:245-259"
"multi-agent-mux-stop/scripts/stop_session.sh:250-264"
],
"mismatch": null,
"note": "state machine"
@@ -463,6 +463,20 @@
"evidence": [],
"mismatch": null,
"note": "post-stop checks; no script offers them as a command - stage 3"
},
{
"skill": "multi-agent-mux-update",
"start": 17,
"end": 19,
"lang": "bash",
"sha": "63f73ade58d3",
"class": "c",
"kind": "invocation",
"evidence": [
"multi-agent-mux-update/scripts/update.sh:3"
],
"mismatch": null,
"note": "entry point call (Issue #72)"
}
],
"removed": [
@@ -476,11 +490,11 @@
"evidence": [
"multi-agent-mux-resume/scripts/resume_session.sh:111-130",
"multi-agent-mux-resume/scripts/resume_session.sh:133-144",
"multi-agent-mux-resume/scripts/resume_session.sh:147-158",
"multi-agent-mux-resume/scripts/resume_session.sh:172-197",
"multi-agent-mux-resume/scripts/resume_session.sh:245",
"multi-agent-mux-resume/scripts/resume_session.sh:249",
"multi-agent-mux-resume/scripts/resume_session.sh:255-258",
"multi-agent-mux-resume/scripts/resume_session.sh:147-170",
"multi-agent-mux-resume/scripts/resume_session.sh:188-213",
"multi-agent-mux-resume/scripts/resume_session.sh:266",
"multi-agent-mux-resume/scripts/resume_session.sh:270",
"multi-agent-mux-resume/scripts/resume_session.sh:276-279",
"lib_py/agents/adapters/claude.py:109"
],
"mismatch": [
@@ -504,8 +518,8 @@
"class": "a",
"evidence": [
"multi-agent-mux-stop/scripts/stop_session.sh:101",
"multi-agent-mux-stop/scripts/stop_session.sh:123-193",
"multi-agent-mux-stop/scripts/stop_session.sh:245-249"
"multi-agent-mux-stop/scripts/stop_session.sh:128-198",
"multi-agent-mux-stop/scripts/stop_session.sh:250-254"
],
"mismatch": [
"T1: the doc offered --force-no-yaml, which no script accepts"
@@ -547,8 +561,8 @@
"multi-agent-mux-create/scripts/create_session.sh:166-168",
"multi-agent-mux-create/scripts/create_session.sh:207",
"multi-agent-mux-create/scripts/create_session.sh:259-274",
"multi-agent-mux-create/scripts/create_session.sh:361-398",
"multi-agent-mux-create/scripts/create_session.sh:410-412"
"multi-agent-mux-create/scripts/create_session.sh:365-402",
"multi-agent-mux-create/scripts/create_session.sh:414-416"
],
"mismatch": [
"C1: an existing session is exit 1 in the doc, exit 3 in the script",
@@ -568,7 +582,7 @@
"base_sha": "f71be0f26fb8",
"class": "a",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:486-604"
"multi-agent-mux-create/scripts/create_session.sh:490-608"
],
"mismatch": [],
"replacement": "bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh --workspace <path> --agent <agent> --role <role>",
@@ -601,7 +615,7 @@
"other"
],
"passthrough": {
"1": "multi-agent-mux-stop/scripts/stop_session.sh:384"
"1": "multi-agent-mux-stop/scripts/stop_session.sh:389"
}
},
"multi-agent-mux-create/scripts/create_session.sh": {
@@ -619,14 +633,14 @@
"other"
],
"passthrough": {
"4": "multi-agent-mux-create/scripts/create_session.sh:546"
"4": "multi-agent-mux-create/scripts/create_session.sh:550"
}
}
},
"prose_mismatches": [
"C4: create named a nonexistent `agent-sessions-yaml-edit` script; removed (create_session.sh writes the row)",
"C5: create's cmd_full table (claude = `claude`) disagreed with adapter spawn-spec; replaced by \"decided by spawn-spec\"",
"C6: create_session.sh header lists exit 4 \"YAML append failure\"; there is no literal `exit 4`, but the YAML write raises SystemExit(4) for a duplicate running row (create_session.sh:515) and it passes through, so 4 IS a real exit code (the plan assumed it was not)",
"C6: create_session.sh header lists exit 4 \"YAML append failure\"; there is no literal `exit 4`, but the YAML write raises SystemExit(4) for a duplicate running row (create_session.sh:516) and it passes through, so 4 IS a real exit code (the plan assumed it was not)",
"R6: resume's \"The script above re-passes them\" and \"as done in Workflow above\" pointed at the removed block; reworded (same for create's Verification note)",
"R7 (review 08e1f1b2): stop's table said exit 1 is always pre-stop; the registry update after the kill chain also raises SystemExit(1) (stop_session.sh:310) when the row disappeared"
],
+19 -7
View File
@@ -48,7 +48,7 @@ class TestDeployLayout(unittest.TestCase):
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".agents", "skills", "lib.sh")))
def test_td6_td7_td8_mam_deploy_layout_and_removal(self):
"""T-D6~D8: Verify remove.sh/update.sh installed in .mam_deploy and work properly."""
"""T-D6~D8: Verify remove.sh installed in .mam_deploy (update.sh not, Issue #72) and works properly."""
subprocess.run(
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
env=self.env,
@@ -59,7 +59,7 @@ class TestDeployLayout(unittest.TestCase):
remover = os.path.join(self.test_dir, ".mam_deploy", "remove.sh")
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
self.assertTrue(os.path.exists(remover))
self.assertTrue(os.path.exists(updater))
self.assertFalse(os.path.exists(updater))
self.assertTrue(os.access(remover, os.X_OK))
# T-D8: Execution from inside .mam_deploy directory targets parent workspace
@@ -129,17 +129,29 @@ class TestDeployLayout(unittest.TestCase):
self.assertTrue(os.path.exists(backups_dir))
def test_td27_td28_update_preserves_mam_state(self):
"""T-D27~D28: Verify update.sh preserves .mam state files across cycles (B-5 gate)."""
"""T-D27~D28: Verify the update skill preserves .mam state files across cycles (B-5 gate)."""
subprocess.run(
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
env=self.env,
check=True
)
# Run update.sh
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
res = subprocess.run(["bash", updater, "--force"], cwd=self.test_dir, env=self.env, capture_output=True, text=True)
self.assertEqual(res.returncode, 0, f"update.sh failed: {res.stderr}")
# Run the update skill (Issue #72: .mam_deploy/update.sh is gone)
updater = os.path.join(self.test_dir, ".agents", "skills", "multi-agent-mux-update", "scripts", "update.sh")
env = {k: v for k, v in self.env.items() if not k.startswith("HERDR_")
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
# The update lists the live herdr agents; never ask a real server here.
fake = os.path.join(self.temp_dir, "fake-herdr-bin")
os.makedirs(fake)
with open(os.path.join(fake, "herdr"), "w") as f:
f.write('#!/bin/sh\necho \'{"sessions": []}\'\n')
os.chmod(os.path.join(fake, "herdr"), 0o755)
env["PATH"] = fake + ":" + env["PATH"]
res = subprocess.run(["bash", updater, "--yes"], cwd=self.test_dir, env=env, capture_output=True, text=True,
stdin=subprocess.DEVNULL)
self.assertEqual(res.returncode, 0, f"the update skill failed: {res.stdout}{res.stderr}")
self.assertFalse(os.path.exists(os.path.join(self.test_dir, ".mam_deploy", "update.sh")))
# Verify asset_hashes.txt and version.txt still exist
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".mam", "asset_hashes.txt")))
+2
View File
@@ -59,6 +59,8 @@ INT_KNOBS = {
# once) and the wrapper's registry wait (0: decide at once - clear or busy)
"MAM_ONBOARD_WAIT_S": (1, "600"),
"MAM_DISPATCH_WAIT_S": (0, "600"),
# Issue #72: how long the update waits for the monitor to stop after SIGTERM
"MAM_UPDATE_MONITOR_STOP_S": (1, "30"),
# delegate_wait_started's ack_s (MAM_ONBOARD_ACK_S / DELEGATE_JOB_ACK_S via create_session.sh)
"delegate_wait_started:ack_s": (0, "180"),
# wrapper flags
+2 -2
View File
@@ -115,8 +115,8 @@ def test_inventory_matches_skill_md(skill):
f"actual (start, end, lang, sha): {json.dumps(actual)}")
def test_inventory_covers_eight_skills():
assert len(SKILL_NAMES) == 8
def test_inventory_covers_nine_skills():
assert len(SKILL_NAMES) == 9 # Issue #72: + multi-agent-mux-update
assert {b["skill"] for b in DATA["blocks"]} == set(SKILL_NAMES)
+328
View File
@@ -0,0 +1,328 @@
"""Issue #72 (decision 8, U2): while an update runs, the entries that start
something or write the registry refuse - create, resume, run_loop.sh, delegate
submit, stop (plain and --purge-conversation) and orc-onboard - and change
nothing. A marker its update left behind (`stale`) refuses nothing.
The marker is .mam/update-in-progress as the update skill publishes it (pid +
lstart, loop_lock.sh); `live` here is a sleeping helper process's.
"""
from __future__ import annotations
import hashlib
import json
import os
import subprocess
import sys
from pathlib import Path
import pytest
import yaml
UUID = "11111111-2222-3333-4444-555555555555"
ORC = "99999999-8888-7777-6666-555555555555"
ORC_OLD = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
@pytest.fixture
def marker(mam_sandbox):
procs = []
def make(live: bool, root: Path = None):
p = (root or mam_sandbox) / ".mam" / "update-in-progress"
if live:
proc = subprocess.Popen(["sleep", "120"], start_new_session=True)
procs.append(proc)
ls = " ".join(subprocess.run(["ps", "-p", str(proc.pid), "-o", "lstart="],
capture_output=True, text=True).stdout.split())
p.write_text("pid=%d\nlstart=%s\nstarted=x\n" % (proc.pid, ls))
else:
dead = subprocess.Popen(["true"])
dead.wait()
p.write_text("pid=%d\nlstart=Thu Jan 1 00:00:00 1970\nstarted=x\n" % dead.pid)
return p
yield make
for proc in procs:
proc.kill()
proc.wait()
def _seed(sandbox, rows, **top):
lib = sandbox / ".agents" / "skills" / "lib.sh"
mutation = "d['herdr_sessions'] = %r\n" % (rows,) + "".join("d[%r] = %r\n" % kv for kv in top.items())
res = subprocess.run(["bash", "-c", 'source "$1" && atomic_dump_yaml "$AGENT_SESSIONS_YAML"', "_", str(lib)],
input=mutation, capture_output=True, text=True)
assert res.returncode == 0, res.stderr
def _registry(sandbox):
out = {}
for f in ("agent-sessions.yaml", "agent-sessions.db"):
p = sandbox / ".mam" / f
out[f] = hashlib.sha256(p.read_bytes()).hexdigest() if p.exists() else None
return out
def _script(sandbox, rel):
return str(sandbox / ".agents" / "skills" / rel)
def _run(sandbox, argv, env=None, timeout=120):
return subprocess.run(argv, capture_output=True, text=True, cwd=str(sandbox), timeout=timeout,
env=dict(os.environ, **(env or {})), stdin=subprocess.DEVNULL)
def _refused(res, sandbox):
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s is in progress" % sandbox in res.stderr, res.stdout + res.stderr
def _row(name, status="running", agent="claude"):
r = {"name": name, "status": status, "role": "creator", "herdr_session": "default", "herdr_server": "default",
"pane": {"cwd": "", "cmd": agent}, "agent": agent}
if agent == "claude":
r["claude_session_id_own"] = UUID
return r
def _agent(cwd, pane):
return {"agent": "claude", "status": "running", "cwd": str(cwd), "workspace_id": "w1",
"pid": 9999, "pane_id": pane, "command": "claude", "buffer": "Ready"}
def _set_mock(state_file, **kw):
st = json.loads(Path(state_file).read_text())
st.update(kw)
Path(state_file).write_text(json.dumps(st))
def test_create_refuses_while_an_update_runs(mam_sandbox, mock_herdr, mock_agents, marker):
marker(True)
before = _registry(mam_sandbox)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-create/scripts/create_session.sh"),
"--workspace", str(mam_sandbox), "--agent", "claude", "--role", "creator",
"--session", "gated-create", "--no-onboard"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
def test_resume_refuses_on_the_spawn_and_the_alive_path(mam_sandbox, mock_herdr, mock_agents, marker):
_seed(mam_sandbox, [dict(_row("gated-agy", status="stopped", agent="agy"), pane={"cwd": str(mam_sandbox), "cmd": "agy"})])
marker(True)
before = _registry(mam_sandbox)
script = _script(mam_sandbox, "multi-agent-mux-resume/scripts/resume_session.sh")
res = _run(mam_sandbox, ["bash", script, "--workspace", str(mam_sandbox), "--agent", "agy",
"--session", "gated-agy"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
_set_mock(mock_herdr, agents={"gated-agy": _agent(mam_sandbox, "w1:p9")})
res = _run(mam_sandbox, ["bash", script, "--workspace", str(mam_sandbox), "--agent", "agy",
"--session", "gated-agy"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
def test_run_loop_refuses_before_its_loop_marker(mam_sandbox, marker):
marker(True)
loop_marker = mam_sandbox / ".mam" / "loop-guard-active"
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-loop/scripts/run_loop.sh"),
"--creator", "c", "--task", "t"],
env={"MAM_LOOP_NO_FREEZE": "1", "MAM_REAL_ROOT": str(mam_sandbox)})
_refused(res, mam_sandbox)
assert not loop_marker.exists()
def test_delegate_submit_registers_no_job(mam_sandbox, mock_herdr, marker):
marker(True)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-delegate-job/multi-agent-mux-delegate-job"),
"submit", "--agent", "claude-code", "--agent-session", "herdr:anyone", "--prompt", "hi"])
_refused(res, mam_sandbox)
jobs = mam_sandbox / ".mam" / "jobs"
assert not list(jobs.glob("*.json")) if jobs.exists() else True
@pytest.fixture
def other_ws(tmp_path):
"""A second installed workspace B (update.sh's own test: .mam_deploy/remove.sh),
outside the sandbox (A)."""
b = Path(os.path.realpath(tmp_path)) / "ws-b"
(b / ".mam").mkdir(parents=True)
(b / ".mam_deploy").mkdir()
(b / ".mam_deploy" / "remove.sh").write_text("#!/bin/sh\n")
(b / "sub").mkdir()
return b
def _submit_to(sandbox, session, env=None):
return _run(sandbox, ["bash", _script(sandbox, "multi-agent-mux-delegate-job/multi-agent-mux-delegate-job"),
"submit", "--agent", "claude-code", "--agent-session", session, "--prompt", "hi"], env=env)
def _no_job(sandbox):
jobs = sandbox / ".mam" / "jobs"
return not jobs.exists() or not list(jobs.glob("*.json"))
def _server_for(sandbox, name):
"""The server the wrapper delivers a rowless target on (resolve_herdr_session)."""
res = subprocess.run(["bash", "-c", 'source "$1" && resolve_herdr_session "$2" "$3"', "_",
str(sandbox / ".agents" / "skills" / "lib.sh"), name, str(sandbox)],
capture_output=True, text=True, cwd=str(sandbox))
assert res.returncode == 0 and res.stdout.strip(), res.stderr
return res.stdout.strip()
def _target_in(kind, sandbox, mock_herdr, b):
"""Put the delegate target `tgt` in workspace B as <kind>; the env for submit."""
if kind == "rowless-live":
_set_mock(mock_herdr, agents={"tgt": _agent(b / "sub", "w1:p6")})
return {"MAM_SESS": _server_for(sandbox, "tgt")}
r = _row("tgt")
if kind == "row-pane-cwd":
r["pane"]["cwd"] = str(b / "sub")
elif kind == "row-control-plane":
r["pane"]["cwd"] = "/nonexistent/tgt"
r["control_plane"] = {"workspace_root": str(b)}
elif kind == "row-under-incidental-mam":
(b / "sub" / ".mam").mkdir() # runtime state only: not an installed tree
r["pane"]["cwd"] = str(b / "sub")
r["control_plane"] = {"workspace_root": str(b)}
_seed(sandbox, [r])
return {}
KINDS = ["row-pane-cwd", "row-control-plane", "row-under-incidental-mam", "rowless-live"]
@pytest.mark.parametrize("kind", KINDS)
def test_delegate_submit_takes_the_target_workspace_marker(mam_sandbox, mock_herdr, marker, other_ws, kind):
"""R1 (rounds 1, 2): every installed workspace the target is in - by its row,
or for a rowless target by the live agent's cwd - is checked, not only the
caller's; an incidental .mam below B does not hide B."""
env = _target_in(kind, mam_sandbox, mock_herdr, other_ws)
marker(True, other_ws)
res = _submit_to(mam_sandbox, "herdr:tgt", env=env)
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s is in progress" % other_ws in res.stderr, res.stdout + res.stderr
assert "registered job" not in res.stdout
assert _no_job(mam_sandbox)
@pytest.mark.parametrize("kind", KINDS)
def test_delegate_submit_waits_for_the_target_workspace_admission_lock(mam_sandbox, mock_herdr, other_ws, kind):
env = _target_in(kind, mam_sandbox, mock_herdr, other_ws)
holder = subprocess.Popen([sys.executable, "-c", "import fcntl, sys, time\n"
"f = open(sys.argv[1], 'a'); fcntl.flock(f, fcntl.LOCK_EX)\n"
"print('locked', flush=True); time.sleep(120)",
str(other_ws / ".mam" / "update-admission.lock")],
stdout=subprocess.PIPE, text=True, start_new_session=True)
try:
assert holder.stdout.readline().strip() == "locked"
res = _submit_to(mam_sandbox, "herdr:tgt", env=dict(env, MAM_UPDATE_ADMISSION_WAIT_S="1"))
finally:
holder.kill()
holder.wait()
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s holds its admission lock" % other_ws in res.stderr, res.stdout + res.stderr
assert _no_job(mam_sandbox)
def _start_in(sandbox, entry, b, env=None):
"""create or resume (spawn path) of an agent whose pane cwd is B/sub."""
if entry == "create":
argv = ["bash", _script(sandbox, "multi-agent-mux-create/scripts/create_session.sh"),
"--workspace", str(b / "sub"), "--agent", "claude", "--role", "creator",
"--session", "gated-in-b", "--no-onboard"]
else:
_seed(sandbox, [dict(_row("gated-in-b", status="stopped", agent="agy"),
pane={"cwd": str(b / "sub"), "cmd": "agy"})])
argv = ["bash", _script(sandbox, "multi-agent-mux-resume/scripts/resume_session.sh"),
"--workspace", str(b / "sub"), "--agent", "agy", "--session", "gated-in-b"]
return _run(sandbox, argv, env=env, timeout=300)
@pytest.mark.parametrize("entry", ["create", "resume"])
def test_create_and_resume_take_the_pane_workspace_marker(mam_sandbox, mock_herdr, mock_agents, marker,
other_ws, entry):
"""An agent started in an installed workspace B other than the control root
is one B's update counts, so B's marker refuses it too."""
marker(True, other_ws)
res = _start_in(mam_sandbox, entry, other_ws)
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s is in progress" % other_ws in res.stderr, res.stdout + res.stderr
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
@pytest.mark.parametrize("entry", ["create", "resume"])
def test_create_and_resume_wait_for_the_pane_workspace_admission_lock(mam_sandbox, mock_herdr, mock_agents,
other_ws, entry):
holder = subprocess.Popen([sys.executable, "-c", "import fcntl, sys, time\n"
"f = open(sys.argv[1], 'a'); fcntl.flock(f, fcntl.LOCK_EX)\n"
"print('locked', flush=True); time.sleep(120)",
str(other_ws / ".mam" / "update-admission.lock")],
stdout=subprocess.PIPE, text=True, start_new_session=True)
try:
assert holder.stdout.readline().strip() == "locked"
res = _start_in(mam_sandbox, entry, other_ws, env={"MAM_UPDATE_ADMISSION_WAIT_S": "1"})
finally:
holder.kill()
holder.wait()
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s holds its admission lock" % other_ws in res.stderr, res.stdout + res.stderr
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
@pytest.mark.parametrize("purge", [False, True])
def test_stop_refuses_and_changes_nothing(mam_sandbox, mock_herdr, marker, purge):
_seed(mam_sandbox, [_row("gated-stop")])
_set_mock(mock_herdr, agents={"gated-stop": _agent(mam_sandbox, "w1:p8")})
marker(True)
before = _registry(mam_sandbox)
extra = ["--purge-conversation", "--yes"] if purge else []
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-stop/scripts/stop_session.sh"),
"--session", "gated-stop", "--agent", "claude", *extra])
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert "gated-stop" in json.loads(Path(mock_herdr).read_text())["agents"], "the session was not touched"
assert not (mam_sandbox / ".mam" / "purging-gated-stop").exists()
def test_orc_onboard_refuses_and_changes_nothing(mam_sandbox, marker):
_seed(mam_sandbox, [], orchestrator_uuids=[ORC_OLD])
marker(True)
before = _registry(mam_sandbox)
script = _script(mam_sandbox, "multi-agent-mux-orc-onboard/scripts/orc_onboard.sh")
for args in (["--uuid", ORC], ["--remove", ORC_OLD]):
res = _run(mam_sandbox, ["bash", script, *args])
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
def test_a_stale_marker_refuses_nothing(mam_sandbox, mock_herdr, marker):
_seed(mam_sandbox, [_row("stale-stop")], orchestrator_uuids=[])
_set_mock(mock_herdr, agents={"stale-stop": _agent(mam_sandbox, "w1:p7")})
marker(False)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-orc-onboard/scripts/orc_onboard.sh"),
"--uuid", ORC])
assert res.returncode == 0, res.stdout + res.stderr
assert "stale update marker" in res.stderr
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-stop/scripts/stop_session.sh"),
"--session", "stale-stop", "--agent", "claude"])
assert res.returncode == 0, res.stdout + res.stderr
d = yaml.safe_load((mam_sandbox / ".mam" / "agent-sessions.yaml").read_text())
assert d["orchestrator_uuids"] == [ORC]
assert {r["name"]: r["status"] for r in d["herdr_sessions"]} == {"stale-stop": "stopped"}
def test_the_admission_lock_is_released_after_the_record(mam_sandbox, marker):
"""An entry takes the lock shared and drops it once its record is written:
a later exclusive taker (the update) gets it at once."""
lib = mam_sandbox / ".agents" / "skills" / "lib.sh"
res = subprocess.run(["bash", "-c", 'source "$1" && mam_update_admit && mam_update_admit_release && '
'exec 7>>"$WORKSPACE_ROOT/.mam/update-admission.lock" && '
'MAM_UPDATE_ADMISSION_WAIT_S=1 python3 -m lib_py.pane_lock acquire --fd 7 --wait-env MAM_UPDATE_ADMISSION_WAIT_S',
"_", str(lib)], capture_output=True, text=True, timeout=60)
assert res.returncode == 0 and res.stdout.strip() == "locked", res.stdout + res.stderr
+184
View File
@@ -0,0 +1,184 @@
"""Issue #72: the attach hints create, stop and the delegate-job wrapper print
are the session row's attach_command (Issue #71); with no row, the one
lib_py.agents.sanitize.attach_command builds for the server the script used.
"""
import json
import os
import re
import subprocess
import sys
import time
from pathlib import Path
import pytest
import yaml
REPO = Path(__file__).resolve().parent.parent
SKILLS = REPO / ".agents" / "skills"
sys.path.insert(0, str(SKILLS))
from lib_py.agents.sanitize import attach_command, sanitize_herdr_agent_name # noqa: E402
LONG = "lab-issue72-a-very-long-branch-name-creator-claude" # > 32 characters
UPPER = "Lab-Upper72-creator-claude"
def _set_mock(mock_herdr, **kw):
state = json.loads(Path(mock_herdr).read_text())
state.update(kw)
Path(mock_herdr).write_text(json.dumps(state))
def _rows(sandbox):
return yaml.safe_load((sandbox / ".mam" / "agent-sessions.yaml").read_text())["herdr_sessions"]
def _seed(sandbox, rows):
lib = sandbox / ".agents" / "skills" / "lib.sh"
mutation = "d['herdr_sessions'] = " + repr(rows) + "\n"
res = subprocess.run(["bash", "-c", 'source "$1" && atomic_dump_yaml "$AGENT_SESSIONS_YAML"', "_", str(lib)],
input=mutation, capture_output=True, text=True)
assert res.returncode == 0, res.stderr
def _row(name, server, cwd, pid=1):
return {"name": name, "status": "running", "role": "creator", "agent": "claude",
"herdr_session": server, "herdr_server": server, "attach_command": attach_command(server, name),
"pane": {"index": 0, "pid": pid, "cmd": "claude", "cwd": str(cwd)}}
def _seeded_attach(sandbox, name, server):
(row,) = [r for r in _rows(sandbox) if r["name"] == name]
assert row["attach_command"] == attach_command(server, name)
return row["attach_command"]
def _agent(cwd, pane, pid=9999):
return {"agent": "claude", "status": "running", "agent_status": "idle", "cwd": str(cwd), "workspace_id": "w1",
"pid": pid, "pane_id": pane, "command": "claude", "buffer": "Ready"}
# --- create ---------------------------------------------------------------------
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_create_prints_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
assert sanitize_herdr_agent_name(name) != name, "the herdr name differs from the session name"
_set_mock(mock_herdr, strict_agent_names=True)
script = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-create" / "scripts" / "create_session.sh"
res = subprocess.run(["bash", str(script), "--workspace", str(mam_sandbox), "--agent", "claude",
"--role", "creator", "--session", name, "--herdr-session", "mam-iso72", "--no-onboard"],
capture_output=True, text=True, cwd=str(mam_sandbox), timeout=300,
env=dict(os.environ, MAM_NO_SUBSCRIBER_DAEMON="1"))
assert res.returncode == 0, res.stdout + res.stderr
(row,) = [r for r in _rows(mam_sandbox) if r["name"] == name]
hints = [l for l in res.stdout.splitlines() if l.startswith("Attach:")]
assert hints == ["Attach: " + row["attach_command"]], res.stdout
assert row["attach_command"] == attach_command("mam-iso72", name)
# --- stop: the stuck-TUI message ------------------------------------------------
def _undying_herdr(mam_sandbox):
"""The mock herdr behind a wrapper that ignores every way to close a pane,
so the session survives stop's kill chain."""
bin_dir = mam_sandbox / "bin"
(bin_dir / "herdr").rename(bin_dir / "herdr-mock72")
(bin_dir / "herdr").write_text(
'#!/usr/bin/env bash\nfor a in "$@"; do case "$a" in kill-session|close) exit 0 ;; esac; done\n'
'exec "%s" "$@"\n' % (bin_dir / "herdr-mock72"))
(bin_dir / "herdr").chmod(0o755)
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_stop_stuck_tui_message_shows_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
# The SIGKILL fallback kills the pane pid: make it a child of this test.
pane = subprocess.Popen(["sleep", "300"])
try:
# The row's server is not the one passed on the command line: the row wins.
_seed(mam_sandbox, [_row(name, "mam-row72", mam_sandbox, pid=pane.pid)])
want = _seeded_attach(mam_sandbox, name, "mam-row72")
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72", pid=pane.pid)})
_undying_herdr(mam_sandbox)
script = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-stop" / "scripts" / "stop_session.sh"
res = subprocess.run(["bash", str(script), "--session", name, "--agent", "claude",
"--herdr-session", "mam-arg72", "--purge-conversation", "--yes"],
capture_output=True, text=True, cwd=str(mam_sandbox), timeout=300,
env=dict(os.environ, SKS_IDLE_TIMEOUT_MS="500"))
finally:
pane.kill()
pane.wait()
assert res.returncode == 4, res.stdout + res.stderr
hints = [l for l in res.stderr.splitlines() if "Diagnose the stuck TUI" in l]
assert hints == [" Diagnose the stuck TUI (%s), then re-run" % want], res.stderr
# --- delegate-job wrapper ---------------------------------------------------------
def _delegate(mam_sandbox, mock_herdr, name):
"""Submit a job to herdr:<name>, acknowledge it as the agent would, and
return (stdout, the herdr server the wrapper prompted)."""
reg_dir = mam_sandbox / ".mam" / "jobs"
reg_dir.mkdir(parents=True, exist_ok=True)
wrapper = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-delegate-job" / "multi-agent-mux-delegate-job"
publish = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-delegate-job" / "scripts" / "publish_event.py"
env = dict(os.environ, DELEGATE_JOB_PYTHON=sys.executable, MQTT_HOST="127.0.0.1", MQTT_PORT="65534",
SKS_IDLE_TIMEOUT_MS="500", MAM_INJECT_ACK_GRACE_S="20")
proc = subprocess.Popen(["bash", str(wrapper), "submit", "--agent", "claude-code",
"--agent-session", "herdr:" + name, "--prompt", "issue 72 attach hint",
"--registry-dir", str(reg_dir), "--workdir", str(mam_sandbox),
"--timeout", "6", "--idle-timeout", "2"],
cwd=str(mam_sandbox), env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
try:
tok, jid = "", ""
deadline = time.time() + 60
while time.time() < deadline and not tok:
for f in reg_dir.glob("*.json"):
try:
tok, jid = json.loads(f.read_text()).get("dispatch_token") or "", f.stem
except ValueError:
pass
time.sleep(0.2)
assert tok, "the job was not dispatched"
ack = subprocess.run([sys.executable, str(publish), "--registry-dir", str(reg_dir), "--job", jid,
"--event", "started", "--expect-dispatch-token", tok, "--attempt-id", "1"],
capture_output=True, text=True, cwd=str(mam_sandbox), env=env, timeout=60)
# publish_event records `started` locally before its (here unreachable) broker publish
assert '"started"' in (reg_dir / (jid + ".events.log")).read_text(), ack.stderr
stdout, stderr = proc.communicate(timeout=180)
finally:
if proc.poll() is None:
proc.kill()
proc.communicate()
calls = json.loads(Path(mock_herdr).read_text())["calls"]
prompts = [c for c in calls if "agent" in c[:3] and "prompt" in c[:4]]
assert prompts, (calls, stdout, stderr)
servers = {c[1] if c[0] == "--session" else "default" for c in prompts}
assert len(servers) == 1, prompts
return stdout + stderr, servers.pop()
def _delegate_hint(out):
hints = [m.group(1) for m in (re.search(r"\(연결하려면: (.*)\)$", l) for l in out.splitlines()) if m]
assert len(hints) == 1, out
return hints[0]
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_delegate_prints_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
_seed(mam_sandbox, [_row(name, "mam-row72", mam_sandbox)])
want = _seeded_attach(mam_sandbox, name, "mam-row72")
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72")})
out, server = _delegate(mam_sandbox, mock_herdr, name)
assert server == "mam-row72"
assert _delegate_hint(out) == want
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_delegate_without_a_row_builds_it_for_the_server_it_used(mam_sandbox, mock_herdr, mock_agents, name):
assert not any(r["name"] == name for r in _rows(mam_sandbox))
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72")})
out, server = _delegate(mam_sandbox, mock_herdr, name)
assert _delegate_hint(out) == attach_command(server, name)
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Issue #72 — the installer side of the multi-agent-mux-update skill.
The update skill runs the new install.sh in place over an existing install
(no remove.sh, .mam kept). These pin what that in-place run must do:
install_state keeps one line per key, no updater is shipped any more and an
old one is cleaned up, and framework files the new version no longer ships
are deleted unless the user modified them.
"""
from __future__ import annotations
import os
import shutil
import subprocess
import tempfile
import pytest
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
# Extra framework-owned files only the "old" source ships.
RETIRED_SKILL = ".agents/skills/multi-agent-mux-retired72"
RETIRED_UNMODIFIED = [RETIRED_SKILL + "/SKILL.md", RETIRED_SKILL + "/scripts/old.sh"]
RETIRED_SHARED_DIR = ".agents/skills/multi-agent-mux-retired72b"
RETIRED_SHARED = RETIRED_SHARED_DIR + "/SKILL.md"
RETIRED_MODIFIED = ".agents/hooks/retired72_hook.sh"
def _copy_tree(dest):
shutil.copytree(REPO_ROOT, dest,
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", ".mam_deploy",
"__pycache__", ".pytest_cache"),
symlinks=True)
@pytest.fixture(scope="module")
def sources():
"""(new, old): the tree as is, and the same tree plus files it no longer ships."""
tmp = tempfile.mkdtemp(prefix="mam_issue72_src_")
new, old = os.path.join(tmp, "new"), os.path.join(tmp, "old")
_copy_tree(new)
_copy_tree(old)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
path = os.path.join(old, rel)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w") as f:
f.write("# shipped by the old version only: %s\n" % rel)
try:
yield new, old
finally:
shutil.rmtree(tmp, ignore_errors=True)
@pytest.fixture
def ws():
tmp = tempfile.mkdtemp(prefix="mam_issue72_ws_")
try:
yield os.path.join(tmp, "workspace")
finally:
shutil.rmtree(tmp, ignore_errors=True)
def _install(src, ws):
env = dict(os.environ)
env.update({"MAM_REPO_URL": src, "MAM_SKIP_VENV": "1"})
res = subprocess.run(["bash", os.path.join(src, "deploy", "install.sh"), ws],
env=env, capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return res
def _remove(ws):
res = subprocess.run(["bash", os.path.join(ws, ".mam_deploy", "remove.sh"), "--force"],
cwd=ws, capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return res
def _read(ws, rel):
with open(os.path.join(ws, rel)) as f:
return f.read()
def _manifest(ws):
return _read(ws, ".mam/install_manifest.txt").splitlines()
def _hashed_paths(ws):
return [line.split(None, 1)[1] for line in _read(ws, ".mam/asset_hashes.txt").splitlines()
if line.strip()]
def test_in_place_reinstall_keeps_one_line_per_install_state_key(sources, ws):
new, _ = sources
_install(new, ws)
with open(os.path.join(ws, ".mam", "install_state"), "w") as f:
f.write("first=a\ngitignore_created=1\nlast=b\n")
_install(new, ws)
assert _read(ws, ".mam/install_state").splitlines() == [
"first=a", "gitignore_created=1", "last=b"]
# remove.sh must read the key as 1: the .gitignore it created goes away.
_remove(ws)
assert not os.path.exists(os.path.join(ws, ".gitignore"))
def test_fresh_install_ships_no_updater(sources, ws):
new, _ = sources
res = _install(new, ws)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
manifest = _manifest(ws)
assert ".mam_deploy/update.sh" not in manifest and "update.sh" not in manifest
assert "bash .mam_deploy/update.sh" not in res.stdout
assert "multi-agent-mux-update/scripts/update.sh" in res.stdout
def test_in_place_reinstall_deletes_an_old_mam_deploy_updater(sources, ws):
new, _ = sources
_install(new, ws)
with open(os.path.join(ws, ".mam_deploy", "update.sh"), "w") as f:
f.write("#!/usr/bin/env bash\n# old updater\n")
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
f.write(".mam_deploy/update.sh\n")
_install(new, ws)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
assert ".mam_deploy/update.sh" not in _manifest(ws)
def test_legacy_root_updater_goes_only_when_the_manifest_lists_it(sources, ws):
new, _ = sources
_install(new, ws)
root_updater = os.path.join(ws, "update.sh")
# Listed: the installer put it there (very old layout).
with open(root_updater, "w") as f:
f.write("# legacy updater\n")
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
f.write("update.sh\n")
_install(new, ws)
assert not os.path.exists(root_updater)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
manifest = _manifest(ws)
assert "update.sh" not in manifest and ".mam_deploy/update.sh" not in manifest
# Not listed: the user's own file stays untouched.
with open(root_updater, "w") as f:
f.write("# user's own script\n")
_install(new, ws)
assert _read(ws, "update.sh") == "# user's own script\n"
def test_files_the_new_version_no_longer_ships(sources, ws):
new, old = sources
_install(old, ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
assert rel in _manifest(ws) and rel in _hashed_paths(ws), rel
# The user edits one retired file and adds their own file next to another.
with open(os.path.join(ws, RETIRED_MODIFIED), "a") as f:
f.write("# my change\n")
with open(os.path.join(ws, RETIRED_SHARED_DIR, "notes.txt"), "w") as f:
f.write("mine\n")
# A base/ copy of a retired file must go with it.
base_copy = os.path.join(ws, ".mam", "base", RETIRED_UNMODIFIED[0])
os.makedirs(os.path.dirname(base_copy))
shutil.copy(os.path.join(ws, RETIRED_UNMODIFIED[0]), base_copy)
res = _install(new, ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED]:
assert not os.path.exists(os.path.join(ws, rel)), rel
assert not os.path.exists(os.path.join(ws, RETIRED_SKILL)), "emptied directory left behind"
assert _read(ws, RETIRED_SHARED_DIR + "/notes.txt") == "mine\n"
assert "# my change" in _read(ws, RETIRED_MODIFIED)
assert "Local modification detected" in res.stderr and RETIRED_MODIFIED in res.stderr
for rel in RETIRED_UNMODIFIED:
assert rel not in res.stderr, rel
assert not os.path.exists(base_copy)
manifest, hashed = _manifest(ws), _hashed_paths(ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
assert rel not in manifest and rel not in hashed, rel
# Still-shipped framework files are untouched by the cleanup.
assert ".agents/skills/lib.sh" in manifest and ".agents/skills/lib.sh" in hashed
# The kept file is the user's now: uninstalling leaves it in place.
_remove(ws)
assert "# my change" in _read(ws, RETIRED_MODIFIED)
@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0,
reason="root reads a mode-000 file")
def test_local_source_update_backup_is_not_staged(sources, ws):
new, _ = sources
backup = os.path.join(new, ".mam", "update-backup")
os.makedirs(backup)
unreadable = os.path.join(backup, "unreadable")
with open(unreadable, "w") as f:
f.write("x")
os.chmod(unreadable, 0)
try:
_install(new, ws)
finally:
os.chmod(unreadable, 0o600)
shutil.rmtree(os.path.join(new, ".mam"))
+183
View File
@@ -0,0 +1,183 @@
"""Issue #72 (decision 2): resume ensures the monitor daemon, as create does.
Only create_session.sh started the per-workspace subscriber daemon, so a
session brought back by resume_session.sh had no monitor unless one already
ran. Resume now calls the same `start_watchdog` with the control root as the
workdir (Issue #21 A8: never a worktree --workspace's copy), on the spawn path
and the already-running path, after the row is written.
What is real: the shipped create/stop/resume scripts and the real
start_watchdog -> lib_py ensure-daemon, against the mock herdr in a sandbox.
What is not: the sandbox's monitor script is a stub that records how it was
started and sleeps, so the daemon's `pgrep -f "bash <path> --subscribe"`
check sees it but nothing reconciles. Every stub is killed in teardown.
"""
import os
import signal
import subprocess
import time
import pytest
CREATE = "multi-agent-mux-create/scripts/create_session.sh"
STOP = "multi-agent-mux-stop/scripts/stop_session.sh"
RESUME = "multi-agent-mux-resume/scripts/resume_session.sh"
MONITOR_REL = os.path.join(".agents", "skills", "multi-agent-mux-monitor", "scripts", "reconcile.sh")
def _script(root, rel):
return str(root / ".agents" / "skills" / rel)
def _install_stub_monitor(base, log):
"""Replace <base>'s monitor with a stub that writes `pid|$0|cwd|args`."""
path = base / MONITOR_REL
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('#!/bin/bash\nprintf \'%%s|%%s|%%s|%%s\\n\' "$$" "$0" "$PWD" "$*" >> "%s"\n'
'sleep 60 &\nwait $!\n' % log)
path.chmod(0o755)
return str(path)
def _starts(log):
return [line.split("|") for line in log.read_text().splitlines()] if log.exists() else []
def _wait_starts(log, n, timeout=10.0):
deadline = time.time() + timeout
while time.time() < deadline and len(_starts(log)) < n:
time.sleep(0.1)
return _starts(log)
@pytest.fixture
def stub(mam_sandbox):
"""The control root's stub monitor; kills every stub process it recorded
(the mam_sandbox reaper also sweeps anything under the sandbox)."""
logs = []
def install(base, name):
log = mam_sandbox / ("monitor-starts-%s.log" % name)
logs.append(log)
return _install_stub_monitor(base, log), log
yield install
for log in logs:
for row in _starts(log):
pid = int(row[0])
for target in subprocess.run(["pgrep", "-P", str(pid)], capture_output=True,
text=True).stdout.split() + [str(pid)]:
try:
os.kill(int(target), signal.SIGKILL)
except OSError:
pass
def _create(root, ws, name, stop):
subprocess.run(["bash", _script(root, CREATE), "--workspace", ws, "--agent", "claude",
"--role", "creator", "--session", name, "--no-onboard"],
check=True, cwd=str(root), capture_output=True, text=True)
if stop:
subprocess.run(["bash", _script(root, STOP), "--session", name, "--agent", "claude"],
check=True, cwd=str(root), capture_output=True, text=True)
def _resume(root, ws, name):
return subprocess.run(["bash", _script(root, RESUME), "--workspace", ws, "--agent", "claude",
"--session", name], cwd=str(root), capture_output=True, text=True, timeout=120)
def _assert_root_monitor_started(root, monitor, log, res):
starts = _wait_starts(log, 1)
assert len(starts) == 1, starts
pid, script, cwd, args = starts[0]
assert os.path.realpath(script) == os.path.realpath(monitor)
assert os.path.realpath(cwd) == os.path.realpath(str(root))
assert args == "--subscribe --idle-timeout 0"
assert "watchdog PID: %s" % pid in res.stdout, res.stdout
def test_spawn_path_ensures_the_control_root_monitor(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-spawn", stop=True)
monitor, log = stub(root, "root")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-spawn")
assert res.returncode == 0, res.stdout + res.stderr
assert "Successfully resumed" in res.stdout
_assert_root_monitor_started(root, monitor, log, res)
def test_already_running_path_ensures_the_monitor_once(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-alive", stop=False)
monitor, log = stub(root, "root")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-alive")
assert res.returncode == 0, res.stdout + res.stderr
assert "already running" in res.stdout
_assert_root_monitor_started(root, monitor, log, res)
# A second resume finds the running daemon and starts no other.
pid = _starts(log)[0][0]
again = _resume(root, str(root), "issue72-alive")
assert again.returncode == 0, again.stdout + again.stderr
assert "watchdog PID: %s" % pid in again.stdout
time.sleep(1)
assert len(_starts(log)) == 1
@pytest.mark.parametrize("stop", [True, False], ids=["spawn", "already-running"])
def test_worktree_session_uses_the_control_root_monitor(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch, stop):
"""A session whose pane cwd (--workspace) is not the control root still
gets the control root's monitor, never the copy in its own tree."""
root = mam_sandbox
wt = root / "wt-issue72"
wt.mkdir()
name = "issue72-wt-" + ("spawn" if stop else "alive")
_create(root, str(wt), name, stop=stop)
monitor, log = stub(root, "root")
_, wt_log = stub(wt, "worktree")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(wt), name)
assert res.returncode == 0, res.stdout + res.stderr
_assert_root_monitor_started(root, monitor, log, res)
assert _starts(wt_log) == []
def test_opt_out_starts_nothing_and_reports_a_running_daemon(mam_sandbox, mock_herdr, mock_agents, stub):
"""MAM_NO_SUBSCRIBER_DAEMON=1 (the sandbox default): resume starts no
daemon; one that already runs is still found and reported."""
root = mam_sandbox
assert os.environ.get("MAM_NO_SUBSCRIBER_DAEMON") == "1"
_create(root, str(root), "issue72-optout", stop=True)
monitor, log = stub(root, "root")
res = _resume(root, str(root), "issue72-optout") # spawn path
assert res.returncode == 0, res.stdout + res.stderr
time.sleep(1)
assert _starts(log) == []
pre = subprocess.Popen(["bash", monitor, "--subscribe", "--idle-timeout", "0"], cwd=str(root),
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
start_new_session=True)
try:
assert len(_wait_starts(log, 1)) == 1
res = _resume(root, str(root), "issue72-optout") # already-running path
assert res.returncode == 0, res.stdout + res.stderr
assert "watchdog PID: %d" % pre.pid in res.stdout, res.stdout
time.sleep(1)
assert len(_starts(log)) == 1
finally:
pre.kill()
pre.wait()
def test_a_missing_monitor_warns_and_the_resume_still_succeeds(mam_sandbox, mock_herdr, mock_agents, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-nomon", stop=True)
os.remove(str(root / MONITOR_REL))
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-nomon")
assert res.returncode == 0, res.stdout + res.stderr
assert "Successfully resumed" in res.stdout
assert "WARNING: could not start the monitor daemon" in res.stderr
+966
View File
@@ -0,0 +1,966 @@
"""Issue #72 - the multi-agent-mux-update skill.
Every test updates a copy of one workspace installed from this checkout
(MAM_REPO_URL, MAM_SKIP_VENV=1). herdr is a fake that answers `session list
--json` and `--session S agent list` from a JSON spec, so the pre-check's view
of the live agents is set per test; the monitor daemon is a stub reconcile.sh
that holds .mam/monitor.lock like the real one. Nothing here touches a real
herdr server or the control root's monitor.
"""
from __future__ import annotations
import fcntl
import hashlib
import json
import os
import shutil
import signal
import sqlite3
import subprocess
import sys
import time
from pathlib import Path
import pytest
import yaml
REPO_ROOT = Path(__file__).resolve().parent.parent
UPDATE_REL = ".agents/skills/multi-agent-mux-update/scripts/update.sh"
MONITOR_REL = ".agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh"
FAKE_HERDR = r'''#!/usr/bin/env python3
import json, os, sys
spec = json.load(open(os.environ["FAKE_HERDR_SPEC"]))
a = sys.argv[1:]
server = "default"
if a[:1] == ["--session"]:
server, a = a[1], a[2:]
with open(os.environ["FAKE_HERDR_SPEC"] + ".calls", "a") as f:
f.write(json.dumps([server] + a) + "\n")
if a[:2] == ["session", "list"]:
if spec.get("fail") == "session-list":
sys.exit(1)
print(json.dumps({"sessions": [{"name": n, "running": r} for n, r in spec.get("sessions", {"default": True}).items()]}))
elif a[:2] == ["agent", "list"]:
if spec.get("fail") == "agent-list:" + server:
sys.exit(1)
print(json.dumps({"result": {"agents": spec.get("agents", {}).get(server, [])}}))
elif a[:2] == ["agent", "get"]:
hit = [x for x in spec.get("agents", {}).get(server, []) if a[2] in (x.get("name"), x.get("pane_id"))]
if not hit:
sys.exit(1)
print(json.dumps({"result": {"agent": hit[0]}}))
elif a[:2] == ["pane", "close"]:
spec.setdefault("agents", {})[server] = [x for x in spec.get("agents", {}).get(server, [])
if x.get("pane_id") != a[2]]
json.dump(spec, open(os.environ["FAKE_HERDR_SPEC"], "w"))
'''
STUB_MONITOR = r'''#!/usr/bin/env bash
# Issue #72 test stub of reconcile.sh: a wrapper whose python child holds monitor.lock.
d="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd -P)"
echo "start $$ $*" >> "$d/.mam/stub-monitor.log"
python3 -c '
import fcntl, os, signal, sys, time
if os.environ.get("STUB_MONITOR_IGNORE_TERM") == "1":
signal.signal(signal.SIGTERM, signal.SIG_IGN)
f = open(sys.argv[1], "w")
fcntl.flock(f, fcntl.LOCK_EX)
time.sleep(600)
' "$d/.mam/monitor.lock" &
child=$!
if [ "${STUB_MONITOR_IGNORE_TERM:-0}" = "1" ]; then
trap '' TERM
else
trap 'kill $child 2>/dev/null; exit 143' TERM
fi
while kill -0 "$child" 2>/dev/null; do wait "$child"; done
'''
def _clean_env():
env = {k: v for k, v in os.environ.items()
if not k.startswith(("HERDR_", "XDG_", "MAM_", "STUB_"))
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "DELEGATE_JOB_LOGS_DIR", "PYTHONPATH")}
env.update(MAM_REPO_URL=str(REPO_ROOT), MAM_SKIP_VENV="1",
MAM_INSTALLER_URL="file://" + str(REPO_ROOT / "deploy" / "install.sh"),
MAM_ENV_FILE="/nonexistent/.mam.env", PYTHONDONTWRITEBYTECODE="1")
return env
@pytest.fixture(scope="module")
def template(tmp_path_factory):
ws = tmp_path_factory.mktemp("tpl72") / "ws"
ws.mkdir()
res = subprocess.run(["bash", str(REPO_ROOT / "deploy" / "install.sh"), str(ws)], env=_clean_env(),
capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return ws
class Box:
def __init__(self, root: Path, ws: Path):
self.root, self.ws = root, ws
self.mam = ws / ".mam"
self.bin = root / "bin"
self.spec = root / "herdr.json"
self.env = _clean_env()
self.env["PATH"] = "%s:%s" % (self.bin, os.environ["PATH"])
self.env["FAKE_HERDR_SPEC"] = str(self.spec)
self.helpers = []
self.herdr()
def herdr(self, sessions=None, agents=None, fail=None):
self.spec.write_text(json.dumps({"sessions": sessions or {"default": True},
"agents": agents or {}, "fail": fail}))
def update(self, *args, env=None, timeout=300):
e = dict(self.env, **(env or {}))
return subprocess.run(["bash", str(self.ws / UPDATE_REL), "--yes", *args], cwd=str(self.ws), env=e,
stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=timeout)
def lib(self, script, env=None, timeout=120):
e = dict(self.env, WORKSPACE_ROOT=str(self.ws), **(env or {}))
return subprocess.run(["bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(self.ws)],
env=e, capture_output=True, text=True, timeout=timeout)
def installer(self, body):
p = self.root / "installer.sh"
p.write_text("#!/usr/bin/env bash\nws=\"$1\"\n" + body)
self.env["MAM_INSTALLER_URL"] = "file://" + str(p)
def helper(self, *argv, **kw):
p = subprocess.Popen(list(argv), start_new_session=True, **kw)
self.helpers.append(p)
return p
def stub_monitor(self):
p = self.ws / MONITOR_REL
p.write_text(STUB_MONITOR)
def start_monitor(self, env=None):
"""The way create and resume start it: lib.sh start_watchdog (detached, so
not a child of this test that would linger as a zombie)."""
res = self.lib('start_watchdog test "$WORKSPACE_ROOT"', env=dict(env or {}))
assert res.returncode == 0 and res.stdout.strip(), res.stdout + res.stderr
_until(lambda: _lock_held(self.mam / "monitor.lock"))
def monitor_pids(self):
return _exact_monitor_pids(self.ws / MONITOR_REL)
def decoy(self):
"""A process that only names this workspace's monitor in an argument."""
return self.helper("/bin/bash", "-c", "sleep 120 & wait $!",
"bash %s --subscribe" % (self.ws / MONITOR_REL))
def monitor_starts(self):
p = self.mam / "stub-monitor.log"
return p.read_text().count("start ") if p.exists() else 0
def _until(cond, timeout=15.0):
end = time.monotonic() + timeout
while time.monotonic() < end:
if cond():
return True
time.sleep(0.1)
raise AssertionError("condition not met within %ss" % timeout)
def _exact_monitor_pids(script) -> list:
"""`bash <script> --subscribe ...` processes, judged here independently of
the code under test (a `pgrep -f` would also count a decoy)."""
out = subprocess.run(["ps", "-A", "-ww", "-o", "pid=", "-o", "args="], capture_output=True, text=True).stdout
want = "%s --subscribe" % script
pids = []
for line in out.splitlines():
pid, _, args = line.strip().partition(" ")
argv0, _, rest = args.partition(" ")
if os.path.basename(argv0) == "bash" and (rest == want or rest.startswith(want + " ")):
pids.append(pid)
return pids
def _descendants(pid) -> list:
kids = subprocess.run(["pgrep", "-P", str(pid)], capture_output=True, text=True).stdout.split()
return [d for k in kids for d in [k] + _descendants(k)]
def _processes_naming(path) -> list:
"""Live processes whose command line names PATH (read only, to report leftovers)."""
out = subprocess.run(["ps", "-A", "-ww", "-o", "pid=", "-o", "args="], capture_output=True, text=True).stdout
return [line.strip() for line in out.splitlines()
if str(path) in line and line.split(None, 1)[0] != str(os.getpid())]
def _lock_held(path: Path) -> bool:
if not path.exists():
return False
with open(path, "a") as fh:
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fh, fcntl.LOCK_UN)
return False
except BlockingIOError:
return True
@pytest.fixture
def box(template, tmp_path):
root = Path(os.path.realpath(tmp_path))
ws = root / "ws"
shutil.copytree(template, ws, symlinks=True)
(root / "bin").mkdir()
herdr = root / "bin" / "herdr"
herdr.write_text(FAKE_HERDR)
herdr.chmod(0o755)
b = Box(root, ws)
try:
yield b
finally:
# Only what this test started: its helpers' process groups, and the
# monitors whose exact command line is this test's own workspace
# (detached, so in a group of their own), with their children. No
# pattern kill: the host runs other workspaces' monitors.
for p in b.helpers:
try:
os.killpg(p.pid, signal.SIGKILL)
except OSError:
pass
p.wait()
for pid in _exact_monitor_pids(ws / MONITOR_REL):
for q in [pid] + _descendants(pid):
try:
os.kill(int(q), signal.SIGKILL)
except OSError:
pass
try:
_until(lambda: not _processes_naming(ws), timeout=10)
except AssertionError:
raise AssertionError("processes of this test outlived it: %s" % _processes_naming(ws))
# --- helpers ---------------------------------------------------------------------
SKIP_SNAPSHOT = ("shim/", "update-admission.lock", "agent-sessions.db.lock", "monitor.lock", "stub-monitor.log",
"agent-sessions.db-wal", "agent-sessions.db-shm")
def _db_content(p: Path) -> str:
"""The registry DB by its rows: on macOS SQLite keeps -wal/-shm after close and
a read rewrites -shm, and a checkpoint moves the same rows between -wal and
the file, so neither file's bytes say whether the data changed. A file that
is not a database is compared by its bytes."""
conn = sqlite3.connect(str(p))
try:
return "DB:" + hashlib.sha256("\n".join(conn.iterdump()).encode()).hexdigest()
except sqlite3.DatabaseError:
return hashlib.sha256(p.read_bytes()).hexdigest()
finally:
conn.close()
def snapshot(ws: Path) -> dict:
out = {}
for dp, dns, fns in os.walk(ws):
dns[:] = [d for d in dns if d != "__pycache__"]
for f in fns:
p = Path(dp) / f
rel = str(p.relative_to(ws))
if rel.startswith(".mam/") and any(s in rel for s in SKIP_SNAPSHOT):
continue
if rel == ".mam/agent-sessions.db":
out[rel] = _db_content(p)
continue
out[rel] = "L:" + os.readlink(p) if p.is_symlink() else hashlib.sha256(p.read_bytes()).hexdigest()
return out
def write_rows(ws: Path, rows, **top):
d = dict(top, herdr_sessions=rows)
(ws / ".mam" / "agent-sessions.yaml").write_text(yaml.safe_dump(d, sort_keys=False))
def row(name, status="running", server="default", cwd=None, **extra):
r = {"name": name, "status": status, "role": "creator", "herdr_session": server, "herdr_server": server,
"pane": {"cwd": cwd or "/elsewhere", "cmd": "claude"}}
r.update(extra)
return r
def write_job(d: Path, jid, status="running", session="herdr:x", **extra):
d.mkdir(parents=True, exist_ok=True)
rec = dict({"job_id": jid, "status": status, "agent_session": session}, **extra)
(d / (jid + ".json")).write_text(json.dumps(rec))
def live_marker(path: Path, box: Box):
"""A marker owned by a live process (pid + lstart as loop_lock.sh writes them)."""
p = box.helper("sleep", "120")
ls = subprocess.run(["ps", "-p", str(p.pid), "-o", "lstart="], capture_output=True, text=True).stdout
ls = " ".join(ls.split())
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("pid=%d\nlstart=%s\nstarted=x\n" % (p.pid, ls))
return p
def dead_marker(path: Path):
p = subprocess.Popen(["true"])
p.wait()
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("pid=%d\nlstart=Thu Jan 1 00:00:00 1970\nstarted=x\n" % p.pid)
def assert_refused_unchanged(box: Box, before: dict, res, *needles):
assert res.returncode == 1, res.stdout + res.stderr
assert "update refused; nothing was changed" in res.stderr, res.stderr
for n in needles:
assert n in res.stderr, (n, res.stderr)
assert snapshot(box.ws) == before
assert not (box.mam / "update-in-progress").exists(), "the refusing update releases its marker"
assert not (box.mam / "update-backup").exists()
# --- success and the #8 state rules ------------------------------------------------
def test_update_in_place_keeps_mam_and_merges_install_state(box):
mam = box.mam
(mam / "version.txt").write_text("source=old\ncommit=stale0000\nfetched_at=then\nmethod=git\n")
(mam / "herdr_workspace_id").write_text("w9\n")
for d in ("briefs", "diagnostics", "resumelogs", "stoplogs", "jobs", "delegate_job_logs/j1"):
(mam / d).mkdir(parents=True, exist_ok=True)
(mam / d / "keep.txt").write_text(d)
write_job(mam / "jobs", "done1", status="completed")
(mam / "multi-agent-mux-monitor.log").write_text("log\n")
(box.ws / ".mam.env").write_text("MQTT_BROKER=example\n")
with open(mam / "install_state", "a") as fh:
fh.write("custom_marker=kept\n")
write_rows(box.ws, [row("a", status="stopped")], orchestrator_uuids=["u1"])
keep = {k: v for k, v in snapshot(box.ws).items()
if k.startswith((".mam/herdr_workspace_id", ".mam/briefs", ".mam/diagnostics", ".mam/resumelogs",
".mam/stoplogs", ".mam/jobs", ".mam/delegate_job_logs", ".mam/multi-agent-mux-monitor.log"))
or k == ".mam.env"}
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
after = snapshot(box.ws)
assert {k: after.get(k) for k in keep} == keep, "everything that is not the installer's is kept"
assert "stale0000" not in (mam / "version.txt").read_text()
state = (mam / "install_state").read_text().splitlines()
assert "custom_marker=kept" in state and state.count("gitignore_created=1") == 1, state
assert not (box.ws / ".mam_deploy" / "update.sh").exists()
assert not (mam / "update-backup").exists() and not (mam / "update-in-progress").exists()
assert "Update complete." in res.stdout
def test_old_format_registry_is_converted_by_the_new_code(box):
old = row("old-one", status="stopped", start_command="HERDR_SESSION_NAME=x herdr new-session",
kill_command="HERDR_SESSION_NAME=x herdr kill-session", attach_command="HERDR_SESSION_NAME=x herdr agent attach old-one")
write_rows(box.ws, [old])
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry-format: 1 -> 2, rows changed: 1" in res.stdout
y = yaml.safe_load((box.mam / "agent-sessions.yaml").read_text())
assert y["registry_format"] == 2
conn = sqlite3.connect(str(box.mam / "agent-sessions.db"))
try:
db = {r["name"]: r for r in (json.loads(x[0]) for x in conn.execute("SELECT data FROM sessions"))}
st = json.loads(conn.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
finally:
conn.close()
assert {r["name"]: r for r in y["herdr_sessions"]} == db and st["registry_format"] == 2
assert "kill_command" not in db["old-one"] and "start_command" not in db["old-one"]
def test_no_registry_is_nothing_to_convert(box):
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry: nothing to convert (no registry)" in res.stdout
assert not (box.mam / "agent-sessions.yaml").exists()
def test_a_failed_conversion_keeps_the_install_and_exits_nonzero(box, tmp_path):
src = tmp_path / "src"
shutil.copytree(REPO_ROOT, src, symlinks=True,
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", "__pycache__", "tests", "nats-docker"))
fmt = src / ".agents" / "skills" / "lib_py" / "registry_format.py"
fmt.write_text("import sys\nif __name__ == '__main__':\n sys.exit(3)\n" + fmt.read_text())
box.env["MAM_REPO_URL"] = str(src)
box.stub_monitor()
box.start_monitor()
write_rows(box.ws, [row("a", status="stopped")])
res = box.update()
assert res.returncode == 3, res.stdout + res.stderr
assert "the registry was not converted" in res.stderr
assert (box.ws / ".agents/skills/lib_py/registry_format.py").read_text() == fmt.read_text(), "install kept"
assert box.monitor_pids() and box.monitor_starts() == 2, "the monitor is started again anyway"
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
def test_refuses_outside_an_install_and_without_yes(box):
(box.ws / ".mam_deploy" / "remove.sh").unlink()
res = box.update()
assert res.returncode == 1 and "not an installed MAM workspace" in res.stderr
shutil.copy(REPO_ROOT / "deploy" / "remove.sh", box.ws / ".mam_deploy" / "remove.sh")
res = subprocess.run(["bash", str(box.ws / UPDATE_REL)], env=box.env, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=60)
assert res.returncode == 1 and "-y/--yes" in res.stderr
res = box.update("--force")
assert res.returncode == 2, "--force is not an alias of --yes"
# --- refusals (decision 1) ---------------------------------------------------------
def test_refuses_a_running_row(box):
write_rows(box.ws, [row("busy-creator", cwd="/somewhere/else")])
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "session busy-creator is running", "stop_session.sh --session busy-creator")
def test_refuses_an_unreadable_registry(box):
(box.mam / "agent-sessions.db").write_text("not a database")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "cannot be read")
def test_refuses_a_rowless_agent_on_any_running_server(box):
box.herdr(sessions={"default": True, "picked-by-hand": True, "down": False},
agents={"picked-by-hand": [{"name": "stray", "pane_id": "w3:p1", "cwd": str(box.ws / "sub")}],
"down": [{"name": "ghost", "pane_id": "w4:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "herdr agent stray (pane w3:p1, server picked-by-hand)")
assert "ghost" not in res.stderr, "a server that is not running is not asked"
calls = (box.root / "herdr.json.calls").read_text()
assert '"down"' not in calls
def test_the_rowless_agent_hint_stops_it_on_the_server_it_was_found_on(box):
"""R3: the printed stop command names the server; without it stop looks on
the workspace's server, where a same-named agent runs outside the workspace."""
write_rows(box.ws, [])
box.herdr(sessions={"default": True, "unrecorded-server": True},
agents={"unrecorded-server": [{"name": "stray", "pane_id": "w3:p1", "cwd": str(box.ws / "sub")}],
"default": [{"name": "stray", "pane_id": "w1:p1", "cwd": str(box.root / "elsewhere")}]})
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
line = next(ln for ln in res.stderr.splitlines() if "herdr agent stray" in ln)
cmd = line.split("stop it: ", 1)[1]
assert "--herdr-session unrecorded-server" in cmd, cmd
stop = subprocess.run(["bash", "-c", cmd], cwd=str(box.ws), env=box.env, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=120)
assert stop.returncode == 0, stop.stdout + stop.stderr
agents = json.loads(box.spec.read_text())["agents"]
assert agents["unrecorded-server"] == []
assert [x["pane_id"] for x in agents["default"]] == ["w1:p1"], "the same name elsewhere is left alone"
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
def test_an_agent_outside_the_workspace_does_not_refuse(box):
box.herdr(agents={"default": [{"name": "other", "pane_id": "w1:p2", "cwd": str(box.root / "elsewhere")}]})
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
@pytest.mark.parametrize("fail", ["session-list", "agent-list:default"])
def test_refuses_when_herdr_cannot_answer(box, fail):
box.herdr(fail=fail)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "cannot list the live agents")
def test_open_jobs_refuse_and_terminal_or_dry_run_do_not(box, tmp_path):
jobs = box.mam / "jobs"
write_job(jobs, "done1", status="completed")
write_job(jobs, "err1", status="error")
write_job(jobs, "can1", status="cancelled")
write_job(jobs, "dry1", status="pending", dry_run=True)
assert box.update().returncode == 0
write_job(jobs, "open1", status="pending", session="remote:x")
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "job open1", "close --job open1 --request cancelled")
(jobs / "open1.json").unlink()
(jobs / "broken.json").write_text("{not json")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job record %s cannot be read" % (jobs / "broken.json"))
(jobs / "broken.json").unlink()
extra = tmp_path / "mam-jobs-dir"
write_job(extra, "env1", status="running")
box.env["MAM_JOBS_DIR"] = str(extra)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job env1")
del box.env["MAM_JOBS_DIR"]
@pytest.mark.skipif(os.geteuid() == 0, reason="root reads a mode-000 directory")
def test_a_jobs_directory_that_cannot_be_listed_refuses(box, tmp_path):
"""R2: an unreadable .mam/jobs (or MAM_JOBS_DIR under an unreadable parent)
is not an empty registry."""
jobs = box.mam / "jobs"
write_job(jobs, "hidden1", status="running")
before = snapshot(box.ws)
jobs.chmod(0)
try:
res = box.update()
finally:
jobs.chmod(0o755)
assert_refused_unchanged(box, before, res, "jobs directory %s cannot be listed" % jobs)
(jobs / "hidden1.json").unlink()
parent = tmp_path / "locked"
write_job(parent / "jobs", "env1", status="running")
box.env["MAM_JOBS_DIR"] = str(parent / "jobs")
before = snapshot(box.ws)
parent.chmod(0)
try:
res = box.update()
finally:
parent.chmod(0o755)
assert_refused_unchanged(box, before, res, "jobs directory %s cannot be listed" % (parent / "jobs"))
del box.env["MAM_JOBS_DIR"]
def test_other_control_planes_count_only_jobs_addressed_to_this_registry(box, tmp_path):
other = tmp_path / "other-plane" / "jobs"
write_job(other, "theirs", status="running", session="herdr:not-ours")
write_rows(box.ws, [row("wt-creator", status="stopped", control_plane={"mam_jobs_dir": str(other)})])
assert box.update().returncode == 0, "a job of another registry's session is not ours"
write_job(other, "ours", status="running", session="herdr:wt-creator")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job ours")
def test_loop_marker_live_refuses_stale_does_not(box):
live_marker(box.mam / "loop-guard-active", box)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "a review loop is running")
dead_marker(box.mam / "loop-guard-active")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "stale loop marker" in res.stdout
# --- the calling agent (decision 7, N2) ----------------------------------------------
def _me_env(server="mam-x", pane="w5:p1"):
return {"HERDR_SESSION": server, "HERDR_PANE_ID": pane}
def test_the_calling_agent_its_row_and_its_running_job_are_excluded(box):
me = {"name": "me-creator", "pane_id": "w5:p1", "cwd": str(box.ws)}
box.herdr(sessions={"default": True, "mam-x": True}, agents={"mam-x": [me]})
write_rows(box.ws, [row("me-creator", server="mam-x", cwd=str(box.ws))])
write_job(box.mam / "jobs", "myjob", status="running", session="herdr:me-creator")
res = box.update(env=_me_env())
assert res.returncode == 0, res.stdout + res.stderr
write_job(box.mam / "jobs", "myjob2", status="pending", session="herdr:me-creator")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job myjob2")
def test_same_pane_id_on_another_server_is_not_excluded(box):
box.herdr(sessions={"default": True, "mam-x": True, "mam-y": True},
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}],
"mam-y": [{"name": "twin", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update(env=_me_env())
assert_refused_unchanged(box, before, res, "herdr agent twin (pane w5:p1, server mam-y)")
assert "agent me " not in res.stderr
def test_a_nameless_caller_pane_is_excluded_by_pane_only(box):
box.herdr(sessions={"default": True, "mam-x": True},
agents={"mam-x": [{"name": None, "pane_id": "w5:p1", "cwd": str(box.ws)}]})
assert box.update(env=_me_env()).returncode == 0
write_job(box.mam / "jobs", "nameless", status="running", session="herdr:")
write_job(box.mam / "jobs", "nonename", status="running", session="herdr:None")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job nameless", "job nonename")
@pytest.mark.parametrize("env", [{}, {"HERDR_PANE_ID": "w5:p1"}, {"HERDR_SESSION": "mam-x"}])
def test_nothing_is_excluded_when_the_caller_is_not_identified(box, env):
box.herdr(sessions={"default": True, "mam-x": True},
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update(env=env)
assert_refused_unchanged(box, before, res, "herdr agent me", "nothing was excluded as the calling agent")
# --- the monitor (decision 2, U3) ----------------------------------------------------
def test_a_running_monitor_is_stopped_and_started_again(box):
box.stub_monitor()
box.start_monitor()
first = set(box.monitor_pids())
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "monitor: stopping the daemon" in res.stdout and "monitor: restarted" in res.stdout
now = set(box.monitor_pids())
assert now and not (now & first) and box.monitor_starts() == 2
def test_no_monitor_before_means_none_after(box):
box.stub_monitor()
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert not box.monitor_pids() and box.monitor_starts() == 0
def test_no_subscriber_daemon_flag_wins(box):
box.stub_monitor()
box.start_monitor()
res = box.update(env={"MAM_NO_SUBSCRIBER_DAEMON": "1"})
assert res.returncode == 0, res.stdout + res.stderr
assert "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)" in res.stdout
assert not box.monitor_pids()
def test_a_monitor_that_ignores_term_is_killed_restarted_and_nothing_installed(box):
box.stub_monitor()
box.start_monitor(env={"STUB_MONITOR_IGNORE_TERM": "1"})
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
before = snapshot(box.ws)
res = box.update(env={"MAM_UPDATE_MONITOR_STOP_S": "1"})
assert res.returncode == 1, res.stdout + res.stderr
assert "it was killed and is started again" in res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert snapshot(box.ws) == before
assert box.monitor_pids() and box.monitor_starts() == 2
assert not (box.mam / "update-in-progress").exists()
def test_a_monitor_lock_held_by_an_unknown_process_refuses(box):
lock = box.mam / "monitor.lock"
with open(lock, "w") as fh:
fcntl.flock(fh, fcntl.LOCK_EX)
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "monitor.lock is held by a monitor the skills did not start")
@pytest.mark.parametrize("finder", ["python", "shell"])
def test_only_this_workspaces_exact_monitor_command_line_is_its_monitor(tmp_path, finder):
"""The update signals what `lib_py.update monitor` returns, and the legacy
start_watchdog (MAM_DELEGATION_PY=0) reuses what lib.sh _monitor_pids finds:
`bash <this workspace's reconcile.sh> --subscribe ...` only - not another
workspace's daemon the path matched as a regex (ws.a, wsXa), nor a process
that names the path in an argument. A path with a space is still found."""
root = Path(os.path.realpath(tmp_path))
procs = []
def daemon_in(ws):
script = ws / MONITOR_REL
script.parent.mkdir(parents=True)
script.write_text("sleep 120 & wait $!\n")
procs.append(subprocess.Popen(["bash", str(script), "--subscribe", "--idle-timeout", "0"],
start_new_session=True))
return str(procs[-1].pid)
def monitor(ws):
if finder == "shell":
res = subprocess.run(["bash", "-c", 'source "$1" && _monitor_pids "$2"', "_",
str(REPO_ROOT / ".agents" / "skills" / "lib.sh"), str(ws / MONITOR_REL)],
env=dict(_clean_env(), WORKSPACE_ROOT=str(root)), capture_output=True, text=True)
assert res.returncode == 0, res.stderr
return res.stdout.split()
res = subprocess.run([sys.executable, "-m", "lib_py.update", "monitor", "--workspace", str(ws)],
cwd=str(REPO_ROOT / ".agents" / "skills"), capture_output=True, text=True)
assert res.returncode == 0, res.stderr
return json.loads(res.stdout)["pids"]
try:
a, other, spaced = root / "ws.a", root / "wsXa", root / "w s"
a.mkdir()
daemon_in(other)
procs.append(subprocess.Popen(["/bin/bash", "-c", "sleep 120 & wait $!",
"bash %s --subscribe" % (a / MONITOR_REL)], start_new_session=True))
pid = daemon_in(spaced)
_until(lambda: monitor(spaced) == [pid])
assert monitor(a) == []
finally:
for p in procs:
try:
os.killpg(p.pid, signal.SIGKILL)
except OSError:
pass
p.wait()
@pytest.mark.parametrize("case", ["installed", "killed", "rolled-back"])
@pytest.mark.parametrize("mode", ["0", "1"])
def test_every_restart_starts_a_real_monitor_in_both_delegation_modes(box, mode, case):
"""MAM_DELEGATION_PY=0 keeps the legacy start_watchdog, the switch the
delegate skill documents. Its restart, after an install, after a stop
timeout or after a rollback, must start this workspace's monitor again, out
of the caller's process group (decision 2), and not report a process that
only names the monitor path in an argument."""
env = {"MAM_DELEGATION_PY": mode}
box.stub_monitor()
box.start_monitor(env=dict(env, STUB_MONITOR_IGNORE_TERM="1") if case == "killed" else env)
first = set(box.monitor_pids())
decoy = box.decoy()
_until(lambda: str(decoy.pid) in subprocess.run(["ps", "-A", "-ww", "-o", "pid=", "-o", "args="],
capture_output=True, text=True).stdout)
if case == "killed":
env["MAM_UPDATE_MONITOR_STOP_S"] = "1"
if case == "rolled-back":
box.installer("exit 7\n")
res = box.update(env=env)
assert res.returncode == (0 if case == "installed" else 1), res.stdout + res.stderr
assert {"installed": "monitor: restarted", "killed": "it was killed and is started again",
"rolled-back": "rolled back"}[case] in res.stdout + res.stderr
assert "PID %s)" % decoy.pid not in res.stdout, "the decoy is reported as the monitor"
_until(lambda: bool(box.monitor_pids()))
assert not (set(box.monitor_pids()) & first) and box.monitor_starts() == 2
# box.update runs update.sh in this test's process group: the caller's.
assert all(os.getpgid(int(p)) != os.getpgrp() for p in box.monitor_pids()), "left in the caller's group"
assert decoy.poll() is None, "the decoy was not signalled"
# --- rollback (decisions 3, 10; U4) --------------------------------------------------
FAILING_INSTALL = """
echo broken >> "$ws/.agents/skills/lib.sh"
rm -f "$ws/.mam_deploy/remove.sh"
echo new > "$ws/MESSAGING.md"
mkdir -p "$ws/.venv/lib" && echo junk > "$ws/.venv/lib/junk"
echo junk > "$ws/.mam/version.txt"
echo junk > "$ws/.mam/asset_hashes.txt"
mkdir -p "$ws/.agents/skills/half-installed" && echo x > "$ws/.agents/skills/half-installed/f"
python3 -c 'import yaml,sys; p=sys.argv[1]; d=yaml.safe_load(open(p)); d["herdr_sessions"]=[]; yaml.safe_dump(d, open(p, "w"))' "$ws/.mam/agent-sessions.yaml"
exit 7
"""
def test_a_failed_install_rolls_everything_back(box):
write_rows(box.ws, [row("a", status="stopped")])
db = box.mam / "agent-sessions.db"
conn = sqlite3.connect(str(db))
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA wal_autocheckpoint=0")
conn.execute("CREATE TABLE state (id INTEGER PRIMARY KEY, data TEXT)")
conn.execute("CREATE TABLE sessions (name TEXT PRIMARY KEY, status TEXT, pane_cwd TEXT, data JSON)")
conn.execute("INSERT INTO state VALUES (1, ?)", (json.dumps({"orchestrator_uuids": ["in-wal"]}),))
conn.execute("INSERT INTO sessions VALUES ('a', 'stopped', '/x', ?)", (json.dumps(row("a", status="stopped")),))
conn.commit()
assert (box.mam / "agent-sessions.db-wal").stat().st_size > 0, "the commit is still in the WAL"
box.stub_monitor()
box.start_monitor()
box.installer(FAILING_INSTALL)
before = snapshot(box.ws)
try:
res = box.update()
finally:
conn.close()
assert res.returncode == 1, res.stdout + res.stderr
assert "the install failed (rc 7); rolling back" in res.stderr and "rolled back" in res.stdout
after = snapshot(box.ws)
after.pop(".mam/agent-sessions.db", None)
before.pop(".mam/agent-sessions.db", None)
before.pop(".mam/agent-sessions.db-wal", None)
before.pop(".mam/agent-sessions.db-shm", None)
assert after == before, sorted(set(after.items()) ^ set(before.items()))[:10]
assert not (box.ws / "MESSAGING.md").exists() and not (box.ws / ".venv").exists()
check = sqlite3.connect(str(db))
try:
st = json.loads(check.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
finally:
check.close()
assert st == {"orchestrator_uuids": ["in-wal"]}, "the backup carried the WAL's commit"
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
assert box.monitor_pids() and box.monitor_starts() == 2, "the restored tree starts the monitor again"
box.env["MAM_INSTALLER_URL"] = "file://" + str(REPO_ROOT / "deploy" / "install.sh")
res = box.update()
assert res.returncode == 0, "the skill runs again after a rollback: " + res.stdout + res.stderr
# --- leftovers (U4) ------------------------------------------------------------------
def _leftover(box: Box, state):
bk = box.mam / "update-backup"
(bk / "files" / ".agents" / "skills").mkdir(parents=True)
shutil.copy2(box.ws / ".agents/skills/lib.sh", bk / "files/.agents/skills/lib.sh")
(bk / "items").write_text(".agents/skills/lib.sh\nMESSAGING.md\n")
(bk / "absent").write_text("MESSAGING.md\n")
(bk / "registry").mkdir()
(bk / "registry" / "agent-sessions.yaml").write_text("herdr_sessions: []\n")
if state is not None:
(bk / "state").write_text(state + "\n")
return bk
@pytest.mark.parametrize("state", [None, "backing-up", "installed"])
def test_leftover_that_changed_nothing_or_finished_is_removed(box, state):
_leftover(box, state)
(box.ws / "MESSAGING.md").write_text("must survive: nothing is restored\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "removing the backup a previous update left" in res.stdout
assert (box.ws / "MESSAGING.md").exists()
assert not (box.mam / "update-backup").exists()
def test_leftover_installing_restores_files_but_not_the_registry(box):
_leftover(box, "installing")
pristine = (box.ws / ".agents/skills/lib.sh").read_bytes()
with open(box.ws / ".agents/skills/lib.sh", "a") as fh:
fh.write("# half installed\n")
(box.ws / "MESSAGING.md").write_text("made by the dead install\n")
write_rows(box.ws, [row("written-after-crash", status="stopped")])
box.installer("grep -q 'half installed' \"$ws/.agents/skills/lib.sh\" && exit 9\n"
"test -e \"$ws/MESSAGING.md\" && exit 8\n"
"cp \"$ws/.mam/agent-sessions.yaml\" \"$ws/../seen-registry.yaml\"\nexit 0\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "the previous update was rolled back" in res.stdout
assert (box.ws / ".agents/skills/lib.sh").read_bytes() == pristine
assert "written-after-crash" in (box.root / "seen-registry.yaml").read_text(), "registry left as it was"
assert not (box.mam / "update-backup").exists()
def test_leftover_whose_restore_fails_is_kept_and_nothing_more_changes(box):
bk = _leftover(box, "installing")
(bk / "items").write_text(".agents/skills/lib.sh\n.agents/skills/missing-from-backup\n")
box.stub_monitor()
box.start_monitor()
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
assert "restoring the previous update's backup failed" in res.stderr
assert bk.exists() and (bk / "state").read_text().strip() == "installing"
assert not (box.ws / "INSTALLER-RAN").exists()
assert not box.monitor_pids(), "the monitor stays stopped"
assert not (box.mam / "update-in-progress").exists()
def test_a_state_write_failure_before_the_install_changes_nothing_and_restarts_the_monitor(box):
real_mv = shutil.which("mv")
fake = box.bin / "mv"
fake.write_text("#!/usr/bin/env bash\n"
"src=\"${@: -2:1}\"\n"
"case \"$src\" in */update-backup/state.tmp) grep -qx installing \"$src\" && exit 1 ;; esac\n"
"exec %s \"$@\"\n" % real_mv)
fake.chmod(0o755)
box.stub_monitor()
box.start_monitor()
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert box.monitor_starts() == 2, "the monitor it stopped is started again"
_until(lambda: bool(box.monitor_pids()))
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
assert "nothing was changed" in res.stderr
def test_the_backup_lives_in_the_workspace_and_is_gone_after_success(box):
box.installer("test -f \"$ws/.mam/update-backup/state\" && grep -qx installing \"$ws/.mam/update-backup/state\" "
"&& test -f \"$ws/.mam/update-backup/files/.agents/skills/lib.sh\" && touch \"$ws/../saw-backup\"\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert (box.root / "saw-backup").exists()
assert not (box.mam / "update-backup").exists()
# --- the marker and the admission lock (decision 8) -----------------------------------
def test_a_live_update_marker_refuses_a_second_update_and_is_left_alone(box):
live_marker(box.mam / "update-in-progress", box)
text = (box.mam / "update-in-progress").read_text()
before = snapshot(box.ws)
res = box.update()
assert res.returncode == 1 and "another update is in progress" in res.stderr, res.stderr
assert (box.mam / "update-in-progress").read_text() == text
assert snapshot(box.ws) == before
def test_a_stale_update_marker_is_taken_over(box):
dead_marker(box.mam / "update-in-progress")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "stale update marker" in res.stdout
assert not (box.mam / "update-in-progress").exists()
def test_two_updates_at_once_one_proceeds(box):
box.installer("touch \"$ws/../first-installing\"; sleep 4; exit 0\n")
first = box.helper("bash", str(box.ws / UPDATE_REL), "--yes", cwd=str(box.ws), env=box.env,
stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
_until(lambda: (box.root / "first-installing").exists())
marker = (box.mam / "update-in-progress").read_text()
second = box.update()
assert second.returncode == 1 and "another update is in progress" in second.stderr, second.stderr
assert (box.mam / "update-in-progress").read_text() == marker, "the running update's marker is untouched"
out, err = first.communicate(timeout=120)
assert first.returncode == 0, out + err
def test_entries_are_refused_while_the_copy_installs(box):
box.installer('if bash -c \'source "$1/.agents/skills/lib.sh" && mam_update_admit\' _ "$ws" 2> "$ws/../admit.err"; '
'then echo admitted > "$ws/../admit"; else echo refused > "$ws/../admit"; fi\n')
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert (box.root / "admit").read_text().strip() == "refused"
assert "an update of %s is in progress" % box.ws in (box.root / "admit.err").read_text()
@pytest.mark.parametrize("wait,needle", [("30", "session late-row is running"), ("1", "admission lock past")])
def test_an_entry_holding_the_admission_lock_is_waited_for_or_refuses(box, wait, needle):
script = ('mam_update_admit || exit 1\necho admitted\nsleep 3\n'
'printf "herdr_sessions:\\n- name: late-row\\n status: running\\n" > "$WORKSPACE_ROOT/.mam/agent-sessions.yaml"\n'
'mam_update_admit_release\nsleep 30\n')
entry = box.helper("bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(box.ws),
env=dict(box.env, WORKSPACE_ROOT=str(box.ws)), stdout=subprocess.PIPE, text=True)
assert entry.stdout.readline().strip() == "admitted"
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update(env={"MAM_UPDATE_ADMISSION_WAIT_S": wait})
assert res.returncode == 1, res.stdout + res.stderr
assert needle in res.stderr, res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert not (box.mam / "update-in-progress").exists()
# --- the first transition (decision 4, criterion 5) ------------------------------------
def test_the_old_updater_installs_the_skill_which_then_converts_the_registry(tmp_path):
"""A workspace installed by v4.11.0 still has .mam_deploy/update.sh. Run once,
it installs the new version - the skill, and no updater left behind - and
copies the registry unconverted; the skill run right after converts it."""
old_src = tmp_path / "v4.11.0"
old_src.mkdir()
arch = subprocess.run(["git", "-C", str(REPO_ROOT), "archive", "0e1e2e8"], capture_output=True)
if arch.returncode != 0:
pytest.skip("no git history to rebuild v4.11.0 from")
subprocess.run(["tar", "-x", "-C", str(old_src)], input=arch.stdout, check=True)
root = Path(os.path.realpath(tmp_path))
ws = root / "ws"
ws.mkdir()
(root / "bin").mkdir()
(root / "bin" / "herdr").write_text(FAKE_HERDR)
(root / "bin" / "herdr").chmod(0o755)
b = Box(root, ws)
res = subprocess.run(["bash", str(old_src / "deploy" / "install.sh"), str(ws)],
env=dict(b.env, MAM_REPO_URL=str(old_src)), capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
assert (ws / ".mam_deploy" / "update.sh").exists()
write_rows(ws, [row("old-one", status="stopped", kill_command="x", start_command="y")])
res = subprocess.run(["bash", str(ws / ".mam_deploy" / "update.sh"), "--force"], cwd=str(ws), env=b.env,
capture_output=True, text=True, timeout=600, stdin=subprocess.DEVNULL)
assert res.returncode == 0, res.stdout + res.stderr
assert (ws / UPDATE_REL).exists() and not (ws / ".mam_deploy" / "update.sh").exists()
assert "registry_format" not in yaml.safe_load((ws / ".mam" / "agent-sessions.yaml").read_text())
res = b.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry-format: 1 -> 2, rows changed: 1" in res.stdout
assert yaml.safe_load((ws / ".mam" / "agent-sessions.yaml").read_text())["registry_format"] == 2
+18 -4
View File
@@ -1,6 +1,8 @@
#!/usr/bin/env python3
"""Issue #8 — update.sh must keep the installer's fresh version.txt and
asset_hashes.txt instead of restoring the pre-update copies.
"""Issue #8 — an update must keep the installer's fresh version.txt and
asset_hashes.txt instead of restoring the pre-update copies. Since Issue #72
the update is the multi-agent-mux-update skill (in place, no remove.sh) and
no .mam_deploy/update.sh is installed.
Restoring them left version.txt naming the previous commit and made remove.sh
compare against stale hashes, so every updated-but-pristine file was backed up
@@ -51,8 +53,19 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
with open(mam / "install_state", "a") as fh:
fh.write("custom_marker=kept\n")
res = subprocess.run(["bash", str(ws / ".mam_deploy" / "update.sh"), "--force"],
cwd=str(ws), env=env, capture_output=True, text=True, timeout=600)
assert not (ws / ".mam_deploy" / "update.sh").exists()
upd_env = {k: v for k, v in env.items() if not k.startswith("HERDR_")
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
upd_env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
# The update lists the live herdr agents; never ask a real server here.
fake = ws.parent / "fake-herdr-bin"
fake.mkdir()
(fake / "herdr").write_text('#!/bin/sh\necho \'{"sessions": []}\'\n')
(fake / "herdr").chmod(0o755)
upd_env["PATH"] = "%s:%s" % (fake, upd_env["PATH"])
res = subprocess.run(["bash", str(ws / ".agents/skills/multi-agent-mux-update/scripts/update.sh"), "--yes"],
cwd=str(ws), env=upd_env, capture_output=True, text=True, timeout=600,
stdin=subprocess.DEVNULL)
assert res.returncode == 0, res.stderr + res.stdout
version = (mam / "version.txt").read_text()
@@ -64,3 +77,4 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
state = (mam / "install_state").read_text()
assert "custom_marker=kept" in state, state
assert state.count("gitignore_created=1") == 1, "merged keys must not be duplicated:\n" + state
assert not (ws / ".mam_deploy" / "update.sh").exists()
+3 -2
View File
@@ -27,6 +27,7 @@ SKILL_NAMES = [
"multi-agent-mux-delegate-job",
"multi-agent-mux-loop",
"multi-agent-mux-orc-onboard",
"multi-agent-mux-update",
]
@@ -75,7 +76,7 @@ def _get_skill_md_frontmatter_version(skill_name: str):
def test_three_way_version_lockstep():
"""T-4: lib.sh(MAM_VERSION) == VERSIONS.md(Current + Matrix 8 items) == 8x SKILL.md frontmatters."""
"""T-4: lib.sh(MAM_VERSION) == VERSIONS.md(Current + Matrix 9 items) == 9x SKILL.md frontmatters."""
# 1. lib.sh runtime constant
lib_ver = _get_lib_sh_mam_version()
assert lib_ver != "", "MAM_VERSION in lib.sh is empty"
@@ -88,7 +89,7 @@ def test_three_way_version_lockstep():
assert skill in vmd_matrix, f"Skill {skill} missing from VERSIONS.md matrix"
assert vmd_matrix[skill] == lib_ver, f"VERSIONS.md matrix for {skill} ({vmd_matrix[skill]}) != lib.sh ({lib_ver})"
# 3. 8x SKILL.md files
# 3. 9x SKILL.md files
for skill in SKILL_NAMES:
fm_ver = _get_skill_md_frontmatter_version(skill)
assert fm_ver == lib_ver, f"{skill}/SKILL.md version ({fm_ver}) != lib.sh ({lib_ver})"