Compare commits

...
12 Commits
Author SHA1 Message Date
GodopuandClaude Opus 5.5 454b77215f fix(update): a state write failure before the install restarts the monitor; the exit table names the conversion code (Issue #72, review notes)
install_phase.sh exited when it could not mark the backup `installing`,
before the installer ran, without starting the monitor phase 1 had stopped
(decision 2) and leaving the backup behind. Nothing was changed at that
point, so it now removes the backup, restarts the monitor if it ran,
releases the marker and exits 1.

SKILL.md's exit row said 1 for every failure; a failed registry conversion
after a good install passes the converter's nonzero code through, and a
cancelled prompt exits 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 14:35:35 +00:00
GodopuandClaude Opus 5.5 253a0aa56e fix(delegate): admission covers a rowless target and the installed parent of an incidental .mam (Issue #72, review round 2)
R1: a target with no registry row skipped the target gate. Its cwd is now
read from the live agent on the server delivery uses (resolve_herdr_session),
through the real herdr only when that server is running, so no server is
started; a herdr that cannot answer refuses before registration.

R2: any .mam counted as a workspace and stopped the walk, so B/sub/.mam hid
an installed B and the row's control_plane root was dropped. Workspaces are
now update.sh's installed trees (.mam_deploy/remove.sh or remove.sh): every
installed ancestor of the cwd, plus the recorded control_plane root.

lib_py.update target-workspaces prints them one per line; the wrapper holds
each on fds 20.. until the record is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 11:09:30 +00:00
GodopuandClaude Opus 5.5 ba75718ac3 fix(update): an unlistable jobs directory refuses; the rowless-agent hint names its server (Issue #72, review R2, R3)
R2: glob() hid a listing error, so an unreadable .mam/jobs (or MAM_JOBS_DIR,
or another row's jobs dir) counted as empty and the update installed over an
open job. Records are now listed with os.listdir; a directory that cannot be
listed, or stat'ed, is a blocker with its path.

R3: a discovered agent with no row was told to run stop with --session only;
stop then looked on the workspace's server, not the one the agent was found
on. The hint now carries --herdr-session <server>, quoted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 08:15:23 +00:00
GodopuandClaude Opus 5.5 e3394655b3 fix(delegate): submit also takes the target workspace's update admission (Issue #72, review R1)
Decision 8: delegate submit uses the admission lock of the workspace the
target session belongs to. The wrapper resolved only its own
WORKSPACE_ROOT, so a job could be registered and delivered to an agent of
another installed workspace while that workspace was being updated.

lib_py.update target-workspace resolves a herdr:<name> row's workspace (the
.mam at or above its pane.cwd, else its control_plane.workspace_root);
mam_update_admit takes an fd, and submit holds the caller root on fd 7 and
the target on fd 8 until the record is written. Rowless and remote targets
keep the caller-root gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 08:15:23 +00:00
GodopuandClaude Opus 5.5 595e34b2d1 fix(gate): no admission lock in a tree without .mam; MAM_UPDATE_MONITOR_STOP_S in the knob table (Issue #72)
mam_update_admit created .mam/ to hold its lock, so a checkout running its
own run_loop.sh got a .mam (test_loop_marker_isolation L1, test_issue17 W1).
A tree without .mam is not an installed workspace, the only kind the update
runs in, so the gate admits there without a lock. The update's monitor-stop
wait is a normalised integer knob and is listed in INT_KNOBS
(test_issue26_ints B5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:48:43 +00:00
GodopuandClaude Opus 5.5 8f325de979 docs(resume): the flow ends with the monitor daemon (Issue #72)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:16:38 +00:00
GodopuandClaude Opus 5.5 515b3206b9 chore(shellcheck): no new findings in the update skill, lib.sh and install.sh (Issue #72)
CI now shellchecks the update skill's scripts: SC1091 is disabled at their
`source` lines as in the other skills, and the two findings this branch added
(lib.sh's loop_lock.sh source, install.sh's one-item legacy loop) are
disabled at their lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:08:57 +00:00
GodopuandClaude Opus 5.5 fce9459e42 test(update): the first transition reaches the skill and the conversion (Issue #72, criterion 5)
A v4.11.0 workspace's .mam_deploy/update.sh installs the new version - the
skill present, no updater left - and the skill run after it converts the
registry. An update-backup in an unknown state now restarts the monitor it
stopped: nothing was changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:07:42 +00:00
GodopuandClaude Opus 5.5 47fe24e07c feat: multi-agent-mux-update skill and in-place install (Issue #72)
The update is a skill (decision 4); deploy/update.sh and the installed
.mam_deploy/update.sh are gone.

- Phase 1 (installed code, update.sh): publish the update marker under the
  exclusive admission lock; refuse before any change while a running row, a
  live herdr agent in the workspace on any running server, an open job
  (unreadable JSON included, dry_run not), a live loop marker or a
  monitor.lock held by a monitor the skills did not start (U3) is there - the
  calling agent (server + pane, its row and one running job) excepted
  (lib_py/update.py precheck, decisions 1 and 7). Stop the monitor (TERM, then
  KILL and restart it if it does not end; decision 2), handle a leftover
  backup by its state (U4), back up .agents, .mam_deploy, root files, .venv,
  the installer's .mam files and the registry (SQLite backup API under the
  registry lock) into .mam/update-backup/, then exec a copy of
  install_phase.sh outside the tree (decision 5).
- The copy (no lib.sh/lib_py, bash 3.2) runs the installer in place; on
  failure it restores the backup and restarts the monitor from the restored
  tree; on success it runs phase 2, post_update.sh of the new tree, which
  converts the registry with lib_py.registry_format and restarts the monitor
  if it ran. The backup and the marker are removed at the end.
- install.sh updates in place (decisions 3, 4, 9): install_state keys are set,
  one line each; no updater is installed and an old one (.mam_deploy/update.sh,
  or a root update.sh the manifest lists) is removed; framework files the new
  version no longer ships are deleted when unmodified, kept and listed when
  modified, and dropped from base/, the manifest and asset_hashes.txt; a local
  source's .mam is not staged. remove.sh knows the new skill; CI shellchecks
  its scripts; docs point at the skill.
- test_issue8_update_state and test_deploy_layout td6/td27 run the skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:57 +00:00
GodopuandClaude Opus 5.5 6359d65999 feat: update admission gate - entries refuse while an update runs (Issue #72, decision 8, U2)
The update skill publishes .mam/update-in-progress (pid + lstart, the
loop_lock.sh rules) while it holds .mam/update-admission.lock exclusively.
create, resume, run_loop.sh and delegate submit (decision 8) and stop and
orc-onboard (U2: a rollback would restore the registry without their write)
take that lock shared (lib.sh mam_update_admit, fd 7), refuse while the marker
is live or unknown, and hold the lock until their record is written. A stale
marker refuses nothing. lib_py.pane_lock acquire gains --shared and
--wait-env (MAM_UPDATE_ADMISSION_WAIT_S, default 120); create's wrapper path
closes fd 7 for the long-lived child. The exit-1 rows of create/resume/stop
name the refusal; skill_code_blocks.json pins follow the shifted lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:42 +00:00
GodopuandClaude Opus 5.5 179473e5d5 feat(resume): resume ensures the monitor daemon (Issue #72, decision 2)
Only create started the per-workspace monitor, and only on its job path, so
the stop -> update -> resume flow the update skill requires left no monitor.
resume now calls start_watchdog with the control root as workdir (as create
does since #21 A8, also for a worktree session) after the row is written, on
both the already-running and the spawn path. MAM_NO_SUBSCRIBER_DAEMON=1 starts
none; a failure warns and does not fail the resume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:06:32 +00:00
GodopuandClaude Opus 5.5 ec378e82c9 fix(attach): hints print the row's attach_command (Issue #72, comments 926/930, U1)
create's closing hint, the delegate-job wrapper's dispatch hint and stop's
stuck-TUI message printed the pre-#71 forms (`herdr session attach <name>`,
`HERDR_SESSION_NAME=... herdr agent attach <name>`), which miss an agent whose
name is over 32 characters or has upper case. They now print the session row's
attach_command, or lib_py.agents.sanitize.attach_command(server, name) when
there is no row (lib.sh session_attach_command). deploy/INSTALL.md and
install_mam.sh show the `herdr --session <server> agent attach <herdr name>`
form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 04:05:00 +00:00
35 changed files with 2868 additions and 346 deletions

No files matched your search

+74
View File
@@ -1671,6 +1671,22 @@ print(resolved)
"
}
# session_attach_command <session_name> [server]
#
# Issue #72: the attach hint a script prints - the session row's
# attach_command, else the one lib_py.agents.sanitize.attach_command builds for
# <server> (default: `default`).
session_attach_command() {
MAM_STATE_JSON="$(load_state_json)" SESSION_NAME="$1" ATTACH_SERVER="${2:-default}" python3 -c "
import os, json
from lib_py.agents.sanitize import attach_command
name = os.environ['SESSION_NAME']
d = json.loads(os.environ.get('MAM_STATE_JSON') or '{}')
row = next((s for s in d.get('herdr_sessions', []) if s.get('name') == name), {})
print(row.get('attach_command') or attach_command(os.environ['ATTACH_SERVER'], name))
"
}
# resolve_herdr_session <session_name> [workspace]
#
# returns the isolated herdr *session* name (socket/daemon) to use for this MAM session row,
@@ -3231,3 +3247,61 @@ mam_control_plane_prefix() {
[ -n "${1:-}" ] || return 0
PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" "$(_delegate_py_bin)" -m lib_py.control_plane prefix "$1"
}
# ---------------------------------------------------------------------------
# Issue #72 (decision 8): the update admission gate.
#
# The update skill publishes .mam/update-in-progress (pid + lstart, loop_lock.sh
# rules) while it holds .mam/update-admission.lock exclusively, then runs its
# pre-check. Every entry that starts something the pre-check must see - create,
# resume, run_loop.sh, delegate submit - or writes the registry - stop,
# orc-onboard - takes that lock shared on fd 7, refuses while the marker is
# `live` or `unknown`, and keeps fd 7 until its record is written
# (mam_update_admit_release). An update therefore either waits for that record
# and sees it, or publishes first and the entry refuses. The lock is a BSD
# flock taken by lib_py.pane_lock on the open fd: it ends when every process
# holding fd 7 has ended, however that ends, so nothing long-lived may inherit
# fd 7 (`7>&-` on such a child, as pane_lock's fd 9). The wait is bounded by
# MAM_UPDATE_ADMISSION_WAIT_S (default 120).
#
# mam_update_admit [workspace [fd]]: rc 0 admitted (fd 7, or the given one,
# held unless the workspace has no .mam), 1 refused (stderr). Delegate submit
# also takes the workspaces its target is in on fds 20 and up.
mam_update_admit() {
local root="${1:-$WORKSPACE_ROOT}" fd="${2:-7}"
local lock="$root/.mam/update-admission.lock" marker="$root/.mam/update-in-progress" st
# No .mam: not an installed workspace, which is all the update runs in (and a
# checkout running its own scripts gets no .mam from this).
[ -d "$root/.mam" ] || return 0
if ! eval "exec $fd>>\"\$lock\""; then
echo "ERROR: cannot open the update admission lock $lock" >&2
return 1
fi
case "$(PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" python3 -m lib_py.pane_lock acquire --fd "$fd" --shared --wait-env MAM_UPDATE_ADMISSION_WAIT_S)" in
locked) ;;
timeout)
mam_update_admit_release "$fd"
echo "ERROR: an update of $root holds its admission lock (MAM_UPDATE_ADMISSION_WAIT_S passed); try again when it has finished" >&2
return 1 ;;
*)
mam_update_admit_release "$fd"
echo "ERROR: the update admission lock $lock could not be decided (lib_py.pane_lock)" >&2
return 1 ;;
esac
[ -f "$marker" ] || return 0
# shellcheck disable=SC1091
declare -F _mam_lock_holder_state >/dev/null || source "$SKILL_DIR/multi-agent-mux-loop/scripts/loop_lock.sh"
st="$(_mam_lock_holder_state "$marker")"
case "$st" in
stale)
echo "NOTE: stale update marker $marker (its update is gone); ignored" >&2
return 0 ;;
esac
mam_update_admit_release "$fd"
echo "ERROR: an update of $root is in progress (marker $marker, holder $st); try again when it has finished" >&2
return 1
}
mam_update_admit_release() {
eval "exec ${1:-7}>&-"
}
+10 -5
View File
@@ -18,6 +18,9 @@ one lock per server from the choice to the claim.
pane_lock.py acquire --fd N flock(LOCK_EX) on the inherited fd N; prints
`locked`, or `timeout` once MAM_PANE_CLAIM_WAIT_S
(default 120, minimum 1) has passed
[--shared] [--wait-env VAR] Issue #72: LOCK_SH instead, and the wait read
from VAR (same default and minimum) - the
update admission lock (lib.sh mam_update_admit)
The lock is a BSD flock, which belongs to the open file on fd 9: it outlives
this helper and ends when the last process holding that fd ends, however it
@@ -63,11 +66,11 @@ def lock_path(session: Optional[str], env: Mapping[str, str]) -> str:
return os.path.join(home, ".local", "state", "mam", "pane-lock", session + ".lock")
def wait_s(env: Mapping[str, str]) -> int:
return ints.positive_int(env.get("MAM_PANE_CLAIM_WAIT_S"), 1) or DEFAULT_WAIT_S
def wait_s(env: Mapping[str, str], var: str = "MAM_PANE_CLAIM_WAIT_S") -> int:
return ints.positive_int(env.get(var), 1) or DEFAULT_WAIT_S
def acquire(fd: int, wait: float) -> str:
def acquire(fd: int, wait: float, shared: bool = False) -> str:
"""`locked`, or `timeout` when the lock was not free within `wait` s. The
deadline is on the monotonic clock; a step of the wall clock does not
stretch or cut it. Only contention is waited out: any other error (a bad
@@ -75,7 +78,7 @@ def acquire(fd: int, wait: float) -> str:
deadline = time.monotonic() + wait
while True:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fd, (fcntl.LOCK_SH if shared else fcntl.LOCK_EX) | fcntl.LOCK_NB)
return "locked"
except BlockingIOError:
if time.monotonic() >= deadline:
@@ -90,11 +93,13 @@ def main(argv: Optional[List[str]] = None) -> int:
s.add_argument("--session", default="")
s = sub.add_parser("acquire")
s.add_argument("--fd", type=int, required=True)
s.add_argument("--shared", action="store_true")
s.add_argument("--wait-env", default="MAM_PANE_CLAIM_WAIT_S")
a = p.parse_args(argv)
if a.cmd == "path":
print(lock_path(a.session, os.environ))
else:
print(acquire(a.fd, wait_s(os.environ)))
print(acquire(a.fd, wait_s(os.environ, a.wait_env), a.shared))
return 0
+410
View File
@@ -0,0 +1,410 @@
"""Decisions behind the multi-agent-mux-update skill (Issue #72).
update.sh keeps the orchestration in shell (phase 1: this tree's code) and asks
this module what needs parsing. Standard library only, Python 3.9.
python -m lib_py.update precheck --workspace W --herdr H
What refuses the update (comment 921, decisions 1 and 7): the running
rows of the registry (MAM_STATE_JSON, from lib.sh `load_state_json
strict`), every live herdr agent whose cwd is inside W on any server
`herdr session list --json` reports running (read only: the real herdr H,
never the shim, so no server is started), and the open jobs. The agent
that called the update - by (server, pane): pane_lock.caller_server and
HERDR_PANE_ID - is not counted, nor its row and its one running job.
Prints one line per blocker with how to clear it; rc 0 nothing blocks,
1 something does (a herdr or registry that cannot be read included).
python -m lib_py.update monitor --workspace W
JSON {"pids": [...], "lock": "free"|"held"}: the monitor daemons
daemon.find_running recognises, and whether .mam/monitor.lock is held.
python -m lib_py.update target-workspaces --workspace W --agent-session S --server V --herdr H
The workspaces other than W whose update admission a delegate submit
to `herdr:<name>` must take (decision 8), one per line: every installed
workspace at or above the target's cwd (its row's pane.cwd; with no row
the live agent's on server V) and its row's control_plane
workspace_root. Nothing for a non-herdr target. rc 1: herdr cannot
answer for a rowless target.
python -m lib_py.update backup-registry --workspace W --dest D
Copies the registry into D under the registry's own write lock
(agent-sessions.db.lock, as atomic_yaml takes it): the DB through the
SQLite backup API (commits still in -wal included), the YAML and its
.bak as files. Prints the names it copied.
"""
from __future__ import annotations
import argparse
import fcntl
import json
import os
import shlex
import shutil
import sqlite3
import subprocess
import sys
from typing import Dict, Iterable, List, Mapping, Optional, Tuple
from lib_py.agents.sanitize import sanitize_herdr_agent_name
from lib_py.delegation import daemon
from lib_py.delegation.engine import TERMINAL_STATUSES
from lib_py.pane_lock import caller_server
REGISTRY_FILES = ("agent-sessions.yaml", "agent-sessions.yaml.bak", "agent-sessions.db")
HERDR_TIMEOUT_S = 20
def _real(p: str) -> str:
return os.path.realpath(p)
def _inside(path: str, root: str) -> bool:
if not path:
return False
p, r = _real(path), _real(root).rstrip(os.sep)
return p == r or p.startswith(r + os.sep)
def row_server(row: dict) -> str:
return row.get("herdr_session") or row.get("herdr_server") or "default"
# --------------------------------------------------------------------------- herdr
class HerdrError(Exception):
pass
def _herdr_json(herdr: str, args: List[str], env: Mapping[str, str]) -> dict:
# A pane's own scope ids would narrow the answer to the caller's workspace.
cenv = {k: v for k, v in env.items()
if k not in ("HERDR_WORKSPACE_ID", "HERDR_PANE_ID", "HERDR_TAB_ID")}
try:
res = subprocess.run([herdr, *args], env=cenv, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=HERDR_TIMEOUT_S)
except (OSError, subprocess.SubprocessError) as exc:
raise HerdrError("`herdr %s` failed: %s" % (" ".join(args), exc))
if res.returncode != 0:
raise HerdrError("`herdr %s` exited %d: %s" % (" ".join(args), res.returncode,
(res.stderr or "").strip()[:200]))
try:
out = res.stdout
return json.loads(out[out.index("{"):])
except ValueError:
raise HerdrError("`herdr %s` printed no JSON" % " ".join(args))
def running_servers(herdr: str, env: Mapping[str, str]) -> List[str]:
d = _herdr_json(herdr, ["session", "list", "--json"], env)
sessions = d.get("sessions")
if not isinstance(sessions, list):
raise HerdrError("`herdr session list --json` has no sessions list")
return [s["name"] for s in sessions
if isinstance(s, dict) and s.get("running") and s.get("name")]
def server_agents(herdr: str, server: str, env: Mapping[str, str]) -> List[dict]:
d = _herdr_json(herdr, ["--session", server, "agent", "list"], env)
agents = (d.get("result") or {}).get("agents")
if not isinstance(agents, list):
raise HerdrError("`herdr --session %s agent list` has no agents list" % server)
return [a for a in agents if isinstance(a, dict)]
# --------------------------------------------------------------------------- jobs
def _job_records(d: str) -> Iterable[Tuple[str, Optional[dict]]]:
"""(path, record) per *.json; (path, None) when it cannot be read, and
(d, None) when d itself cannot be listed (not an empty registry)."""
try:
names = sorted(n for n in os.listdir(d) if n.endswith(".json") and not n.startswith("."))
except OSError:
yield d, None
return
for p in (os.path.join(d, n) for n in names):
try:
with open(p, encoding="utf-8") as fh:
rec = json.load(fh)
except (OSError, ValueError):
yield p, None
continue
yield p, rec if isinstance(rec, dict) else None
def _is_jobs_dir(d: str) -> bool:
"""d is a directory, or cannot be looked at (an unreadable parent): only an
absent d (or a file) is skipped."""
try:
os.stat(d)
except (FileNotFoundError, NotADirectoryError):
return False
except OSError:
return True
return os.path.isdir(d)
def _is_open(rec: dict) -> bool:
return rec.get("status") not in TERMINAL_STATUSES and rec.get("dry_run") is not True
def open_jobs(workspace: str, state: dict, env: Mapping[str, str]) -> List[dict]:
"""Every open record (not completed/error/cancelled, not dry_run) in this
workspace's .mam/jobs and MAM_JOBS_DIR; in another row's
control_plane.mam_jobs_dir only those addressed to a row of this registry.
A record, or a directory, that cannot be read is open (path only)."""
own, seen = [], set()
for c in (os.path.join(workspace, ".mam", "jobs"), env.get("MAM_JOBS_DIR") or ""):
if c and _is_jobs_dir(c) and _real(c) not in seen:
seen.add(_real(c))
own.append(c)
rows = state.get("herdr_sessions") or []
names = {"herdr:%s" % s.get("name") for s in rows if s.get("name")}
other = []
for s in rows:
c = (s.get("control_plane") or {}).get("mam_jobs_dir") or ""
if c and _is_jobs_dir(c) and _real(c) not in seen:
seen.add(_real(c))
other.append(c)
out = []
for dirs, addressed_only in ((own, False), (other, True)):
for d in dirs:
for path, rec in _job_records(d):
if rec is None:
out.append({"path": path, "unreadable": True, "dir": path == d})
elif _is_open(rec) and (not addressed_only or rec.get("agent_session") in names):
out.append({"path": path, "job_id": rec.get("job_id") or "",
"agent_session": rec.get("agent_session") or "",
"status": rec.get("status"), "registry_dir": d})
return out
# --------------------------------------------------------------------------- precheck
def precheck(workspace: str, state: dict, herdr: str, env: Mapping[str, str]) -> List[str]:
"""The lines that refuse the update; [] when nothing does."""
blockers: List[str] = []
stop = ("bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh --session %s"
" (resume it after the update with multi-agent-mux-resume)")
# A rowless agent: stop finds no row to take its server from (#71).
stop_rowless = ("bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh --session %s"
" --herdr-session %s")
# The caller (decision 7): its server and pane, both or nothing.
me_server, me_pane = caller_server(env), env.get("HERDR_PANE_ID") or ""
me_name: Optional[str] = None
me_found = False
agents_by_server: Dict[str, List[dict]] = {}
try:
for server in running_servers(herdr, env):
agents_by_server[server] = server_agents(herdr, server, env)
except HerdrError as exc:
blockers.append("herdr: cannot list the live agents (%s); fix herdr and try again" % exc)
if me_server and me_pane:
for a in agents_by_server.get(me_server, []):
if a.get("pane_id") == me_pane:
me_found = True
me_name = a.get("name") or None # a nameless pane is excluded by pane only
break
rows = state.get("herdr_sessions") or []
me_row = None
if me_name:
for s in rows:
if (s.get("status") == "running" and row_server(s) == me_server
and s.get("name") and sanitize_herdr_agent_name(s["name"]) == me_name):
me_row = s
break
for s in rows:
if s.get("status") == "running" and s is not me_row:
blockers.append("session %s is running (server %s); stop it: %s"
% (s.get("name"), row_server(s), stop % shlex.quote(s.get("name") or "")))
row_keys = {(row_server(s), sanitize_herdr_agent_name(s["name"]))
for s in rows if s.get("status") == "running" and s.get("name")}
for server, agents in agents_by_server.items():
for a in agents:
if me_found and server == me_server and a.get("pane_id") == me_pane:
continue
if not (_inside(a.get("cwd") or "", workspace)
or _inside(a.get("foreground_cwd") or "", workspace)):
continue
name = a.get("name") or ""
if name and (server, name) in row_keys:
continue # already listed as its running row
blockers.append("herdr agent %s (pane %s, server %s) runs in %s; stop it: %s"
% (name or "<unnamed>", a.get("pane_id"), server, a.get("cwd"),
stop_rowless % (shlex.quote(name), shlex.quote(server)) if name else
"herdr --session %s pane close %s"
% (shlex.quote(server), shlex.quote(a.get("pane_id") or ""))))
jobs = open_jobs(workspace, state, env)
if me_row is not None:
mine = [j for j in jobs if not j.get("unreadable")
and j.get("agent_session") == "herdr:%s" % me_row.get("name")
and j.get("status") == "running"]
if len(mine) == 1:
jobs = [j for j in jobs if j is not mine[0]]
for j in jobs:
if j.get("dir"):
blockers.append("jobs directory %s cannot be listed; fix its permissions, then try again"
% j["path"])
elif j.get("unreadable"):
blockers.append("job record %s cannot be read; check it, then move or delete it by hand"
% j["path"])
else:
blockers.append("job %s (%s, %s) is open; wait for it to end or close it: python "
".agents/skills/multi-agent-mux-delegate-job/scripts/registry.py"
" --registry-dir %s close --job %s --request cancelled"
% (j["job_id"], j["agent_session"] or "-", j["status"],
j["registry_dir"], j["job_id"]))
if blockers and not me_found:
blockers.append("note: no herdr agent was found at the caller's server %r and pane %r, so"
" nothing was excluded as the calling agent" % (me_server, me_pane))
return blockers
# --------------------------------------------------------------------------- delegate target
def _installed(d: str) -> bool:
"""update.sh's own test of an installed workspace."""
return os.path.isfile(os.path.join(d, ".mam_deploy", "remove.sh")) or os.path.isfile(os.path.join(d, "remove.sh"))
def target_workspaces(workspace: str, state: dict, agent_session: str, server: str,
herdr: str, env: Mapping[str, str]) -> List[str]:
"""Every installed workspace at or above the target's cwd - an update of
any of them counts that agent (decision 1) - and its row's
control_plane.workspace_root; the caller's own workspace left out. With
no row the cwd is the live agent's on <server> (the one delivery uses),
read with the real herdr. No herdr, a server that is not running, or no
such agent gives nothing (delivery then fails on its own); a herdr that
cannot answer raises HerdrError."""
if not agent_session.startswith("herdr:"):
return []
name = agent_session[len("herdr:"):]
row = next((s for s in state.get("herdr_sessions") or []
if isinstance(s, dict) and s.get("name") == name), None)
cwds, out = [], []
if row is not None:
cwds.append((row.get("pane") or {}).get("cwd") or "")
root = (row.get("control_plane") or {}).get("workspace_root") or ""
if root:
out.append(_real(root))
elif herdr and server in running_servers(herdr, env):
for a in server_agents(herdr, server, env):
if a.get("name") in (name, sanitize_herdr_agent_name(name)):
cwds += [a.get("cwd") or "", a.get("foreground_cwd") or ""]
break
for cwd in cwds:
d = _real(cwd) if cwd else ""
while d:
if _installed(d):
out.append(d)
if os.path.dirname(d) == d:
break
d = os.path.dirname(d)
me = _real(workspace)
return [d for i, d in enumerate(out) if d != me and d not in out[:i]]
# --------------------------------------------------------------------------- monitor
def monitor_state(workspace: str) -> dict:
pids = daemon.find_running(daemon.monitor_script_path(workspace))
lock = os.path.join(workspace, ".mam", "monitor.lock")
held = False
if os.path.exists(lock):
with open(lock, "a") as fh:
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fh, fcntl.LOCK_UN)
except BlockingIOError:
held = True
return {"pids": pids, "lock": "held" if held else "free"}
# --------------------------------------------------------------------------- registry backup
def backup_registry(workspace: str, dest: str) -> List[str]:
mam = os.path.join(workspace, ".mam")
db = os.path.join(mam, "agent-sessions.db")
os.makedirs(dest, exist_ok=True)
copied = []
with open(db + ".lock", "w") as lk:
fcntl.flock(lk, fcntl.LOCK_EX)
try:
if os.path.exists(db):
src = sqlite3.connect(db, timeout=60.0)
dst = sqlite3.connect(os.path.join(dest, "agent-sessions.db"))
try:
src.backup(dst)
finally:
dst.close()
src.close()
copied.append("agent-sessions.db")
for name in REGISTRY_FILES:
p = os.path.join(mam, name)
if name != "agent-sessions.db" and os.path.exists(p):
shutil.copy2(p, os.path.join(dest, name))
copied.append(name)
finally:
fcntl.flock(lk, fcntl.LOCK_UN)
return copied
def _state() -> dict:
return json.loads(os.environ.get("MAM_STATE_JSON") or "{}") or {}
def main(argv: Optional[List[str]] = None) -> int:
p = argparse.ArgumentParser(prog="lib_py.update")
sub = p.add_subparsers(dest="cmd", required=True)
s = sub.add_parser("precheck")
s.add_argument("--workspace", required=True)
s.add_argument("--herdr", required=True)
s = sub.add_parser("monitor")
s.add_argument("--workspace", required=True)
s = sub.add_parser("target-workspaces")
s.add_argument("--workspace", required=True)
s.add_argument("--agent-session", required=True)
s.add_argument("--server", default="default")
s.add_argument("--herdr", default="")
s = sub.add_parser("backup-registry")
s.add_argument("--workspace", required=True)
s.add_argument("--dest", required=True)
a = p.parse_args(argv)
ws = _real(a.workspace)
if a.cmd == "precheck":
try:
state = _state()
except ValueError as exc:
print("registry: cannot parse MAM_STATE_JSON (%s)" % exc)
return 1
lines = precheck(ws, state, a.herdr, os.environ)
for line in lines:
print(line)
return 1 if lines else 0
if a.cmd == "target-workspaces":
try:
state = _state()
except ValueError:
state = {}
try:
roots = target_workspaces(ws, state, a.agent_session, a.server, a.herdr, os.environ)
except HerdrError as exc:
print("cannot find %s live: %s" % (a.agent_session, exc), file=sys.stderr)
return 1
for root in roots:
print(root)
return 0
if a.cmd == "monitor":
print(json.dumps(monitor_state(ws)))
return 0
for name in backup_registry(ws, a.dest):
print(name)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -181,7 +181,7 @@ bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh --workspace
| Exit | Meaning |
|---|---|
| 0 | Created; or `--dry-run` |
| 1 | Pre-flight failure, nothing was spawned: workspace not a directory, herdr not installed, agent CLI not in `PATH`, agent not authenticated, or a same-cwd session of the same Class A agent did not record its conversation ID within `MAM_ID_ORDER_WAIT_S` (default 600). After spawn: the task brief could not be written or the instruction injection failed (the session is rolled back) |
| 1 | Pre-flight failure, nothing was spawned: workspace not a directory, herdr not installed, agent CLI not in `PATH`, agent not authenticated, or a same-cwd session of the same Class A agent did not record its conversation ID within `MAM_ID_ORDER_WAIT_S` (default 600), or an update of the workspace is in progress (`multi-agent-mux-update`). After spawn: the task brief could not be written or the instruction injection failed (the session is rolled back) |
| 2 | Bad arguments; control-plane binding refused for a worktree; whitespace in a control-plane path that must be forwarded |
| 3 | A herdr session of that name already exists (`multi-agent-mux-resume`, or `multi-agent-mux-stop` first) |
| 4 | The YAML already has a `running` row of that name. The YAML write raises it after spawn, and the herdr session is rolled back. A row the monitor registered for this very pane while it was being created is replaced instead |
@@ -260,7 +260,7 @@ spawn() {
SESSION_UUID=""
CMD_FULL="$("$(_delegate_py_bin)" -m lib_py.agents spawn-spec "$AGENT" "$RESOLVED_BIN" "" "true" 2>/dev/null || echo "${RESOLVED_BIN} --dangerously-skip-permissions")"
[ -z "$CP_PREFIX" ] || echo "WARNING: the ~/.local/bin wrapper path bypasses herdr; the control-plane binding is recorded but not forwarded" >&2
nohup "$WRAPPER" >/dev/null 2>&1 &
nohup "$WRAPPER" >/dev/null 2>&1 7>&- &
disown
else
HERDR_SESSION_NAME="$HERDR_SESSION_NAME" _herdr new-session -d -s "$SESSION_NAME" -x 140 -y 40 -c "$WORKSPACE" "$CP_PREFIX$CMD_FULL"
@@ -285,6 +285,9 @@ if [ "$DRY_RUN" = "1" ]; then
fi
# Issue #63: a same-cwd session of this post-discovered agent records its id first (lib.sh mam_id_order_wait).
mam_id_order_wait "$WORKSPACE" "$AGENT" "$SESSION_NAME" || exit 1
# Issue #72: no new session while an update runs; held until the row is written
# (lib.sh mam_update_admit).
mam_update_admit || exit 1
HERDR_EPOCH=$(date +%s)
NOW_ISO=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
spawn
@@ -638,6 +641,7 @@ snap['cwd'] = os.environ['PANE_CWD']
print(f"appended: {name}", flush=True)
PYEOF
ROW_APPENDED=1
mam_update_admit_release
echo
echo "=== created ==="
@@ -767,6 +771,6 @@ fi
trap - EXIT
echo "agent-sessions.yaml updated"
echo
echo "Attach: herdr session attach $SESSION_NAME"
echo "Attach: $(session_attach_command "$SESSION_NAME" "${HERDR_SESSION_NAME:-default}")"
echo "Delete: use multi-agent-mux-stop skill"
echo "Resume: use multi-agent-mux-resume skill (after first message creates a session id)"
@@ -301,6 +301,26 @@ cmd_submit() {
local reg_dry=()
if [[ "$DRY_RUN" == "1" ]]; then reg_dry=(--dry-run); fi
# Issue #72: no job is registered while an update of this registry's
# workspace or of one the target is in (lib_py.update target-workspaces:
# its row's, or with no row the live agent's on the server delivery uses)
# runs; each is held, fd 7 then 20.., until the record is written (lib.sh
# mam_update_admit). A dry-run record is never dispatched and is not
# counted by the update.
local admit_fds=() target_ws tws fd=20
if [[ "$DRY_RUN" != "1" ]]; then
target_ws="$(MAM_STATE_JSON="$(load_state_json)" PYTHONPATH="$SKILL_DIR:${PYTHONPATH:-}" "$PY" -m lib_py.update \
target-workspaces --workspace "$WORKSPACE_ROOT" --agent-session "$AGENT_SESSION" \
--server "$(_is_remote "$AGENT_SESSION" || resolve_herdr_session "${AGENT_SESSION#herdr:}" "$WORKDIR" 2>/dev/null)" \
--herdr "$(_resolve_real_herdr_path || true)")" \
|| { echo "ERROR: cannot resolve the workspace of $AGENT_SESSION; nothing was registered" >&2; exit 1; }
mam_update_admit "$WORKSPACE_ROOT" || exit 1
while IFS= read -r tws; do
[[ -n "$tws" ]] || continue
mam_update_admit "$tws" "$fd" || exit 1
admit_fds+=("$fd"); fd=$((fd + 1))
done <<< "$target_ws"
fi
# 1) register job (prints the new job id)
JOB_ID="$("$PY" "$SCRIPT_DIR/scripts/registry.py" --registry-dir "$REGISTRY_DIR" register \
"${reg_prompt[@]}" --agent "$AGENT" --agent-session "$AGENT_SESSION" --role "$DELEGATE_ROLE" \
@@ -310,6 +330,10 @@ cmd_submit() {
--job-type "$TYPE" --reviewer "$REVIEWER" --reviewer-session "$REVIEWER_SESSION" \
--max-iterations "$MAX_ITERATIONS" ${reg_parent[@]+"${reg_parent[@]}"} ${reg_dry[@]+"${reg_dry[@]}"})"
echo "registered job: $JOB_ID"
if [[ "$DRY_RUN" != "1" ]]; then
mam_update_admit_release
for fd in ${admit_fds[@]+"${admit_fds[@]}"}; do mam_update_admit_release "$fd"; done
fi
# 1-1) Provision job directory and write direct brief.md (MAM Job Restructuring)
local job_dir="$REGISTRY_DIR/$JOB_ID"
@@ -1018,7 +1042,7 @@ inject_agent() {
return 1
fi
echo "작업이 세션 '$sess'에 전송되었습니다. (연결하려면: HERDR_SESSION_NAME=$HERDR_SESSION_NAME herdr agent attach $sess — lib.sh를 source한 셸에서 실행)"
echo "작업이 세션 '$sess'에 전송되었습니다. (연결하려면: $(session_attach_command "$sess" "${HERDR_SESSION_NAME:-default}"))"
trap - EXIT
return 0
}
@@ -244,6 +244,9 @@ log_error() {
echo -e "\033[1;31m[✗]\033[0m $1"
}
# Issue #72: no loop starts while an update runs; held until the loop marker
# is published (lib.sh mam_update_admit), released before the loop body runs.
mam_update_admit "$MAM_REAL_ROOT" || exit 1
_mam_acquire_rc=0
mam_acquire_loop_lock "$MAM_LOOP_MARKER" || _mam_acquire_rc=$?
case "$_mam_acquire_rc" in
@@ -262,6 +265,7 @@ case "$_mam_acquire_rc" in
;;
esac
trap _mam_release_guard EXIT INT TERM HUP
mam_update_admit_release
rm -f "$MAM_REAL_ROOT/.agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job".*.tmp 2>/dev/null || true
# --all-reviewer silently takes precedence over an explicit --reviewer list;
@@ -257,6 +257,11 @@ if [ "$DRY_RUN" = true ]; then
exit 0
fi
# Issue #72 (U2): no registry write while an update runs - a rollback would
# restore the registry without it. Held until the write is done (lib.sh
# mam_update_admit).
mam_update_admit || exit 1
# Issue #13: the orchestrator has no herdr_sessions row, so stop_session.sh
# --purge-conversation can never reach its conversation. Purge FIRST, through
# the adapter (`lib_py.agents purge`): a refusal because the conversation is
@@ -304,6 +309,7 @@ elif op == "remove":
uuids = [u for u in uuids if u != target]
d["orchestrator_uuids"] = uuids
PYEOF
mam_update_admit_release
case "$MODE" in
add) verb="added" ;;
@@ -60,7 +60,7 @@ If the resolved id is empty → for Class A agents (`agy`, `hermes`, `opencode`,
bash .agents/skills/multi-agent-mux-resume/scripts/resume_session.sh --workspace <path> --agent <agent> --session <name>
```
`resume_session.sh` does the whole flow: original-cwd check, control-plane binding, the session-id decision (`lib_py/agents/session_ids.py`), the herdr-alive check, the agent command (adapter `resume-spec`, or `spawn-spec` for a Class A fresh spawn), spawn (just before it, the session's own empty shell left by a server restart is closed: same session id, no agent, nothing running; `lib_py/herdr_restore.py`), start-up dialogs, and the YAML update (`update_yaml_resumed.sh`, which also publishes a `progress "resumed"` event when the row has a `delegate_job_id`). It never attaches; attaching is a separate step that names the server, because the server the script exports does not reach the calling shell and the native `herdr` binary does not route on environment variables: the row's `attach_command`, `herdr --session <server> agent attach <name>` (`<server>` = the `--herdr-session` you passed, or the row's `herdr_session`; `<name>` = the name herdr registered).
`resume_session.sh` does the whole flow: original-cwd check, control-plane binding, the session-id decision (`lib_py/agents/session_ids.py`), the herdr-alive check, the agent command (adapter `resume-spec`, or `spawn-spec` for a Class A fresh spawn), spawn (just before it, the session's own empty shell left by a server restart is closed: same session id, no agent, nothing running; `lib_py/herdr_restore.py`), start-up dialogs, and the YAML update (`update_yaml_resumed.sh`, which also publishes a `progress "resumed"` event when the row has a `delegate_job_id`), then the control root's monitor daemon (`start_watchdog`, as create; not with `MAM_NO_SUBSCRIBER_DAEMON=1`). It never attaches; attaching is a separate step that names the server, because the server the script exports does not reach the calling shell and the native `herdr` binary does not route on environment variables: the row's `attach_command`, `herdr --session <server> agent attach <name>` (`<server>` = the `--herdr-session` you passed, or the row's `herdr_session`; `<name>` = the name herdr registered).
| Option | Meaning |
|---|---|
@@ -75,11 +75,11 @@ bash .agents/skills/multi-agent-mux-resume/scripts/resume_session.sh --workspace
| Exit | Meaning |
|---|---|
| 0 | Resumed; or already running, so only the row was updated (no attach); or `--dry-run` |
| 1 | From the checks before spawn, nothing was changed: no saved session for claude/grok (use `multi-agent-mux-create` first), the row's id is also claimed by another row, a same-cwd Class A session did not record its id within `MAM_ID_ORDER_WAIT_S` (default 600), or agent binary missing/not executable. **The YAML update after spawn (`update_yaml_resumed.sh`) also exits 1** (YAML not found) |
| 1 | From the checks before spawn, nothing was changed: no saved session for claude/grok (use `multi-agent-mux-create` first), the row's id is also claimed by another row, a same-cwd Class A session did not record its id within `MAM_ID_ORDER_WAIT_S` (default 600), agent binary missing/not executable, or an update of the workspace is in progress (`multi-agent-mux-update`). **The YAML update after spawn (`update_yaml_resumed.sh`) also exits 1** (YAML not found) |
| 2 | From the checks before spawn, nothing was changed: bad arguments, `--workspace` differs from the original cwd or it no longer exists, control-plane binding refused, whitespace in the control-plane path. **The YAML update after spawn also exits 2** (bad arguments) |
| other | A later step failed (herdr spawn, start-up dialogs, YAML update) and its code is passed through. resume does not roll back |
A non-zero exit after the spawn can leave the herdr session running while the row is still `stopped`. Before retrying or creating a new session, check with `multi-agent-mux-status`. The `ERROR:` line on stderr tells the two apart: the pre-spawn checks' messages are in `resume_session.sh` (lines 50-243).
A non-zero exit after the spawn can leave the herdr session running while the row is still `stopped`. Before retrying or creating a new session, check with `multi-agent-mux-status`. The `ERROR:` line on stderr tells the two apart: the pre-spawn checks' messages are in `resume_session.sh` (lines 50-259).
## Original cwd only (Issue #21 phase 2)
@@ -154,18 +154,34 @@ if [ -z "$UUID" ]; then
esac
fi
# Issue #72: a resumed session gets the monitor daemon create starts - the
# control root's (Issue #21 A8), also for a worktree --workspace. Called after
# the row is written; MAM_NO_SUBSCRIBER_DAEMON=1 starts none. Never fails the resume.
_ensure_monitor_daemon() {
local wd_pid
if wd_pid=$(start_watchdog "resume" "$WORKSPACE_ROOT"); then
echo "watchdog PID: $wd_pid"
else
echo "WARNING: could not start the monitor daemon of '$WORKSPACE_ROOT'; the session itself was resumed" >&2
fi
}
# 2. If herdr is alive, print warning or attach.
if [ "$HERDR_ALIVE" = "1" ]; then
if [ "${DRY_RUN:-0}" = "1" ]; then
echo "[dry-run] herdr '$SESSION_NAME' already running — nothing to validate"
exit 0
fi
# Issue #72: no registry write while an update runs (lib.sh mam_update_admit).
mam_update_admit || exit 1
echo "herdr '$SESSION_NAME' already running."
# Just update YAML to make sure it's set to running
CONTROL_PLANE_JSON="$CP_JSON" bash "$(dirname "${BASH_SOURCE[0]}")/update_yaml_resumed.sh" \
--session "$SESSION_NAME" --uuid "$UUID" --agent "$AGENT" --workspace "$WORKSPACE" \
--herdr-session "$HERDR_SESSION_NAME" --already-running \
${HERDR_WORKSPACE_OPT:+--herdr-workspace "$HERDR_WORKSPACE_OPT"}
mam_update_admit_release
_ensure_monitor_daemon
exit 0
fi
@@ -238,6 +254,10 @@ if [ "${DRY_RUN:-0}" = "1" ]; then
exit 0
fi
# Issue #72: no new session while an update runs; held until the row is written
# (lib.sh mam_update_admit).
mam_update_admit || exit 1
# Issue #63: wait if another same-cwd session of this post-discovered agent
# is still being created or has not recorded its id yet (lib.sh mam_id_order_wait).
mam_id_order_wait "$WORKSPACE" "$AGENT" "$SESSION_NAME" "$UUID" || exit 1
@@ -274,5 +294,7 @@ CONTROL_PLANE_JSON="$CP_JSON" bash "$(dirname "${BASH_SOURCE[0]}")/update_yaml_r
--herdr-session "$HERDR_SESSION_NAME" --cmd-full "$CMD_FULL" \
${DROP_FLAG:+$DROP_FLAG} \
${HERDR_WORKSPACE_OPT:+--herdr-workspace "$HERDR_WORKSPACE_OPT"}
mam_update_admit_release
_ensure_monitor_daemon
echo "Successfully resumed $SESSION_NAME ($AGENT)"
+1 -1
View File
@@ -58,7 +58,7 @@ bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh \
| Exit | Meaning |
|---|---|
| 0 | Stopped; or already `stopped` (no-op, except with `--purge-conversation`, which still purges) |
| 1 | From the checks before the stop, nothing was changed: YAML not found, its row could not be read, or the session is neither registered nor on the herdr server. **The registry update after the stop also exits 1** (the row disappeared from the YAML during the run) |
| 1 | From the checks before the stop, nothing was changed: YAML not found, its row could not be read, the session is neither registered nor on the herdr server, or an update of the workspace is in progress (`multi-agent-mux-update`). **The registry update after the stop also exits 1** (the row disappeared from the YAML during the run) |
| 2 | Bad arguments (including the retired `--mode` / `--capture-id` / `--graceful`), or `--agent` omitted and not resolvable |
| 3 | `--purge-conversation` without `--yes` |
| 4 | Purge aborted: the herdr session, or an agent process still holding the conversation, survived the kill chain. Records preserved: the registry row and the conversation files are untouched, though the session itself may already be down |
@@ -100,6 +100,11 @@ fi
[ -n "$SESSION_NAME" ] || { echo "ERROR: --session required" >&2; usage; exit 2; }
[ -f "$AGENT_SESSIONS_YAML" ] || { echo "ERROR: $AGENT_SESSIONS_YAML not found" >&2; exit 1; }
# Issue #72 (U2): no registry write while an update runs - a rollback would
# restore the registry without it. Held until the row is written (lib.sh
# mam_update_admit).
mam_update_admit || exit 1
# Implement the purging-<session> file lock mechanism
if [ "$PURGE" = "1" ]; then
touch "$WORKSPACE_ROOT/.mam/purging-$SESSION_NAME"
@@ -332,7 +337,7 @@ if [ "$PURGE" = "1" ] && [ "$HERDR_ALIVE" = "1" ]; then
if herdr has-session -t "$SESSION_NAME" 2>/dev/null; then
echo "ERROR: session '$SESSION_NAME' is still alive after the kill chain." >&2
echo " Refusing registry removal — records preserved (no state was modified)." >&2
echo " Diagnose the stuck TUI (herdr session attach '$SESSION_NAME'), then re-run" >&2
echo " Diagnose the stuck TUI ($(session_attach_command "$SESSION_NAME" "${HERDR_SESSION_NAME:-default}")), then re-run" >&2
echo " stop_session.sh --purge-conversation --yes (retry is safe/idempotent)." >&2
delegate_publish_event "$DELEGATE_JOB_ID" cancelled "purge aborted: session still alive; no state was modified" "$DELEGATE_JOB_TOKEN" "$DELEGATE_JOB_ATTEMPT"
exit 4
@@ -453,6 +458,7 @@ if purge:
else:
print(f"updated: {name} status={target['status']}", flush=True)
PYEOF
mam_update_admit_release
delegate_publish_event "$DELEGATE_JOB_ID" cancelled "session stopped by operator before job completion" "$DELEGATE_JOB_TOKEN" "$DELEGATE_JOB_ATTEMPT"
@@ -0,0 +1,30 @@
---
name: multi-agent-mux-update
description: "Update an installed MAM workspace in place to the latest version. Refuses while a session, an open job, a review loop or an unknown monitor is live; stops the monitor daemon and starts it again afterwards; rolls everything back if the install fails; converts the registry with the newly installed code. Use instead of the removed .mam_deploy/update.sh."
version: 4.11.0
author: godopu
license: MIT
platforms: [linux, macos]
environments: [terminal, herdr]
metadata:
hermes:
tags: [update, install, registry, migration]
related_skills: [multi-agent-mux-stop, multi-agent-mux-resume, multi-agent-mux-monitor]
---
# Multi-Agent Update
```bash
bash .agents/skills/multi-agent-mux-update/scripts/update.sh --yes
```
| Item | Behaviour |
|---|---|
| Where | An installed workspace only (`.mam_deploy/remove.sh` present). A git checkout updates with `git pull`. |
| Confirmation | `[y/N]` prompt; without a terminal `-y`/`--yes` is required. There is no force option. |
| Refused when | a running registry row, a live herdr agent whose cwd is in the workspace (any running server), an open job (an unreadable job JSON or jobs directory included, `dry_run` excluded), a live loop marker, `.mam/monitor.lock` held by a monitor the skills did not start, or another update. The calling agent (its herdr server + pane, row and one running job) is not counted. Each refusal lists the cleanup; nothing is changed. |
| During the update | create, resume, `run_loop.sh`, delegate submit, stop and orc-onboard refuse (`.mam/update-in-progress`, `.mam/update-admission.lock`). |
| Monitor | stopped first, started again afterwards only if it ran (`MAM_NO_SUBSCRIBER_DAEMON=1`: not started). |
| Install | the installer at `MAM_INSTALLER_URL` (default: Gitea `main`) runs in place; `.mam` is kept. On failure `.agents/`, `.mam_deploy/`, root files, `.venv`, the installer's `.mam` files and the registry are restored from `.mam/update-backup/`. |
| Registry | converted by the new code (`lib_py.registry_format`); a failed conversion keeps the install and exits nonzero. |
| Exit | `0` updated (or cancelled at the prompt), `1` refused or failed, `2` bad arguments; a failed registry conversion after a good install exits with the converter's nonzero code. |
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
# install_phase.sh — multi-agent-mux-update, the part that changes the tree (Issue #72, decision 5).
#
# install_phase.sh run <workspace> <monitor_was_running 0|1> <tmpdir>
# install_phase.sh restore <workspace> [--skip-registry]
#
# update.sh copies this file outside the workspace and execs the copy (same
# pid and start time, so its .mam/update-in-progress marker stays live). From
# here on the tree is being replaced, so this calls no lib.sh or lib_py: it runs
# the installer in place, and on failure restores .mam/update-backup/ and starts
# the monitor again with the restored (old, whole) tree's start_watchdog. On
# success phase 2 is the NEW tree's post_update.sh, in a child process. It ends
# by removing the backup and the marker (only its own: pid + lstart).
# Must run on macOS bash 3.2.
set -uo pipefail
MODE="${1:-}"
WS="${2:-}"
[ -n "$MODE" ] && [ -d "$WS" ] || { echo "usage: $0 run|restore <workspace> ..." >&2; exit 2; }
BK="$WS/.mam/update-backup"
set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
# Puts back every backed-up item, removes the ones that did not exist before,
# then (unless --skip-registry) the registry: the DB from the backup, its
# -wal/-shm removed (macOS keeps them after close).
restore() {
local rel n
[ -f "$BK/items" ] || { echo "ERROR: $BK/items is missing" >&2; return 1; }
while IFS= read -r rel; do
[ -n "$rel" ] || continue
rm -rf "${WS:?}/$rel" || return 1
grep -Fqx -- "$rel" "$BK/absent" 2>/dev/null && continue
mkdir -p "$(dirname "$WS/$rel")" && cp -pPR "$BK/files/$rel" "$WS/$rel" || return 1
done < "$BK/items"
[ "${1:-}" = "--skip-registry" ] && return 0
for n in agent-sessions.yaml agent-sessions.yaml.bak agent-sessions.db; do
rm -f "$WS/.mam/$n" || return 1
if [ -f "$BK/registry/$n" ]; then
cp -p "$BK/registry/$n" "$WS/.mam/$n" || return 1
fi
done
rm -f "$WS/.mam/agent-sessions.db-wal" "$WS/.mam/agent-sessions.db-shm"
}
release_marker() {
local m="$WS/.mam/update-in-progress" pid lstart cur
[ -f "$m" ] || return 0
pid="$(grep '^pid=' "$m" | head -n 1 | cut -d= -f2-)"
lstart="$(grep '^lstart=' "$m" | head -n 1 | cut -d= -f2-)"
cur="$(ps -p $$ -o lstart= 2>/dev/null | tr -s ' ' | sed 's/^ *//;s/ *$//')"
if [ "$pid" = "$$" ] && [ "$lstart" = "$cur" ]; then
rm -f "$m"
fi
}
restart_old_monitor() {
[ "$1" = "1" ] || return 0
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
return 0
fi
local pid
if pid="$(bash -c 'source "$1/.agents/skills/lib.sh" && start_watchdog update "$1"' _ "$WS")"; then
echo "monitor: restarted (PID $pid)"
else
echo "ERROR: could not restart the monitor daemon" >&2
fi
}
case "$MODE" in
restore)
restore "${3:-}"
exit $?
;;
run) ;;
*) echo "usage: $0 run|restore <workspace> ..." >&2; exit 2 ;;
esac
MON="${3:-0}"
TMPD="${4:-}"
trap '[ -z "$TMPD" ] || rm -rf "$TMPD"' EXIT
if ! set_state installing; then
# Nothing was installed yet: drop the backup and put the monitor back as phase 1 found it.
echo "ERROR: cannot write $BK/state; nothing was changed" >&2
rm -rf "$BK"
restart_old_monitor "$MON"
release_marker
exit 1
fi
URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
echo "install: $URL"
irc=0
if command -v curl >/dev/null 2>&1; then
curl -fsSL "$URL" | bash -s -- "$WS" || irc=$?
elif command -v wget >/dev/null 2>&1; then
wget -qO- "$URL" | bash -s -- "$WS" || irc=$?
else
echo "ERROR: neither curl nor wget is available to fetch the installer" >&2
irc=1
fi
if [ "$irc" -ne 0 ]; then
echo "ERROR: the install failed (rc $irc); rolling back" >&2
if restore; then
rm -rf "$BK"
echo "rolled back: the workspace is at the version it had before the update"
restart_old_monitor "$MON"
else
echo "ERROR: the rollback failed; the backup is kept at $BK (the next update restores it)" >&2
fi
release_marker
exit 1
fi
set_state installed
POST="$WS/.agents/skills/multi-agent-mux-update/scripts/post_update.sh"
prc=0
if [ -f "$POST" ]; then
bash "$POST" --monitor "$MON" || prc=$?
else
echo "ERROR: the installed version has no $POST; the registry converts on its next write and the monitor was not restarted" >&2
prc=1
fi
rm -rf "$BK"
release_marker
[ "$prc" -eq 0 ] && echo "Update complete."
exit "$prc"
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
# post_update.sh — multi-agent-mux-update phase 2 (Issue #72, decision 5).
# post_update.sh --monitor <0|1>
#
# Run by install_phase.sh from the NEWLY installed tree, in its own process:
# converts the registry with the new code's entry point (lib_py.registry_format,
# #71) and starts the monitor again if it ran before the update. Exit 0, or
# nonzero when the conversion or the monitor restart failed (the install is
# kept either way: the registry converts again on its next write).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
export WORKSPACE_ROOT="$WS"
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
# shellcheck disable=SC1091
source "$WS/.agents/skills/lib.sh"
MON=0
while [ $# -gt 0 ]; do
case "$1" in
--monitor) MON="$2"; shift 2 ;;
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
esac
done
rc=0
if [ ! -f "$AGENT_SESSIONS_YAML" ] && [ ! -f "$WS/.mam/agent-sessions.db" ]; then
echo "registry: nothing to convert (no registry)"
else
"$(_delegate_py_bin)" -m lib_py.registry_format --yaml "$AGENT_SESSIONS_YAML" || rc=$?
if [ "$rc" -ne 0 ]; then
echo "ERROR: the registry was not converted (rc $rc); the install is kept and the next registry write converts it" >&2
fi
fi
if [ "$MON" = "1" ]; then
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
elif pid="$(start_watchdog update "$WS")"; then
echo "monitor: restarted (PID $pid)"
else
echo "ERROR: could not restart the monitor daemon" >&2
rc=1
fi
fi
exit "$rc"
+257
View File
@@ -0,0 +1,257 @@
#!/usr/bin/env bash
# update.sh — multi-agent-mux-update (Issue #72): update an installed workspace in place.
# Usage:
# bash .agents/skills/multi-agent-mux-update/scripts/update.sh [-y|--yes]
#
# Phase 1 (this script, the installed code): publish .mam/update-in-progress
# under the admission lock (lib.sh mam_update_admit is the other side), refuse
# while anything live could see the tree change (lib_py/update.py precheck, a
# live loop marker, a monitor.lock held by a monitor the skills did not start),
# stop the monitor daemon, back up what the install can change into
# .mam/update-backup/, then exec a copy of install_phase.sh outside the tree.
# That copy runs the installer (MAM_INSTALLER_URL) in place, rolls back on
# failure, and runs phase 2 (post_update.sh of the NEW tree) on success.
#
# Exit codes:
# 0 = updated (registry converted or nothing to convert)
# 1 = refused or failed; what was changed and restored is printed
# 2 = invalid args
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
export WORKSPACE_ROOT="$WS"
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
# shellcheck disable=SC1091
source "$WS/.agents/skills/lib.sh"
# shellcheck disable=SC1091
source "$SKILL_DIR/multi-agent-mux-loop/scripts/loop_lock.sh"
YES=0
while [ $# -gt 0 ]; do
case "$1" in
-y|--yes) YES=1; shift ;;
-h|--help) sed -n '2,4p' "${BASH_SOURCE[0]}"; exit 0 ;;
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
esac
done
MAM="$WS/.mam"
MARKER="$MAM/update-in-progress"
BK="$MAM/update-backup"
PY="$(_delegate_py_bin)"
if [ ! -f "$WS/.mam_deploy/remove.sh" ] && [ ! -f "$WS/remove.sh" ]; then
echo "ERROR: $WS is not an installed MAM workspace (no .mam_deploy/remove.sh); a git checkout updates with git pull" >&2
exit 1
fi
if [ "$YES" != "1" ]; then
if [ ! -t 0 ]; then
echo "ERROR: not a terminal; run with -y/--yes" >&2
exit 1
fi
ans=""
read -r -p "Update MAM in $WS to the latest version? [y/N]: " ans || ans=""
case "$ans" in
y|Y|yes|Yes|YES) ;;
*) echo "Update cancelled."; exit 0 ;;
esac
fi
# --- the marker, published under the exclusive admission lock (decision 8) ---
mkdir -p "$MAM"
exec 7>>"$MAM/update-admission.lock"
case "$("$PY" -m lib_py.pane_lock acquire --fd 7 --wait-env MAM_UPDATE_ADMISSION_WAIT_S)" in
locked) ;;
timeout)
echo "ERROR: a create, resume, loop, delegate submit, stop or orc-onboard kept the admission lock past MAM_UPDATE_ADMISSION_WAIT_S; nothing was changed, try again" >&2
exit 1 ;;
*)
echo "ERROR: the admission lock $MAM/update-admission.lock could not be decided (lib_py.pane_lock); nothing was changed" >&2
exit 1 ;;
esac
if [ -f "$MARKER" ] && [ "$(_mam_lock_holder_state "$MARKER")" = "stale" ]; then
echo "NOTE: stale update marker (pid $(_mam_lock_field "$MARKER" pid)) taken over; its update is gone"
fi
marker_rc=0
mam_acquire_loop_lock "$MARKER" || marker_rc=$?
exec 7>&-
case "$marker_rc" in
0) ;;
3) echo "ERROR: another update is in progress (pid $(_mam_lock_field "$MARKER" pid)); nothing was changed" >&2; exit 1 ;;
*) echo "ERROR: cannot tell whether another update is in progress (ps unavailable?); nothing was changed" >&2; exit 1 ;;
esac
TMPD=""
MONITOR_WAS_RUNNING=0
RESTART_MONITOR=0
_phase1_exit() {
local rc=$?
if [ "$RESTART_MONITOR" = "1" ]; then
_restart_monitor || true
fi
mam_release_loop_lock "$MARKER" || true
[ -z "$TMPD" ] || rm -rf "$TMPD"
exit "$rc"
}
trap _phase1_exit EXIT
_restart_monitor() {
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
return 0
fi
local pid
pid=$(start_watchdog update "$WS") || { echo "ERROR: could not restart the monitor daemon" >&2; return 1; }
echo "monitor: restarted (PID $pid)"
}
_monitor_json() {
"$PY" -m lib_py.update monitor --workspace "$WS"
}
_json_field() {
"$PY" -c 'import json,sys; v=json.load(sys.stdin)[sys.argv[1]]; print(" ".join(v) if isinstance(v, list) else v)' "$1"
}
# --- pre-check: refuse before any change (decisions 1, 7; U3) ---
BLOCKERS=""
_block() { BLOCKERS="${BLOCKERS} - $1
"; }
if [ -f "$MAM/loop-guard-active" ]; then
loop_st="$(_mam_lock_holder_state "$MAM/loop-guard-active")"
case "$loop_st" in
stale) echo "NOTE: stale loop marker $MAM/loop-guard-active (its loop is gone); not a reason to refuse" ;;
*) _block "a review loop is running (pid $(_mam_lock_field "$MAM/loop-guard-active" pid), holder $loop_st); wait for it to end" ;;
esac
fi
STATE_JSON=""
if ! STATE_JSON="$(load_state_json strict)"; then
_block "the registry $AGENT_SESSIONS_YAML cannot be read; fix it first"
STATE_JSON='{}'
fi
REAL_HERDR="$(_resolve_real_herdr_path || true)"
if [ -z "$REAL_HERDR" ]; then
_block "no herdr binary found on PATH, so the live agents cannot be listed"
else
pre_out=""
pre_rc=0
pre_out="$(MAM_STATE_JSON="$STATE_JSON" "$PY" -m lib_py.update precheck --workspace "$WS" --herdr "$REAL_HERDR")" || pre_rc=$?
if [ "$pre_rc" -ne 0 ]; then
while IFS= read -r line; do
[ -n "$line" ] && _block "$line"
done <<< "${pre_out:-lib_py.update precheck failed (rc $pre_rc)}"
fi
fi
MON_JSON="$(_monitor_json)" || { _block "cannot inspect the monitor daemon (lib_py.update monitor)"; MON_JSON='{"pids": [], "lock": "free"}'; }
MON_PIDS="$(printf '%s' "$MON_JSON" | _json_field pids)"
MON_LOCK="$(printf '%s' "$MON_JSON" | _json_field lock)"
if [ -z "$MON_PIDS" ] && [ "$MON_LOCK" = "held" ]; then
_block "$MAM/monitor.lock is held by a monitor the skills did not start (not 'bash $WS/.agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh --subscribe'); stop it (find it with: pgrep -af reconcile.sh) and try again"
fi
if [ -n "$BLOCKERS" ]; then
echo "ERROR: update refused; nothing was changed:" >&2
printf '%s' "$BLOCKERS" >&2
exit 1
fi
# --- stop the monitor daemon (decision 2) ---
_monitor_gone() {
local p
for p in "$@"; do
kill -0 "$p" 2>/dev/null && return 1
done
[ "$(_monitor_json | _json_field lock)" = "free" ]
}
_wait_monitor_gone() {
local deadline=$(( $(date +%s) + $1 ))
shift
while :; do
_monitor_gone "$@" && return 0
[ "$(date +%s)" -lt "$deadline" ] || return 1
sleep 0.5
done
}
if [ -n "$MON_PIDS" ]; then
MONITOR_WAS_RUNNING=1
# shellcheck disable=SC2086
set -- $MON_PIDS
kids=""
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
echo "monitor: stopping the daemon (PID $MON_PIDS)"
kill -TERM "$@" 2>/dev/null || true
if ! _wait_monitor_gone "$(mam_int_knob MAM_UPDATE_MONITOR_STOP_S 30 1)" "$@"; then
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
echo "WARNING: the monitor did not stop within MAM_UPDATE_MONITOR_STOP_S; sending SIGKILL to $* $kids" >&2
# shellcheck disable=SC2086
kill -KILL "$@" $kids 2>/dev/null || true
# shellcheck disable=SC2086
if _wait_monitor_gone 5 "$@" $kids; then
echo "ERROR: the monitor did not stop in time; it was killed and is started again. Nothing else was changed." >&2
_restart_monitor || true
exit 1
fi
echo "ERROR: could not restore the monitor state: PIDs still alive or monitor.lock still held ($*$kids; lock $(_monitor_json | _json_field lock)). Nothing else was changed." >&2
exit 1
fi
RESTART_MONITOR=1
fi
TMPD="$(mktemp -d "${TMPDIR:-/tmp}/mam-update.XXXXXX")"
cp "$SCRIPT_DIR/install_phase.sh" "$TMPD/install_phase.sh"
# --- a leftover backup of an update that died (U4) ---
if [ -e "$BK" ]; then
left_st="$(cat "$BK/state" 2>/dev/null || true)"
case "$left_st" in
""|backing-up|installed)
echo "NOTE: removing the backup a previous update left (state ${left_st:-none}); nothing to restore"
rm -rf "$BK" ;;
installing)
echo "NOTE: a previous update died during its install; restoring its backup (the registry is left as it is)"
if ! bash "$TMPD/install_phase.sh" restore "$WS" --skip-registry; then
RESTART_MONITOR=0
echo "ERROR: restoring the previous update's backup failed; it is kept at $BK. Nothing more was changed; the monitor stays stopped." >&2
exit 1
fi
rm -rf "$BK"
echo "the previous update was rolled back" ;;
*)
echo "ERROR: $BK has an unknown state '$left_st'; it is kept and nothing was changed. Check it and remove it by hand." >&2
exit 1 ;;
esac
fi
# --- the rollback backup (decisions 3, 10; U4) ---
_set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
ITEMS=".agents .mam_deploy AGENTS.md MESSAGING.md BOOTSTRAP.md BOOTSTRAP.ko.md .mam.env.example .mam.env .gitignore remove.sh update.sh .venv .mam/install_manifest.txt .mam/install_state .mam/asset_hashes.txt .mam/version.txt .mam/base .mam/skill-backups"
_backup() {
mkdir -p "$BK/files" || return 1
_set_state backing-up || return 1
: > "$BK/items" && : > "$BK/absent" || return 1
local rel
for rel in $ITEMS; do
printf '%s\n' "$rel" >> "$BK/items" || return 1
if [ -e "$WS/$rel" ] || [ -L "$WS/$rel" ]; then
mkdir -p "$BK/files/$(dirname "$rel")" && cp -pPR "$WS/$rel" "$BK/files/$rel" || return 1
else
printf '%s\n' "$rel" >> "$BK/absent" || return 1
fi
done
"$PY" -m lib_py.update backup-registry --workspace "$WS" --dest "$BK/registry" >/dev/null
}
echo "backup: $BK"
if ! _backup; then
rm -rf "$BK"
echo "ERROR: the backup could not be made; nothing was changed" >&2
exit 1
fi
# --- hand over to the copy (decision 5): same pid, so the marker stays live ---
RESTART_MONITOR=0
trap - EXIT
exec bash "$TMPD/install_phase.sh" run "$WS" "$MONITOR_WAS_RUNNING" "$TMPD"
+3 -2
View File
@@ -38,7 +38,7 @@ bash deploy/install.sh
- `MQTT_BROKER=localhost` 로 된 초기 `.mam.env` 를 씁니다. 이미 있는 `.mam.env` 는 그대로 둡니다. 예전 `.env` 만 있으면 그것을 `.mam.env` 로 복사하고 원본은 남깁니다.
- `.venv`, `.mam/`, `.mam.env` 를 `.gitignore` 에 추가합니다.
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트와 제거는 `deploy/update.sh`, `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트는 `multi-agent-mux-update` 스킬, 제거는 `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
---
@@ -56,6 +56,7 @@ bash deploy/install.sh
| **`multi-agent-mux-delegate-job`** | 세션(`herdr:<name>`)이나 다른 호스트의 러너(`remote:<name>`)에 잡을 위임하고 결과를 기다립니다. [위임 계약](#-위임-계약)을 보십시오. |
| **`multi-agent-mux-loop`** | 계획·구현·리뷰: 선택적인 planner 와 creator, 그다음 선택한 리뷰어(`--reviewer A,B`, 또는 `--all-reviewer` 로 creator 를 뺀 실행 중인 리뷰어 전원)가 모두 `PASS` 를 낼 때까지(또는 `--max-loop` 까지). 둘 다 없으면 creator 가 스스로 리뷰합니다. 루프 밖에서 끝난 작업은 `--review-only` 로 리뷰합니다. 통과한 리포트는 `.agents/reports/<session>/` 으로 옮겨 보관합니다. |
| **`multi-agent-mux-orc-onboard`** | 오케스트레이터 자신의 세션 ID 를 등록해, 세션 탐색이 그것을 하위 에이전트로 착각하지 않게 합니다. |
| **`multi-agent-mux-update`** | 설치된 작업 공간을 제자리에서 업데이트합니다. 실행 중인 세션·열린 잡·루프가 있으면 거부하고, 모니터를 멈췄다가 원래대로 되돌리며, 설치가 실패하면 되돌리고, 새 코드로 레지스트리를 변환합니다. |
---
@@ -195,7 +196,7 @@ sequenceDiagram
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
│ └── multi-agent-mux-*/ # 스킬마다 디렉터리 하나 (SKILL.md + scripts/)
├── .mam/ # 런타임 상태 (git 제외): 세션, 잡, 감사 로그, shim
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
├── docs/ # contracts/, design/, 에이전트 통합·브리지 가이드
├── nats-docker/ # 사설 NATS 브로커 배포 (서브모듈)
├── tests/ # pytest 스위트
+3 -2
View File
@@ -38,7 +38,7 @@ bash deploy/install.sh
- writes a starter `.mam.env` with `MQTT_BROKER=localhost`. An existing `.mam.env` is kept. If only a legacy `.env` exists, it is copied to `.mam.env` and the original stays;
- adds `.venv`, `.mam/` and `.mam.env` to `.gitignore`.
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). `deploy/update.sh` and `deploy/remove.sh` update and uninstall. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). The `multi-agent-mux-update` skill updates an installed workspace, and `deploy/remove.sh` uninstalls. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
---
@@ -56,6 +56,7 @@ Every skill lives under `.agents/skills/` with a `SKILL.md` that documents its o
| **`multi-agent-mux-delegate-job`** | Delegates a job to a session (`herdr:<name>`) or to a runner on another host (`remote:<name>`) and waits for its result. See [Delegation contract](#-delegation-contract). |
| **`multi-agent-mux-loop`** | Plan, implement, review: an optional planner and a creator, then the selected reviewers (`--reviewer A,B`, or `--all-reviewer` for every running reviewer except the creator) until all of them `PASS` or `--max-loop` is reached. With neither option the creator reviews its own work. `--review-only` reviews work finished outside the loop. Passing reports are promoted to `.agents/reports/<session>/`. |
| **`multi-agent-mux-orc-onboard`** | Registers the orchestrator's own session id, so session discovery never mistakes it for a sub-agent. |
| **`multi-agent-mux-update`** | Updates an installed workspace in place: refuses while sessions, open jobs or a loop are live; stops and restores the monitor; rolls back on a failed install; converts the registry with the new code. |
---
@@ -195,7 +196,7 @@ sequenceDiagram
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
│ └── multi-agent-mux-*/ # One directory per skill (SKILL.md + scripts/)
├── .mam/ # Runtime state (git-ignored): sessions, jobs, audit logs, shim
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
├── docs/ # contracts/, design/, agent integration and bridge guides
├── nats-docker/ # Private NATS broker deployment (submodule)
├── tests/ # pytest suite
+1 -1
View File
@@ -60,7 +60,7 @@ $ bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh \
### 2) 세션 접속 (Attach)
백그라운드에서 구동된 에이전트 TUI 화면에 들어갑니다.
```bash
$ herdr session attach my-project-dev-claude
$ herdr --session multi-agent-mux agent attach my-project-dev-claude
```
* **화면 탈출**: 대화 중 세션을 유지한 채 터미널로 돌아오려면 `Ctrl + B`를 누른 뒤 `D` 키를 차례로 입력합니다.
+3 -3
View File
@@ -9,7 +9,7 @@ This directory contains packaging templates and installation scripts to deploy t
* **`install.sh`**: A self-contained, idempotent remote shell installer (via curl) that checks system requirements (`herdr`, `python3`), sets up a local python virtual environment (`.venv`), and performs 3-way refresh with key-level registry merge (Rev.2).
* **`install_mam.sh`**: A local-clone installer that copies rules/skills (`.agents/`), `AGENTS.md`, manifests, and asset hashes into target projects.
* **`lib_ownership.sh`**: Single source of truth for framework-owned files and key-level registry files.
* **`update.sh`**: In-place updater script installed into target `.mam_deploy/update.sh`.
* **Updates**: installed workspaces update with the `multi-agent-mux-update` skill (`bash .agents/skills/multi-agent-mux-update/scripts/update.sh`).
* **`remove.sh`**: Clean uninstaller script installed into target `.mam_deploy/remove.sh`.
* **`generate-env.sh`**: Environment configuration bootstrap helper.
* **`INSTALL.md`**: Detailed installation and quick-start user manual.
@@ -40,7 +40,7 @@ bash deploy/install_mam.sh --target /path/to/your/project
Refer to **`INSTALL.md`** inside this directory for the full instructions and workflows.
> [!NOTE]
> `install_mam.sh` automatically deploys `.mam_deploy/update.sh` and `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
> `install_mam.sh` automatically deploys `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
### 3. Custom Fork / Private Mirror Installations
If you run a private mirror or fork, you can override the source URLs during installation using environment variables:
@@ -53,7 +53,7 @@ curl -fsSL https://my-mirror.example.com/.../install.sh \
bash
# Updating an existing workspace against a mirror
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash deploy/update.sh
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash .agents/skills/multi-agent-mux-update/scripts/update.sh
```
### 4. Registering as a Workspace Plugin
+1 -1
View File
@@ -41,7 +41,7 @@ jobs:
shellcheck deploy/install.sh
shellcheck deploy/install_mam.sh
shellcheck deploy/generate-env.sh
shellcheck deploy/update.sh
shellcheck .agents/skills/multi-agent-mux-update/scripts/*.sh
shellcheck deploy/remove.sh
echo "✅ ShellCheck completed successfully."
+68 -18
View File
@@ -125,7 +125,9 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
FETCH_METHOD="archive"
if [ -d "$REPO_URL" ]; then
echo "🌐 Copying local working tree into a staging area..."
cp -R "$REPO_URL/." "$STAGE_DIR/"
# Everything but .mam/: runtime state is never staged, and it may hold the
# update skill's rollback copy (.mam/update-backup/, Issue #72).
find "$REPO_URL/." -mindepth 1 -maxdepth 1 ! -name .mam -exec cp -R {} "$STAGE_DIR/" \;
FETCH_METHOD="local"
elif command -v git &>/dev/null; then
echo "🌐 Cloning repository (shallow) into a staging area..."
@@ -157,9 +159,16 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
MANIFEST_FILE=".mam/install_manifest.txt"
touch "$MANIFEST_FILE"
# Migrate legacy root layout for remove.sh and update.sh into .mam_deploy/ if owned
# Drop every line naming this path from the manifest
manifest_drop() {
grep -Fvx -- "$1" "$MANIFEST_FILE" > "$MANIFEST_FILE.tmp" || true
mv -f "$MANIFEST_FILE.tmp" "$MANIFEST_FILE"
}
# Migrate legacy root layout for remove.sh into .mam_deploy/ if owned
mkdir -p .mam_deploy
for legacy_script in remove.sh update.sh; do
# shellcheck disable=SC2043 # one legacy script left (update.sh went with Issue #72)
for legacy_script in remove.sh; do
if [ -f "$legacy_script" ] && grep -Fqx "$legacy_script" "$MANIFEST_FILE" 2>/dev/null; then
mv -f "$legacy_script" ".mam_deploy/$legacy_script"
python3 -c '
@@ -179,6 +188,13 @@ with open(path, "w") as f:
fi
done
# update.sh is no longer shipped (Issue #72; the multi-agent-mux-update skill
# updates). A legacy root update.sh goes only if the manifest says we put it there.
if [ -f update.sh ] && grep -Fqx update.sh "$MANIFEST_FILE" 2>/dev/null; then
rm -f update.sh
manifest_drop update.sh
fi
# Safe refresh & fingerprint checking logic
TS=$(date -u +%Y%m%dT%H%M%SZ)
FRAMEWORK_LEDGER=".mam/.framework_ledger.tmp"
@@ -374,7 +390,7 @@ MERGEREG
fi
done
# Install remove.sh and update.sh into .mam_deploy/
# Install remove.sh into .mam_deploy/
mkdir -p .mam_deploy
if [ -f "$STAGE_DIR/deploy/remove.sh" ]; then
cp "$STAGE_DIR/deploy/remove.sh" .mam_deploy/remove.sh
@@ -384,13 +400,9 @@ MERGEREG
fi
fi
if [ -f "$STAGE_DIR/deploy/update.sh" ]; then
cp "$STAGE_DIR/deploy/update.sh" .mam_deploy/update.sh
chmod 0755 .mam_deploy/update.sh
if ! grep -Fqx ".mam_deploy/update.sh" "$MANIFEST_FILE" 2>/dev/null; then
echo ".mam_deploy/update.sh" >> "$MANIFEST_FILE"
fi
fi
# An older install's updater is not shipped any more (Issue #72)
rm -f .mam_deploy/update.sh
manifest_drop .mam_deploy/update.sh
if [ -f "$STAGE_DIR/.mam.env.example" ] && [ ! -e ".mam.env.example" ]; then
cp "$STAGE_DIR/.mam.env.example" .
@@ -407,6 +419,34 @@ MERGEREG
fi
fi
# Framework-owned files the previous install recorded (manifest and
# asset_hashes.txt) that this version no longer ships (Issue #72). Unmodified
# ones are deleted. Modified ones are kept and become the user's file: they
# leave the manifest, so remove.sh leaves them alone, and asset_hashes.txt
# (rebuilt below from the shipped files only).
if [ -f .mam/asset_hashes.txt ]; then
SHIPPED_LIST=".mam/.shipped_paths.tmp"
cut -f2 "$FRAMEWORK_LEDGER" > "$SHIPPED_LIST"
while read -r _old_sha _old_path; do
[ -n "${_old_path:-}" ] || continue
is_framework_owned "$_old_path" || continue
grep -Fqx -- "$_old_path" "$MANIFEST_FILE" || continue
if grep -Fqx -- "$_old_path" "$SHIPPED_LIST"; then continue; fi
if [ -f "$_old_path" ] && [ "$(python3 -c 'import hashlib, sys; print(hashlib.sha256(open(sys.argv[1], "rb").read()).hexdigest())' "$_old_path")" != "$_old_sha" ]; then
echo "ℹ️ Local modification detected (no longer shipped, kept as yours): $_old_path" >&2
else
rm -f "$_old_path"
_old_dir="$(dirname "$_old_path")"
while [ "$_old_dir" != ".agents" ] && rmdir "$_old_dir" 2>/dev/null; do
_old_dir="$(dirname "$_old_dir")"
done
fi
rm -f ".mam/base/$_old_path"
manifest_drop "$_old_path"
done < .mam/asset_hashes.txt
rm -f "$SHIPPED_LIST"
fi
# Re-build asset_hashes.txt from the UPSTREAM copy of every framework-owned file
python3 - .mam/asset_hashes.txt "$FRAMEWORK_LEDGER" <<'HASHDB'
import os, hashlib, sys
@@ -590,15 +630,25 @@ with open(gi_path, "w") as f:
PY
if [ "${MAM_SKIP_GITIGNORE:-0}" -eq 0 ]; then
# Set the key: overwrite it in place, one line per key, other keys kept
python3 -c '
import os
os.makedirs(".mam", exist_ok=True)
lines = []
if os.path.exists(".mam/install_state"):
with open(".mam/install_state", "a") as f:
f.write("gitignore_created=1\n")
else:
os.makedirs(".mam", exist_ok=True)
with open(".mam/install_state", "w") as f:
f.write("gitignore_created=1\n")
with open(".mam/install_state") as f:
lines = f.read().splitlines()
out, done = [], False
for line in lines:
if line.startswith("gitignore_created="):
if done:
continue
line, done = "gitignore_created=1", True
out.append(line)
if not done:
out.append("gitignore_created=1")
with open(".mam/install_state", "w") as f:
f.write("\n".join(out) + "\n")
'
fi
@@ -617,6 +667,6 @@ cat <<EOF
$ bash .agents/skills/multi-agent-mux-status/scripts/status.sh
3. Update installed skills in the future:
$ bash .mam_deploy/update.sh
$ bash .agents/skills/multi-agent-mux-update/scripts/update.sh
--------------------------------------------------------------------------------
EOF
+3 -7
View File
@@ -132,16 +132,12 @@ if [ -f "$SRC_DIR/.mam.env.example" ]; then
log_ok "Copied .mam.env.example configuration template"
fi
# Deploy remove.sh and update.sh into .mam_deploy/ (R-3)
# Deploy remove.sh into .mam_deploy/ (R-3)
mkdir -p "$TARGET_DIR/.mam_deploy"
if [ -f "$SRC_DIR/deploy/remove.sh" ]; then
cp "$SRC_DIR/deploy/remove.sh" "$TARGET_DIR/.mam_deploy/remove.sh"
chmod 0755 "$TARGET_DIR/.mam_deploy/remove.sh"
fi
if [ -f "$SRC_DIR/deploy/update.sh" ]; then
cp "$SRC_DIR/deploy/update.sh" "$TARGET_DIR/.mam_deploy/update.sh"
chmod 0755 "$TARGET_DIR/.mam_deploy/update.sh"
fi
if [ -f "$SRC_DIR/deploy/generate-env.sh" ]; then
mkdir -p "$TARGET_DIR/scripts"
@@ -161,7 +157,7 @@ mkdir -p "$TARGET_DIR/.mam"
MANIFEST_FILE="$TARGET_DIR/.mam/install_manifest.txt"
: > "$MANIFEST_FILE"
( cd "$TARGET_DIR" && find .agents -type f -print ) >> "$MANIFEST_FILE"
for extra in ".mam.env.example" ".mam_deploy/remove.sh" ".mam_deploy/update.sh" \
for extra in ".mam.env.example" ".mam_deploy/remove.sh" \
"scripts/generate-env.sh"; do
if [ -e "$TARGET_DIR/$extra" ]; then
echo "$extra" >> "$MANIFEST_FILE"
@@ -327,7 +323,7 @@ cat <<EOF
--herdr-server multi-agent-mux
2. Attach to the running session:
$ HERDR_SERVER_NAME=multi-agent-mux herdr agent attach <session_name>
$ herdr --session multi-agent-mux agent attach <herdr name>
3. Gracefully stop the session:
$ bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh \\
+1
View File
@@ -92,6 +92,7 @@ else
".agents/skills/multi-agent-mux-resume"
".agents/skills/multi-agent-mux-status"
".agents/skills/multi-agent-mux-stop"
".agents/skills/multi-agent-mux-update"
".agents/hooks"
".agents/hooks.json"
".agents/MULTI_AGENT_RULES.md"
-269
View File
@@ -1,269 +0,0 @@
#!/usr/bin/env bash
# ==============================================================================
# update.sh — Multi-Agent Mux (MAM) Orchestration Updater
# ==============================================================================
# Safely updates MAM skills, virtual environment, and docs to the latest version.
# Preserves user configuration (.env) and local metadata/jobs database (.mam).
# ==============================================================================
set -euo pipefail
TARGET_DIR=""
FORCE=0
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Parse arguments
while [[ $# -gt 0 ]]; do
case "$1" in
-y|--yes|--force)
FORCE=1
shift
;;
*)
TARGET_DIR="$1"
shift
;;
esac
done
if [ -z "$TARGET_DIR" ]; then
if [ "$(basename "$SCRIPT_DIR")" = ".mam_deploy" ]; then
TARGET_DIR="$(dirname "$SCRIPT_DIR")"
else
TARGET_DIR="$(pwd)"
fi
else
if [ "$(basename "$TARGET_DIR")" = ".mam_deploy" ]; then
TARGET_DIR="$(dirname "$TARGET_DIR")"
fi
if [ ! -d "$TARGET_DIR" ]; then
echo "❌ Error: Target directory '$TARGET_DIR' does not exist." >&2
exit 1
fi
TARGET_DIR="$(cd "$TARGET_DIR" && pwd)"
fi
echo "===================================================================="
echo "⚡ Starting Multi-Agent Mux (MAM) Update"
echo "📂 Target Workspace: $TARGET_DIR"
echo "===================================================================="
cd "$TARGET_DIR"
# 1. Verification of existing install (B-2: dual resolution)
REMOVER=""
for cand in ".mam_deploy/remove.sh" "remove.sh"; do
if [ -f "$cand" ]; then
REMOVER="$cand"
break
fi
done
if [ -z "$REMOVER" ]; then
echo "❌ Error: No MAM installation (remove.sh) found in '$TARGET_DIR'." >&2
echo " Please run install.sh first to set up the workspace." >&2
exit 1
fi
# Request confirmation if not forced
if [ $FORCE -eq 0 ]; then
echo "⚠️ WARNING: This will update MAM orchestration skills, virtual environment, "
echo " and docs to the latest version."
echo " (Your configuration, job history, and custom skills will be preserved)."
if [ ! -t 0 ]; then
echo "❌ Error: Non-interactive terminal detected. Please run with -y/--yes/--force." >&2
exit 1
fi
if ! read -p "❓ Proceed with update? [y/N]: " -r response; then
response="n"
fi
if [[ ! "$response" =~ ^[yY](es)?$ ]]; then
echo "❌ Update cancelled by user."
exit 0
fi
fi
# 1.5 Pre-capture legacy ownership before remove.sh deletes the manifest (M-4)
MAM_LEGACY_ENV_OWNED=0
if [ -f ".mam/install_manifest.txt" ] && grep -Fqx ".env" ".mam/install_manifest.txt" 2>/dev/null; then
MAM_LEGACY_ENV_OWNED=1
fi
export MAM_LEGACY_ENV_OWNED
# 2. Stage backups of user configurations and metadata to prevent deletion
echo "💾 Backing up configuration and database..."
HAS_ENV=0
ENV_BACKUP_SRC=""
ENV_BACKUP_TMP=""
if [ -f ".mam.env" ]; then
HAS_ENV=1
ENV_BACKUP_SRC=".mam.env"
ENV_BACKUP_TMP=".mam.env.update-tmp"
mv ".mam.env" "$ENV_BACKUP_TMP"
elif [ -f ".env" ]; then
HAS_ENV=1
ENV_BACKUP_SRC=".env"
ENV_BACKUP_TMP=".env.update-tmp"
mv ".env" "$ENV_BACKUP_TMP"
fi
HAS_MAM=0
if [ -d ".mam" ]; then
HAS_MAM=1
# Copy database and jobs to temporary backup outside of .mam.
# We do NOT move the .mam folder away so that remove.sh can still read .mam/install_manifest.txt!
mkdir -p .mam.update-tmp
# Copy SQLite databases and session files
for db in .mam/agent-sessions.*; do
if [ -f "$db" ]; then
cp -f "$db" .mam.update-tmp/
fi
done
# Copy jobs history
if [ -d ".mam/jobs" ] && [ "$(ls -A .mam/jobs 2>/dev/null)" ]; then
mkdir -p .mam.update-tmp/jobs
cp -rf .mam/jobs/* .mam.update-tmp/jobs/
fi
# Copy delegate logs
if [ -d ".mam/delegate_job_logs" ] && [ "$(ls -A .mam/delegate_job_logs 2>/dev/null)" ]; then
mkdir -p .mam.update-tmp/delegate_job_logs
cp -rf .mam/delegate_job_logs/* .mam.update-tmp/delegate_job_logs/
fi
# Copy manifest so we have a backup
if [ -f ".mam/install_manifest.txt" ]; then
cp -f .mam/install_manifest.txt .mam.update-tmp/
fi
# C12: Copy MAM state files across update cycle
for st in install_state asset_hashes.txt version.txt; do
if [ -f ".mam/$st" ]; then
cp -f ".mam/$st" .mam.update-tmp/
fi
done
if [ -d ".mam/skill-backups" ]; then
mkdir -p .mam.update-tmp/skill-backups
cp -rf .mam/skill-backups/* .mam.update-tmp/skill-backups/ 2>/dev/null || true
fi
fi
# Define trap to restore backup files on failure
restore_on_failure() {
echo "❌ Update failed. Reverting configuration and database to previous state..."
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC" 2>/dev/null || true
fi
if [ $HAS_MAM -eq 1 ] && [ -d ".mam.update-tmp" ]; then
# Revert to old database/jobs backup by restoring .mam directory
rm -rf .mam 2>/dev/null || true
mkdir -p .mam
cp -f .mam.update-tmp/agent-sessions.* .mam/ 2>/dev/null || true
if [ -d ".mam.update-tmp/jobs" ]; then
cp -rf .mam.update-tmp/jobs .mam/ 2>/dev/null || true
fi
if [ -d ".mam.update-tmp/delegate_job_logs" ]; then
cp -rf .mam.update-tmp/delegate_job_logs .mam/ 2>/dev/null || true
fi
if [ -f ".mam.update-tmp/install_manifest.txt" ]; then
cp -f .mam.update-tmp/install_manifest.txt .mam/ 2>/dev/null || true
fi
for st in install_state asset_hashes.txt version.txt; do
if [ -f ".mam.update-tmp/$st" ]; then
cp -f ".mam.update-tmp/$st" .mam/ 2>/dev/null || true
fi
done
if [ -d ".mam.update-tmp/skill-backups" ]; then
mkdir -p .mam/skill-backups
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
fi
rm -rf .mam.update-tmp 2>/dev/null || true
fi
}
trap restore_on_failure EXIT
# 3. Perform uninstallation of existing files
echo "🗑️ Removing existing installation..."
# remove.sh will run in manifest mode because .mam/install_manifest.txt is still present.
# It will delete .agents/, documents, scripts, .venv, and .mam folder.
bash "$REMOVER" --force "$TARGET_DIR"
# 4. Fetch and run the latest installer from Gitea
echo "📥 Fetching and running the latest installer..."
# Note: MAM_*_URL environment variables are automatically inherited by the child installer bash process.
INSTALLER_URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
if command -v curl &>/dev/null; then
curl -fsSL "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
elif command -v wget &>/dev/null; then
wget -qO- "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
else
echo "❌ Error: Neither 'curl' nor 'wget' is available to fetch the installer." >&2
exit 1
fi
# Disable failure trap since installation succeeded
trap - EXIT
# 5. Restore backups of configuration and database
echo "🔄 Restoring configuration and database..."
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
if [ "$MAM_LEGACY_ENV_OWNED" = "1" ] && [ "$ENV_BACKUP_SRC" = ".env" ] && [ ! -f ".mam.env" ]; then
mv -f "$ENV_BACKUP_TMP" ".mam.env"
chmod 0600 ".mam.env" 2>/dev/null || true
else
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC"
fi
fi
if [ $HAS_MAM -eq 1 ]; then
if [ -d ".mam.update-tmp" ]; then
# The installer created a new .mam directory with a new manifest.
# We want to merge the old .mam database/jobs back while keeping the new manifest.
for db in .mam.update-tmp/agent-sessions.*; do
if [ -f "$db" ]; then
cp -f "$db" .mam/
fi
done
if [ -d ".mam.update-tmp/jobs" ] && [ "$(ls -A .mam.update-tmp/jobs 2>/dev/null)" ]; then
mkdir -p .mam/jobs
cp -rf .mam.update-tmp/jobs/* .mam/jobs/
fi
if [ -d ".mam.update-tmp/delegate_job_logs" ] && [ "$(ls -A .mam.update-tmp/delegate_job_logs 2>/dev/null)" ]; then
mkdir -p .mam/delegate_job_logs
cp -rf .mam.update-tmp/delegate_job_logs/* .mam/delegate_job_logs/
fi
# Issue #8: asset_hashes.txt and version.txt describe the tree the
# installer just wrote, so the pre-update copies must NOT come back here -
# restoring them made version.txt name the previous commit and remove.sh
# treat every updated file as locally modified (spurious backups). Only
# install_state carries history worth keeping: merge in any key the fresh
# install did not write (e.g. gitignore_created from the original install).
# The failure path (restore_on_failure) still puts all three back verbatim.
if [ -f ".mam.update-tmp/install_state" ]; then
touch .mam/install_state
# A fresh install_state without a final newline would glue the first
# merged key onto its last line.
if [ -s .mam/install_state ] && [ -n "$(tail -c1 .mam/install_state)" ]; then
printf '\n' >> .mam/install_state
fi
while IFS= read -r line || [ -n "$line" ]; do
case "$line" in
*=*) ;;
*) continue ;; # not a key=value line
esac
key="${line%%=*}"
[ -n "$key" ] || continue
# exact key match, no regex interpretation of the key
if ! awk -F= -v k="$key" '$1 == k { found = 1 } END { exit !found }' .mam/install_state; then
printf '%s\n' "$line" >> .mam/install_state
fi
done < .mam.update-tmp/install_state
fi
if [ -d ".mam.update-tmp/skill-backups" ]; then
mkdir -p .mam/skill-backups
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
fi
rm -rf ".mam.update-tmp"
fi
fi
echo "===================================================================="
echo "🎉 Update complete!"
echo "===================================================================="
+30 -16
View File
@@ -11,7 +11,7 @@
"kind": "example",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:99",
"multi-agent-mux-create/scripts/create_session.sh:356"
"multi-agent-mux-create/scripts/create_session.sh:359"
],
"mismatch": null,
"note": "HERDR_SESSION_NAME environment example"
@@ -67,7 +67,7 @@
"class": "c",
"kind": "example",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:514-541"
"multi-agent-mux-create/scripts/create_session.sh:517-544"
],
"mismatch": null,
"note": "Output format: the row create_session.sh writes"
@@ -122,7 +122,7 @@
"class": "c",
"kind": "invocation",
"evidence": [
"multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:1046-1057"
"multi-agent-mux-delegate-job/multi-agent-mux-delegate-job:1051-1062"
],
"mismatch": null,
"note": "CLI reference; subcommands submit/status/list/verify/wait/logs match main()"
@@ -433,7 +433,7 @@
"class": "c",
"kind": "diagram",
"evidence": [
"multi-agent-mux-stop/scripts/stop_session.sh:245-259"
"multi-agent-mux-stop/scripts/stop_session.sh:250-264"
],
"mismatch": null,
"note": "state machine"
@@ -463,6 +463,20 @@
"evidence": [],
"mismatch": null,
"note": "post-stop checks; no script offers them as a command - stage 3"
},
{
"skill": "multi-agent-mux-update",
"start": 17,
"end": 19,
"lang": "bash",
"sha": "63f73ade58d3",
"class": "c",
"kind": "invocation",
"evidence": [
"multi-agent-mux-update/scripts/update.sh:3"
],
"mismatch": null,
"note": "entry point call (Issue #72)"
}
],
"removed": [
@@ -476,11 +490,11 @@
"evidence": [
"multi-agent-mux-resume/scripts/resume_session.sh:111-130",
"multi-agent-mux-resume/scripts/resume_session.sh:133-144",
"multi-agent-mux-resume/scripts/resume_session.sh:147-158",
"multi-agent-mux-resume/scripts/resume_session.sh:172-197",
"multi-agent-mux-resume/scripts/resume_session.sh:245",
"multi-agent-mux-resume/scripts/resume_session.sh:249",
"multi-agent-mux-resume/scripts/resume_session.sh:255-258",
"multi-agent-mux-resume/scripts/resume_session.sh:147-170",
"multi-agent-mux-resume/scripts/resume_session.sh:188-213",
"multi-agent-mux-resume/scripts/resume_session.sh:265",
"multi-agent-mux-resume/scripts/resume_session.sh:269",
"multi-agent-mux-resume/scripts/resume_session.sh:275-278",
"lib_py/agents/adapters/claude.py:109"
],
"mismatch": [
@@ -504,8 +518,8 @@
"class": "a",
"evidence": [
"multi-agent-mux-stop/scripts/stop_session.sh:101",
"multi-agent-mux-stop/scripts/stop_session.sh:123-193",
"multi-agent-mux-stop/scripts/stop_session.sh:245-249"
"multi-agent-mux-stop/scripts/stop_session.sh:128-198",
"multi-agent-mux-stop/scripts/stop_session.sh:250-254"
],
"mismatch": [
"T1: the doc offered --force-no-yaml, which no script accepts"
@@ -547,8 +561,8 @@
"multi-agent-mux-create/scripts/create_session.sh:166-168",
"multi-agent-mux-create/scripts/create_session.sh:207",
"multi-agent-mux-create/scripts/create_session.sh:259-274",
"multi-agent-mux-create/scripts/create_session.sh:361-398",
"multi-agent-mux-create/scripts/create_session.sh:410-412"
"multi-agent-mux-create/scripts/create_session.sh:364-401",
"multi-agent-mux-create/scripts/create_session.sh:413-415"
],
"mismatch": [
"C1: an existing session is exit 1 in the doc, exit 3 in the script",
@@ -568,7 +582,7 @@
"base_sha": "f71be0f26fb8",
"class": "a",
"evidence": [
"multi-agent-mux-create/scripts/create_session.sh:486-604"
"multi-agent-mux-create/scripts/create_session.sh:489-607"
],
"mismatch": [],
"replacement": "bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh --workspace <path> --agent <agent> --role <role>",
@@ -601,7 +615,7 @@
"other"
],
"passthrough": {
"1": "multi-agent-mux-stop/scripts/stop_session.sh:384"
"1": "multi-agent-mux-stop/scripts/stop_session.sh:389"
}
},
"multi-agent-mux-create/scripts/create_session.sh": {
@@ -619,7 +633,7 @@
"other"
],
"passthrough": {
"4": "multi-agent-mux-create/scripts/create_session.sh:546"
"4": "multi-agent-mux-create/scripts/create_session.sh:549"
}
}
},
+19 -7
View File
@@ -48,7 +48,7 @@ class TestDeployLayout(unittest.TestCase):
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".agents", "skills", "lib.sh")))
def test_td6_td7_td8_mam_deploy_layout_and_removal(self):
"""T-D6~D8: Verify remove.sh/update.sh installed in .mam_deploy and work properly."""
"""T-D6~D8: Verify remove.sh installed in .mam_deploy (update.sh not, Issue #72) and works properly."""
subprocess.run(
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
env=self.env,
@@ -59,7 +59,7 @@ class TestDeployLayout(unittest.TestCase):
remover = os.path.join(self.test_dir, ".mam_deploy", "remove.sh")
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
self.assertTrue(os.path.exists(remover))
self.assertTrue(os.path.exists(updater))
self.assertFalse(os.path.exists(updater))
self.assertTrue(os.access(remover, os.X_OK))
# T-D8: Execution from inside .mam_deploy directory targets parent workspace
@@ -129,17 +129,29 @@ class TestDeployLayout(unittest.TestCase):
self.assertTrue(os.path.exists(backups_dir))
def test_td27_td28_update_preserves_mam_state(self):
"""T-D27~D28: Verify update.sh preserves .mam state files across cycles (B-5 gate)."""
"""T-D27~D28: Verify the update skill preserves .mam state files across cycles (B-5 gate)."""
subprocess.run(
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
env=self.env,
check=True
)
# Run update.sh
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
res = subprocess.run(["bash", updater, "--force"], cwd=self.test_dir, env=self.env, capture_output=True, text=True)
self.assertEqual(res.returncode, 0, f"update.sh failed: {res.stderr}")
# Run the update skill (Issue #72: .mam_deploy/update.sh is gone)
updater = os.path.join(self.test_dir, ".agents", "skills", "multi-agent-mux-update", "scripts", "update.sh")
env = {k: v for k, v in self.env.items() if not k.startswith("HERDR_")
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
# The update lists the live herdr agents; never ask a real server here.
fake = os.path.join(self.temp_dir, "fake-herdr-bin")
os.makedirs(fake)
with open(os.path.join(fake, "herdr"), "w") as f:
f.write('#!/bin/sh\necho \'{"sessions": []}\'\n')
os.chmod(os.path.join(fake, "herdr"), 0o755)
env["PATH"] = fake + ":" + env["PATH"]
res = subprocess.run(["bash", updater, "--yes"], cwd=self.test_dir, env=env, capture_output=True, text=True,
stdin=subprocess.DEVNULL)
self.assertEqual(res.returncode, 0, f"the update skill failed: {res.stdout}{res.stderr}")
self.assertFalse(os.path.exists(os.path.join(self.test_dir, ".mam_deploy", "update.sh")))
# Verify asset_hashes.txt and version.txt still exist
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".mam", "asset_hashes.txt")))
+2
View File
@@ -59,6 +59,8 @@ INT_KNOBS = {
# once) and the wrapper's registry wait (0: decide at once - clear or busy)
"MAM_ONBOARD_WAIT_S": (1, "600"),
"MAM_DISPATCH_WAIT_S": (0, "600"),
# Issue #72: how long the update waits for the monitor to stop after SIGTERM
"MAM_UPDATE_MONITOR_STOP_S": (1, "30"),
# delegate_wait_started's ack_s (MAM_ONBOARD_ACK_S / DELEGATE_JOB_ACK_S via create_session.sh)
"delegate_wait_started:ack_s": (0, "180"),
# wrapper flags
+2 -2
View File
@@ -115,8 +115,8 @@ def test_inventory_matches_skill_md(skill):
f"actual (start, end, lang, sha): {json.dumps(actual)}")
def test_inventory_covers_eight_skills():
assert len(SKILL_NAMES) == 8
def test_inventory_covers_nine_skills():
assert len(SKILL_NAMES) == 9 # Issue #72: + multi-agent-mux-update
assert {b["skill"] for b in DATA["blocks"]} == set(SKILL_NAMES)
+283
View File
@@ -0,0 +1,283 @@
"""Issue #72 (decision 8, U2): while an update runs, the entries that start
something or write the registry refuse - create, resume, run_loop.sh, delegate
submit, stop (plain and --purge-conversation) and orc-onboard - and change
nothing. A marker its update left behind (`stale`) refuses nothing.
The marker is .mam/update-in-progress as the update skill publishes it (pid +
lstart, loop_lock.sh); `live` here is a sleeping helper process's.
"""
from __future__ import annotations
import hashlib
import json
import os
import subprocess
import sys
from pathlib import Path
import pytest
import yaml
UUID = "11111111-2222-3333-4444-555555555555"
ORC = "99999999-8888-7777-6666-555555555555"
ORC_OLD = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
@pytest.fixture
def marker(mam_sandbox):
procs = []
def make(live: bool, root: Path = None):
p = (root or mam_sandbox) / ".mam" / "update-in-progress"
if live:
proc = subprocess.Popen(["sleep", "120"], start_new_session=True)
procs.append(proc)
ls = " ".join(subprocess.run(["ps", "-p", str(proc.pid), "-o", "lstart="],
capture_output=True, text=True).stdout.split())
p.write_text("pid=%d\nlstart=%s\nstarted=x\n" % (proc.pid, ls))
else:
dead = subprocess.Popen(["true"])
dead.wait()
p.write_text("pid=%d\nlstart=Thu Jan 1 00:00:00 1970\nstarted=x\n" % dead.pid)
return p
yield make
for proc in procs:
proc.kill()
proc.wait()
def _seed(sandbox, rows, **top):
lib = sandbox / ".agents" / "skills" / "lib.sh"
mutation = "d['herdr_sessions'] = %r\n" % (rows,) + "".join("d[%r] = %r\n" % kv for kv in top.items())
res = subprocess.run(["bash", "-c", 'source "$1" && atomic_dump_yaml "$AGENT_SESSIONS_YAML"', "_", str(lib)],
input=mutation, capture_output=True, text=True)
assert res.returncode == 0, res.stderr
def _registry(sandbox):
out = {}
for f in ("agent-sessions.yaml", "agent-sessions.db"):
p = sandbox / ".mam" / f
out[f] = hashlib.sha256(p.read_bytes()).hexdigest() if p.exists() else None
return out
def _script(sandbox, rel):
return str(sandbox / ".agents" / "skills" / rel)
def _run(sandbox, argv, env=None, timeout=120):
return subprocess.run(argv, capture_output=True, text=True, cwd=str(sandbox), timeout=timeout,
env=dict(os.environ, **(env or {})), stdin=subprocess.DEVNULL)
def _refused(res, sandbox):
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s is in progress" % sandbox in res.stderr, res.stdout + res.stderr
def _row(name, status="running", agent="claude"):
r = {"name": name, "status": status, "role": "creator", "herdr_session": "default", "herdr_server": "default",
"pane": {"cwd": "", "cmd": agent}, "agent": agent}
if agent == "claude":
r["claude_session_id_own"] = UUID
return r
def _agent(cwd, pane):
return {"agent": "claude", "status": "running", "cwd": str(cwd), "workspace_id": "w1",
"pid": 9999, "pane_id": pane, "command": "claude", "buffer": "Ready"}
def _set_mock(state_file, **kw):
st = json.loads(Path(state_file).read_text())
st.update(kw)
Path(state_file).write_text(json.dumps(st))
def test_create_refuses_while_an_update_runs(mam_sandbox, mock_herdr, mock_agents, marker):
marker(True)
before = _registry(mam_sandbox)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-create/scripts/create_session.sh"),
"--workspace", str(mam_sandbox), "--agent", "claude", "--role", "creator",
"--session", "gated-create", "--no-onboard"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
def test_resume_refuses_on_the_spawn_and_the_alive_path(mam_sandbox, mock_herdr, mock_agents, marker):
_seed(mam_sandbox, [dict(_row("gated-agy", status="stopped", agent="agy"), pane={"cwd": str(mam_sandbox), "cmd": "agy"})])
marker(True)
before = _registry(mam_sandbox)
script = _script(mam_sandbox, "multi-agent-mux-resume/scripts/resume_session.sh")
res = _run(mam_sandbox, ["bash", script, "--workspace", str(mam_sandbox), "--agent", "agy",
"--session", "gated-agy"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert json.loads(Path(mock_herdr).read_text())["agents"] == {}, "nothing was spawned"
_set_mock(mock_herdr, agents={"gated-agy": _agent(mam_sandbox, "w1:p9")})
res = _run(mam_sandbox, ["bash", script, "--workspace", str(mam_sandbox), "--agent", "agy",
"--session", "gated-agy"], timeout=300)
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
def test_run_loop_refuses_before_its_loop_marker(mam_sandbox, marker):
marker(True)
loop_marker = mam_sandbox / ".mam" / "loop-guard-active"
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-loop/scripts/run_loop.sh"),
"--creator", "c", "--task", "t"],
env={"MAM_LOOP_NO_FREEZE": "1", "MAM_REAL_ROOT": str(mam_sandbox)})
_refused(res, mam_sandbox)
assert not loop_marker.exists()
def test_delegate_submit_registers_no_job(mam_sandbox, mock_herdr, marker):
marker(True)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-delegate-job/multi-agent-mux-delegate-job"),
"submit", "--agent", "claude-code", "--agent-session", "herdr:anyone", "--prompt", "hi"])
_refused(res, mam_sandbox)
jobs = mam_sandbox / ".mam" / "jobs"
assert not list(jobs.glob("*.json")) if jobs.exists() else True
@pytest.fixture
def other_ws(tmp_path):
"""A second installed workspace B (update.sh's own test: .mam_deploy/remove.sh),
outside the sandbox (A)."""
b = Path(os.path.realpath(tmp_path)) / "ws-b"
(b / ".mam").mkdir(parents=True)
(b / ".mam_deploy").mkdir()
(b / ".mam_deploy" / "remove.sh").write_text("#!/bin/sh\n")
(b / "sub").mkdir()
return b
def _submit_to(sandbox, session, env=None):
return _run(sandbox, ["bash", _script(sandbox, "multi-agent-mux-delegate-job/multi-agent-mux-delegate-job"),
"submit", "--agent", "claude-code", "--agent-session", session, "--prompt", "hi"], env=env)
def _no_job(sandbox):
jobs = sandbox / ".mam" / "jobs"
return not jobs.exists() or not list(jobs.glob("*.json"))
def _server_for(sandbox, name):
"""The server the wrapper delivers a rowless target on (resolve_herdr_session)."""
res = subprocess.run(["bash", "-c", 'source "$1" && resolve_herdr_session "$2" "$3"', "_",
str(sandbox / ".agents" / "skills" / "lib.sh"), name, str(sandbox)],
capture_output=True, text=True, cwd=str(sandbox))
assert res.returncode == 0 and res.stdout.strip(), res.stderr
return res.stdout.strip()
def _target_in(kind, sandbox, mock_herdr, b):
"""Put the delegate target `tgt` in workspace B as <kind>; the env for submit."""
if kind == "rowless-live":
_set_mock(mock_herdr, agents={"tgt": _agent(b / "sub", "w1:p6")})
return {"MAM_SESS": _server_for(sandbox, "tgt")}
r = _row("tgt")
if kind == "row-pane-cwd":
r["pane"]["cwd"] = str(b / "sub")
elif kind == "row-control-plane":
r["pane"]["cwd"] = "/nonexistent/tgt"
r["control_plane"] = {"workspace_root": str(b)}
elif kind == "row-under-incidental-mam":
(b / "sub" / ".mam").mkdir() # runtime state only: not an installed tree
r["pane"]["cwd"] = str(b / "sub")
r["control_plane"] = {"workspace_root": str(b)}
_seed(sandbox, [r])
return {}
KINDS = ["row-pane-cwd", "row-control-plane", "row-under-incidental-mam", "rowless-live"]
@pytest.mark.parametrize("kind", KINDS)
def test_delegate_submit_takes_the_target_workspace_marker(mam_sandbox, mock_herdr, marker, other_ws, kind):
"""R1 (rounds 1, 2): every installed workspace the target is in - by its row,
or for a rowless target by the live agent's cwd - is checked, not only the
caller's; an incidental .mam below B does not hide B."""
env = _target_in(kind, mam_sandbox, mock_herdr, other_ws)
marker(True, other_ws)
res = _submit_to(mam_sandbox, "herdr:tgt", env=env)
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s is in progress" % other_ws in res.stderr, res.stdout + res.stderr
assert "registered job" not in res.stdout
assert _no_job(mam_sandbox)
@pytest.mark.parametrize("kind", KINDS)
def test_delegate_submit_waits_for_the_target_workspace_admission_lock(mam_sandbox, mock_herdr, other_ws, kind):
env = _target_in(kind, mam_sandbox, mock_herdr, other_ws)
holder = subprocess.Popen([sys.executable, "-c", "import fcntl, sys, time\n"
"f = open(sys.argv[1], 'a'); fcntl.flock(f, fcntl.LOCK_EX)\n"
"print('locked', flush=True); time.sleep(120)",
str(other_ws / ".mam" / "update-admission.lock")],
stdout=subprocess.PIPE, text=True, start_new_session=True)
try:
assert holder.stdout.readline().strip() == "locked"
res = _submit_to(mam_sandbox, "herdr:tgt", env=dict(env, MAM_UPDATE_ADMISSION_WAIT_S="1"))
finally:
holder.kill()
holder.wait()
assert res.returncode != 0, res.stdout + res.stderr
assert "an update of %s holds its admission lock" % other_ws in res.stderr, res.stdout + res.stderr
assert _no_job(mam_sandbox)
@pytest.mark.parametrize("purge", [False, True])
def test_stop_refuses_and_changes_nothing(mam_sandbox, mock_herdr, marker, purge):
_seed(mam_sandbox, [_row("gated-stop")])
_set_mock(mock_herdr, agents={"gated-stop": _agent(mam_sandbox, "w1:p8")})
marker(True)
before = _registry(mam_sandbox)
extra = ["--purge-conversation", "--yes"] if purge else []
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-stop/scripts/stop_session.sh"),
"--session", "gated-stop", "--agent", "claude", *extra])
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
assert "gated-stop" in json.loads(Path(mock_herdr).read_text())["agents"], "the session was not touched"
assert not (mam_sandbox / ".mam" / "purging-gated-stop").exists()
def test_orc_onboard_refuses_and_changes_nothing(mam_sandbox, marker):
_seed(mam_sandbox, [], orchestrator_uuids=[ORC_OLD])
marker(True)
before = _registry(mam_sandbox)
script = _script(mam_sandbox, "multi-agent-mux-orc-onboard/scripts/orc_onboard.sh")
for args in (["--uuid", ORC], ["--remove", ORC_OLD]):
res = _run(mam_sandbox, ["bash", script, *args])
_refused(res, mam_sandbox)
assert _registry(mam_sandbox) == before
def test_a_stale_marker_refuses_nothing(mam_sandbox, mock_herdr, marker):
_seed(mam_sandbox, [_row("stale-stop")], orchestrator_uuids=[])
_set_mock(mock_herdr, agents={"stale-stop": _agent(mam_sandbox, "w1:p7")})
marker(False)
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-orc-onboard/scripts/orc_onboard.sh"),
"--uuid", ORC])
assert res.returncode == 0, res.stdout + res.stderr
assert "stale update marker" in res.stderr
res = _run(mam_sandbox, ["bash", _script(mam_sandbox, "multi-agent-mux-stop/scripts/stop_session.sh"),
"--session", "stale-stop", "--agent", "claude"])
assert res.returncode == 0, res.stdout + res.stderr
d = yaml.safe_load((mam_sandbox / ".mam" / "agent-sessions.yaml").read_text())
assert d["orchestrator_uuids"] == [ORC]
assert {r["name"]: r["status"] for r in d["herdr_sessions"]} == {"stale-stop": "stopped"}
def test_the_admission_lock_is_released_after_the_record(mam_sandbox, marker):
"""An entry takes the lock shared and drops it once its record is written:
a later exclusive taker (the update) gets it at once."""
lib = mam_sandbox / ".agents" / "skills" / "lib.sh"
res = subprocess.run(["bash", "-c", 'source "$1" && mam_update_admit && mam_update_admit_release && '
'exec 7>>"$WORKSPACE_ROOT/.mam/update-admission.lock" && '
'MAM_UPDATE_ADMISSION_WAIT_S=1 python3 -m lib_py.pane_lock acquire --fd 7 --wait-env MAM_UPDATE_ADMISSION_WAIT_S',
"_", str(lib)], capture_output=True, text=True, timeout=60)
assert res.returncode == 0 and res.stdout.strip() == "locked", res.stdout + res.stderr
+184
View File
@@ -0,0 +1,184 @@
"""Issue #72: the attach hints create, stop and the delegate-job wrapper print
are the session row's attach_command (Issue #71); with no row, the one
lib_py.agents.sanitize.attach_command builds for the server the script used.
"""
import json
import os
import re
import subprocess
import sys
import time
from pathlib import Path
import pytest
import yaml
REPO = Path(__file__).resolve().parent.parent
SKILLS = REPO / ".agents" / "skills"
sys.path.insert(0, str(SKILLS))
from lib_py.agents.sanitize import attach_command, sanitize_herdr_agent_name # noqa: E402
LONG = "lab-issue72-a-very-long-branch-name-creator-claude" # > 32 characters
UPPER = "Lab-Upper72-creator-claude"
def _set_mock(mock_herdr, **kw):
state = json.loads(Path(mock_herdr).read_text())
state.update(kw)
Path(mock_herdr).write_text(json.dumps(state))
def _rows(sandbox):
return yaml.safe_load((sandbox / ".mam" / "agent-sessions.yaml").read_text())["herdr_sessions"]
def _seed(sandbox, rows):
lib = sandbox / ".agents" / "skills" / "lib.sh"
mutation = "d['herdr_sessions'] = " + repr(rows) + "\n"
res = subprocess.run(["bash", "-c", 'source "$1" && atomic_dump_yaml "$AGENT_SESSIONS_YAML"', "_", str(lib)],
input=mutation, capture_output=True, text=True)
assert res.returncode == 0, res.stderr
def _row(name, server, cwd, pid=1):
return {"name": name, "status": "running", "role": "creator", "agent": "claude",
"herdr_session": server, "herdr_server": server, "attach_command": attach_command(server, name),
"pane": {"index": 0, "pid": pid, "cmd": "claude", "cwd": str(cwd)}}
def _seeded_attach(sandbox, name, server):
(row,) = [r for r in _rows(sandbox) if r["name"] == name]
assert row["attach_command"] == attach_command(server, name)
return row["attach_command"]
def _agent(cwd, pane, pid=9999):
return {"agent": "claude", "status": "running", "agent_status": "idle", "cwd": str(cwd), "workspace_id": "w1",
"pid": pid, "pane_id": pane, "command": "claude", "buffer": "Ready"}
# --- create ---------------------------------------------------------------------
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_create_prints_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
assert sanitize_herdr_agent_name(name) != name, "the herdr name differs from the session name"
_set_mock(mock_herdr, strict_agent_names=True)
script = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-create" / "scripts" / "create_session.sh"
res = subprocess.run(["bash", str(script), "--workspace", str(mam_sandbox), "--agent", "claude",
"--role", "creator", "--session", name, "--herdr-session", "mam-iso72", "--no-onboard"],
capture_output=True, text=True, cwd=str(mam_sandbox), timeout=300,
env=dict(os.environ, MAM_NO_SUBSCRIBER_DAEMON="1"))
assert res.returncode == 0, res.stdout + res.stderr
(row,) = [r for r in _rows(mam_sandbox) if r["name"] == name]
hints = [l for l in res.stdout.splitlines() if l.startswith("Attach:")]
assert hints == ["Attach: " + row["attach_command"]], res.stdout
assert row["attach_command"] == attach_command("mam-iso72", name)
# --- stop: the stuck-TUI message ------------------------------------------------
def _undying_herdr(mam_sandbox):
"""The mock herdr behind a wrapper that ignores every way to close a pane,
so the session survives stop's kill chain."""
bin_dir = mam_sandbox / "bin"
(bin_dir / "herdr").rename(bin_dir / "herdr-mock72")
(bin_dir / "herdr").write_text(
'#!/usr/bin/env bash\nfor a in "$@"; do case "$a" in kill-session|close) exit 0 ;; esac; done\n'
'exec "%s" "$@"\n' % (bin_dir / "herdr-mock72"))
(bin_dir / "herdr").chmod(0o755)
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_stop_stuck_tui_message_shows_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
# The SIGKILL fallback kills the pane pid: make it a child of this test.
pane = subprocess.Popen(["sleep", "300"])
try:
# The row's server is not the one passed on the command line: the row wins.
_seed(mam_sandbox, [_row(name, "mam-row72", mam_sandbox, pid=pane.pid)])
want = _seeded_attach(mam_sandbox, name, "mam-row72")
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72", pid=pane.pid)})
_undying_herdr(mam_sandbox)
script = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-stop" / "scripts" / "stop_session.sh"
res = subprocess.run(["bash", str(script), "--session", name, "--agent", "claude",
"--herdr-session", "mam-arg72", "--purge-conversation", "--yes"],
capture_output=True, text=True, cwd=str(mam_sandbox), timeout=300,
env=dict(os.environ, SKS_IDLE_TIMEOUT_MS="500"))
finally:
pane.kill()
pane.wait()
assert res.returncode == 4, res.stdout + res.stderr
hints = [l for l in res.stderr.splitlines() if "Diagnose the stuck TUI" in l]
assert hints == [" Diagnose the stuck TUI (%s), then re-run" % want], res.stderr
# --- delegate-job wrapper ---------------------------------------------------------
def _delegate(mam_sandbox, mock_herdr, name):
"""Submit a job to herdr:<name>, acknowledge it as the agent would, and
return (stdout, the herdr server the wrapper prompted)."""
reg_dir = mam_sandbox / ".mam" / "jobs"
reg_dir.mkdir(parents=True, exist_ok=True)
wrapper = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-delegate-job" / "multi-agent-mux-delegate-job"
publish = mam_sandbox / ".agents" / "skills" / "multi-agent-mux-delegate-job" / "scripts" / "publish_event.py"
env = dict(os.environ, DELEGATE_JOB_PYTHON=sys.executable, MQTT_HOST="127.0.0.1", MQTT_PORT="65534",
SKS_IDLE_TIMEOUT_MS="500", MAM_INJECT_ACK_GRACE_S="20")
proc = subprocess.Popen(["bash", str(wrapper), "submit", "--agent", "claude-code",
"--agent-session", "herdr:" + name, "--prompt", "issue 72 attach hint",
"--registry-dir", str(reg_dir), "--workdir", str(mam_sandbox),
"--timeout", "6", "--idle-timeout", "2"],
cwd=str(mam_sandbox), env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
try:
tok, jid = "", ""
deadline = time.time() + 60
while time.time() < deadline and not tok:
for f in reg_dir.glob("*.json"):
try:
tok, jid = json.loads(f.read_text()).get("dispatch_token") or "", f.stem
except ValueError:
pass
time.sleep(0.2)
assert tok, "the job was not dispatched"
ack = subprocess.run([sys.executable, str(publish), "--registry-dir", str(reg_dir), "--job", jid,
"--event", "started", "--expect-dispatch-token", tok, "--attempt-id", "1"],
capture_output=True, text=True, cwd=str(mam_sandbox), env=env, timeout=60)
# publish_event records `started` locally before its (here unreachable) broker publish
assert '"started"' in (reg_dir / (jid + ".events.log")).read_text(), ack.stderr
stdout, stderr = proc.communicate(timeout=180)
finally:
if proc.poll() is None:
proc.kill()
proc.communicate()
calls = json.loads(Path(mock_herdr).read_text())["calls"]
prompts = [c for c in calls if "agent" in c[:3] and "prompt" in c[:4]]
assert prompts, (calls, stdout, stderr)
servers = {c[1] if c[0] == "--session" else "default" for c in prompts}
assert len(servers) == 1, prompts
return stdout + stderr, servers.pop()
def _delegate_hint(out):
hints = [m.group(1) for m in (re.search(r"\(연결하려면: (.*)\)$", l) for l in out.splitlines()) if m]
assert len(hints) == 1, out
return hints[0]
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_delegate_prints_the_rows_attach_command(mam_sandbox, mock_herdr, mock_agents, name):
_seed(mam_sandbox, [_row(name, "mam-row72", mam_sandbox)])
want = _seeded_attach(mam_sandbox, name, "mam-row72")
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72")})
out, server = _delegate(mam_sandbox, mock_herdr, name)
assert server == "mam-row72"
assert _delegate_hint(out) == want
@pytest.mark.parametrize("name", [LONG, UPPER])
def test_delegate_without_a_row_builds_it_for_the_server_it_used(mam_sandbox, mock_herdr, mock_agents, name):
assert not any(r["name"] == name for r in _rows(mam_sandbox))
_set_mock(mock_herdr, strict_agent_names=True,
agents={sanitize_herdr_agent_name(name): _agent(mam_sandbox, "w1:p72")})
out, server = _delegate(mam_sandbox, mock_herdr, name)
assert _delegate_hint(out) == attach_command(server, name)
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Issue #72 — the installer side of the multi-agent-mux-update skill.
The update skill runs the new install.sh in place over an existing install
(no remove.sh, .mam kept). These pin what that in-place run must do:
install_state keeps one line per key, no updater is shipped any more and an
old one is cleaned up, and framework files the new version no longer ships
are deleted unless the user modified them.
"""
from __future__ import annotations
import os
import shutil
import subprocess
import tempfile
import pytest
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
# Extra framework-owned files only the "old" source ships.
RETIRED_SKILL = ".agents/skills/multi-agent-mux-retired72"
RETIRED_UNMODIFIED = [RETIRED_SKILL + "/SKILL.md", RETIRED_SKILL + "/scripts/old.sh"]
RETIRED_SHARED_DIR = ".agents/skills/multi-agent-mux-retired72b"
RETIRED_SHARED = RETIRED_SHARED_DIR + "/SKILL.md"
RETIRED_MODIFIED = ".agents/hooks/retired72_hook.sh"
def _copy_tree(dest):
shutil.copytree(REPO_ROOT, dest,
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", ".mam_deploy",
"__pycache__", ".pytest_cache"),
symlinks=True)
@pytest.fixture(scope="module")
def sources():
"""(new, old): the tree as is, and the same tree plus files it no longer ships."""
tmp = tempfile.mkdtemp(prefix="mam_issue72_src_")
new, old = os.path.join(tmp, "new"), os.path.join(tmp, "old")
_copy_tree(new)
_copy_tree(old)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
path = os.path.join(old, rel)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w") as f:
f.write("# shipped by the old version only: %s\n" % rel)
try:
yield new, old
finally:
shutil.rmtree(tmp, ignore_errors=True)
@pytest.fixture
def ws():
tmp = tempfile.mkdtemp(prefix="mam_issue72_ws_")
try:
yield os.path.join(tmp, "workspace")
finally:
shutil.rmtree(tmp, ignore_errors=True)
def _install(src, ws):
env = dict(os.environ)
env.update({"MAM_REPO_URL": src, "MAM_SKIP_VENV": "1"})
res = subprocess.run(["bash", os.path.join(src, "deploy", "install.sh"), ws],
env=env, capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return res
def _remove(ws):
res = subprocess.run(["bash", os.path.join(ws, ".mam_deploy", "remove.sh"), "--force"],
cwd=ws, capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return res
def _read(ws, rel):
with open(os.path.join(ws, rel)) as f:
return f.read()
def _manifest(ws):
return _read(ws, ".mam/install_manifest.txt").splitlines()
def _hashed_paths(ws):
return [line.split(None, 1)[1] for line in _read(ws, ".mam/asset_hashes.txt").splitlines()
if line.strip()]
def test_in_place_reinstall_keeps_one_line_per_install_state_key(sources, ws):
new, _ = sources
_install(new, ws)
with open(os.path.join(ws, ".mam", "install_state"), "w") as f:
f.write("first=a\ngitignore_created=1\nlast=b\n")
_install(new, ws)
assert _read(ws, ".mam/install_state").splitlines() == [
"first=a", "gitignore_created=1", "last=b"]
# remove.sh must read the key as 1: the .gitignore it created goes away.
_remove(ws)
assert not os.path.exists(os.path.join(ws, ".gitignore"))
def test_fresh_install_ships_no_updater(sources, ws):
new, _ = sources
res = _install(new, ws)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
manifest = _manifest(ws)
assert ".mam_deploy/update.sh" not in manifest and "update.sh" not in manifest
assert "bash .mam_deploy/update.sh" not in res.stdout
assert "multi-agent-mux-update/scripts/update.sh" in res.stdout
def test_in_place_reinstall_deletes_an_old_mam_deploy_updater(sources, ws):
new, _ = sources
_install(new, ws)
with open(os.path.join(ws, ".mam_deploy", "update.sh"), "w") as f:
f.write("#!/usr/bin/env bash\n# old updater\n")
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
f.write(".mam_deploy/update.sh\n")
_install(new, ws)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
assert ".mam_deploy/update.sh" not in _manifest(ws)
def test_legacy_root_updater_goes_only_when_the_manifest_lists_it(sources, ws):
new, _ = sources
_install(new, ws)
root_updater = os.path.join(ws, "update.sh")
# Listed: the installer put it there (very old layout).
with open(root_updater, "w") as f:
f.write("# legacy updater\n")
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
f.write("update.sh\n")
_install(new, ws)
assert not os.path.exists(root_updater)
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
manifest = _manifest(ws)
assert "update.sh" not in manifest and ".mam_deploy/update.sh" not in manifest
# Not listed: the user's own file stays untouched.
with open(root_updater, "w") as f:
f.write("# user's own script\n")
_install(new, ws)
assert _read(ws, "update.sh") == "# user's own script\n"
def test_files_the_new_version_no_longer_ships(sources, ws):
new, old = sources
_install(old, ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
assert rel in _manifest(ws) and rel in _hashed_paths(ws), rel
# The user edits one retired file and adds their own file next to another.
with open(os.path.join(ws, RETIRED_MODIFIED), "a") as f:
f.write("# my change\n")
with open(os.path.join(ws, RETIRED_SHARED_DIR, "notes.txt"), "w") as f:
f.write("mine\n")
# A base/ copy of a retired file must go with it.
base_copy = os.path.join(ws, ".mam", "base", RETIRED_UNMODIFIED[0])
os.makedirs(os.path.dirname(base_copy))
shutil.copy(os.path.join(ws, RETIRED_UNMODIFIED[0]), base_copy)
res = _install(new, ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED]:
assert not os.path.exists(os.path.join(ws, rel)), rel
assert not os.path.exists(os.path.join(ws, RETIRED_SKILL)), "emptied directory left behind"
assert _read(ws, RETIRED_SHARED_DIR + "/notes.txt") == "mine\n"
assert "# my change" in _read(ws, RETIRED_MODIFIED)
assert "Local modification detected" in res.stderr and RETIRED_MODIFIED in res.stderr
for rel in RETIRED_UNMODIFIED:
assert rel not in res.stderr, rel
assert not os.path.exists(base_copy)
manifest, hashed = _manifest(ws), _hashed_paths(ws)
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
assert rel not in manifest and rel not in hashed, rel
# Still-shipped framework files are untouched by the cleanup.
assert ".agents/skills/lib.sh" in manifest and ".agents/skills/lib.sh" in hashed
# The kept file is the user's now: uninstalling leaves it in place.
_remove(ws)
assert "# my change" in _read(ws, RETIRED_MODIFIED)
@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0,
reason="root reads a mode-000 file")
def test_local_source_update_backup_is_not_staged(sources, ws):
new, _ = sources
backup = os.path.join(new, ".mam", "update-backup")
os.makedirs(backup)
unreadable = os.path.join(backup, "unreadable")
with open(unreadable, "w") as f:
f.write("x")
os.chmod(unreadable, 0)
try:
_install(new, ws)
finally:
os.chmod(unreadable, 0o600)
shutil.rmtree(os.path.join(new, ".mam"))
+183
View File
@@ -0,0 +1,183 @@
"""Issue #72 (decision 2): resume ensures the monitor daemon, as create does.
Only create_session.sh started the per-workspace subscriber daemon, so a
session brought back by resume_session.sh had no monitor unless one already
ran. Resume now calls the same `start_watchdog` with the control root as the
workdir (Issue #21 A8: never a worktree --workspace's copy), on the spawn path
and the already-running path, after the row is written.
What is real: the shipped create/stop/resume scripts and the real
start_watchdog -> lib_py ensure-daemon, against the mock herdr in a sandbox.
What is not: the sandbox's monitor script is a stub that records how it was
started and sleeps, so the daemon's `pgrep -f "bash <path> --subscribe"`
check sees it but nothing reconciles. Every stub is killed in teardown.
"""
import os
import signal
import subprocess
import time
import pytest
CREATE = "multi-agent-mux-create/scripts/create_session.sh"
STOP = "multi-agent-mux-stop/scripts/stop_session.sh"
RESUME = "multi-agent-mux-resume/scripts/resume_session.sh"
MONITOR_REL = os.path.join(".agents", "skills", "multi-agent-mux-monitor", "scripts", "reconcile.sh")
def _script(root, rel):
return str(root / ".agents" / "skills" / rel)
def _install_stub_monitor(base, log):
"""Replace <base>'s monitor with a stub that writes `pid|$0|cwd|args`."""
path = base / MONITOR_REL
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('#!/bin/bash\nprintf \'%%s|%%s|%%s|%%s\\n\' "$$" "$0" "$PWD" "$*" >> "%s"\n'
'sleep 60 &\nwait $!\n' % log)
path.chmod(0o755)
return str(path)
def _starts(log):
return [line.split("|") for line in log.read_text().splitlines()] if log.exists() else []
def _wait_starts(log, n, timeout=10.0):
deadline = time.time() + timeout
while time.time() < deadline and len(_starts(log)) < n:
time.sleep(0.1)
return _starts(log)
@pytest.fixture
def stub(mam_sandbox):
"""The control root's stub monitor; kills every stub process it recorded
(the mam_sandbox reaper also sweeps anything under the sandbox)."""
logs = []
def install(base, name):
log = mam_sandbox / ("monitor-starts-%s.log" % name)
logs.append(log)
return _install_stub_monitor(base, log), log
yield install
for log in logs:
for row in _starts(log):
pid = int(row[0])
for target in subprocess.run(["pgrep", "-P", str(pid)], capture_output=True,
text=True).stdout.split() + [str(pid)]:
try:
os.kill(int(target), signal.SIGKILL)
except OSError:
pass
def _create(root, ws, name, stop):
subprocess.run(["bash", _script(root, CREATE), "--workspace", ws, "--agent", "claude",
"--role", "creator", "--session", name, "--no-onboard"],
check=True, cwd=str(root), capture_output=True, text=True)
if stop:
subprocess.run(["bash", _script(root, STOP), "--session", name, "--agent", "claude"],
check=True, cwd=str(root), capture_output=True, text=True)
def _resume(root, ws, name):
return subprocess.run(["bash", _script(root, RESUME), "--workspace", ws, "--agent", "claude",
"--session", name], cwd=str(root), capture_output=True, text=True, timeout=120)
def _assert_root_monitor_started(root, monitor, log, res):
starts = _wait_starts(log, 1)
assert len(starts) == 1, starts
pid, script, cwd, args = starts[0]
assert os.path.realpath(script) == os.path.realpath(monitor)
assert os.path.realpath(cwd) == os.path.realpath(str(root))
assert args == "--subscribe --idle-timeout 0"
assert "watchdog PID: %s" % pid in res.stdout, res.stdout
def test_spawn_path_ensures_the_control_root_monitor(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-spawn", stop=True)
monitor, log = stub(root, "root")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-spawn")
assert res.returncode == 0, res.stdout + res.stderr
assert "Successfully resumed" in res.stdout
_assert_root_monitor_started(root, monitor, log, res)
def test_already_running_path_ensures_the_monitor_once(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-alive", stop=False)
monitor, log = stub(root, "root")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-alive")
assert res.returncode == 0, res.stdout + res.stderr
assert "already running" in res.stdout
_assert_root_monitor_started(root, monitor, log, res)
# A second resume finds the running daemon and starts no other.
pid = _starts(log)[0][0]
again = _resume(root, str(root), "issue72-alive")
assert again.returncode == 0, again.stdout + again.stderr
assert "watchdog PID: %s" % pid in again.stdout
time.sleep(1)
assert len(_starts(log)) == 1
@pytest.mark.parametrize("stop", [True, False], ids=["spawn", "already-running"])
def test_worktree_session_uses_the_control_root_monitor(mam_sandbox, mock_herdr, mock_agents, stub, monkeypatch, stop):
"""A session whose pane cwd (--workspace) is not the control root still
gets the control root's monitor, never the copy in its own tree."""
root = mam_sandbox
wt = root / "wt-issue72"
wt.mkdir()
name = "issue72-wt-" + ("spawn" if stop else "alive")
_create(root, str(wt), name, stop=stop)
monitor, log = stub(root, "root")
_, wt_log = stub(wt, "worktree")
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(wt), name)
assert res.returncode == 0, res.stdout + res.stderr
_assert_root_monitor_started(root, monitor, log, res)
assert _starts(wt_log) == []
def test_opt_out_starts_nothing_and_reports_a_running_daemon(mam_sandbox, mock_herdr, mock_agents, stub):
"""MAM_NO_SUBSCRIBER_DAEMON=1 (the sandbox default): resume starts no
daemon; one that already runs is still found and reported."""
root = mam_sandbox
assert os.environ.get("MAM_NO_SUBSCRIBER_DAEMON") == "1"
_create(root, str(root), "issue72-optout", stop=True)
monitor, log = stub(root, "root")
res = _resume(root, str(root), "issue72-optout") # spawn path
assert res.returncode == 0, res.stdout + res.stderr
time.sleep(1)
assert _starts(log) == []
pre = subprocess.Popen(["bash", monitor, "--subscribe", "--idle-timeout", "0"], cwd=str(root),
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
start_new_session=True)
try:
assert len(_wait_starts(log, 1)) == 1
res = _resume(root, str(root), "issue72-optout") # already-running path
assert res.returncode == 0, res.stdout + res.stderr
assert "watchdog PID: %d" % pre.pid in res.stdout, res.stdout
time.sleep(1)
assert len(_starts(log)) == 1
finally:
pre.kill()
pre.wait()
def test_a_missing_monitor_warns_and_the_resume_still_succeeds(mam_sandbox, mock_herdr, mock_agents, monkeypatch):
root = mam_sandbox
_create(root, str(root), "issue72-nomon", stop=True)
os.remove(str(root / MONITOR_REL))
monkeypatch.delenv("MAM_NO_SUBSCRIBER_DAEMON", raising=False)
res = _resume(root, str(root), "issue72-nomon")
assert res.returncode == 0, res.stdout + res.stderr
assert "Successfully resumed" in res.stdout
assert "WARNING: could not start the monitor daemon" in res.stderr
+826
View File
@@ -0,0 +1,826 @@
"""Issue #72 - the multi-agent-mux-update skill.
Every test updates a copy of one workspace installed from this checkout
(MAM_REPO_URL, MAM_SKIP_VENV=1). herdr is a fake that answers `session list
--json` and `--session S agent list` from a JSON spec, so the pre-check's view
of the live agents is set per test; the monitor daemon is a stub reconcile.sh
that holds .mam/monitor.lock like the real one. Nothing here touches a real
herdr server or the control root's monitor.
"""
from __future__ import annotations
import fcntl
import hashlib
import json
import os
import shutil
import signal
import sqlite3
import subprocess
import sys
import time
from pathlib import Path
import pytest
import yaml
REPO_ROOT = Path(__file__).resolve().parent.parent
UPDATE_REL = ".agents/skills/multi-agent-mux-update/scripts/update.sh"
MONITOR_REL = ".agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh"
FAKE_HERDR = r'''#!/usr/bin/env python3
import json, os, sys
spec = json.load(open(os.environ["FAKE_HERDR_SPEC"]))
a = sys.argv[1:]
server = "default"
if a[:1] == ["--session"]:
server, a = a[1], a[2:]
with open(os.environ["FAKE_HERDR_SPEC"] + ".calls", "a") as f:
f.write(json.dumps([server] + a) + "\n")
if a[:2] == ["session", "list"]:
if spec.get("fail") == "session-list":
sys.exit(1)
print(json.dumps({"sessions": [{"name": n, "running": r} for n, r in spec.get("sessions", {"default": True}).items()]}))
elif a[:2] == ["agent", "list"]:
if spec.get("fail") == "agent-list:" + server:
sys.exit(1)
print(json.dumps({"result": {"agents": spec.get("agents", {}).get(server, [])}}))
elif a[:2] == ["agent", "get"]:
hit = [x for x in spec.get("agents", {}).get(server, []) if a[2] in (x.get("name"), x.get("pane_id"))]
if not hit:
sys.exit(1)
print(json.dumps({"result": {"agent": hit[0]}}))
elif a[:2] == ["pane", "close"]:
spec.setdefault("agents", {})[server] = [x for x in spec.get("agents", {}).get(server, [])
if x.get("pane_id") != a[2]]
json.dump(spec, open(os.environ["FAKE_HERDR_SPEC"], "w"))
'''
STUB_MONITOR = r'''#!/usr/bin/env bash
# Issue #72 test stub of reconcile.sh: a wrapper whose python child holds monitor.lock.
d="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd -P)"
echo "start $$ $*" >> "$d/.mam/stub-monitor.log"
python3 -c '
import fcntl, os, signal, sys, time
if os.environ.get("STUB_MONITOR_IGNORE_TERM") == "1":
signal.signal(signal.SIGTERM, signal.SIG_IGN)
f = open(sys.argv[1], "w")
fcntl.flock(f, fcntl.LOCK_EX)
time.sleep(600)
' "$d/.mam/monitor.lock" &
child=$!
if [ "${STUB_MONITOR_IGNORE_TERM:-0}" = "1" ]; then
trap '' TERM
else
trap 'kill $child 2>/dev/null; exit 143' TERM
fi
while kill -0 "$child" 2>/dev/null; do wait "$child"; done
'''
def _clean_env():
env = {k: v for k, v in os.environ.items()
if not k.startswith(("HERDR_", "XDG_", "MAM_", "STUB_"))
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "DELEGATE_JOB_LOGS_DIR", "PYTHONPATH")}
env.update(MAM_REPO_URL=str(REPO_ROOT), MAM_SKIP_VENV="1",
MAM_INSTALLER_URL="file://" + str(REPO_ROOT / "deploy" / "install.sh"),
MAM_ENV_FILE="/nonexistent/.mam.env", PYTHONDONTWRITEBYTECODE="1")
return env
@pytest.fixture(scope="module")
def template(tmp_path_factory):
ws = tmp_path_factory.mktemp("tpl72") / "ws"
ws.mkdir()
res = subprocess.run(["bash", str(REPO_ROOT / "deploy" / "install.sh"), str(ws)], env=_clean_env(),
capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
return ws
class Box:
def __init__(self, root: Path, ws: Path):
self.root, self.ws = root, ws
self.mam = ws / ".mam"
self.bin = root / "bin"
self.spec = root / "herdr.json"
self.env = _clean_env()
self.env["PATH"] = "%s:%s" % (self.bin, os.environ["PATH"])
self.env["FAKE_HERDR_SPEC"] = str(self.spec)
self.helpers = []
self.herdr()
def herdr(self, sessions=None, agents=None, fail=None):
self.spec.write_text(json.dumps({"sessions": sessions or {"default": True},
"agents": agents or {}, "fail": fail}))
def update(self, *args, env=None, timeout=300):
e = dict(self.env, **(env or {}))
return subprocess.run(["bash", str(self.ws / UPDATE_REL), "--yes", *args], cwd=str(self.ws), env=e,
stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=timeout)
def lib(self, script, env=None, timeout=120):
e = dict(self.env, WORKSPACE_ROOT=str(self.ws), **(env or {}))
return subprocess.run(["bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(self.ws)],
env=e, capture_output=True, text=True, timeout=timeout)
def installer(self, body):
p = self.root / "installer.sh"
p.write_text("#!/usr/bin/env bash\nws=\"$1\"\n" + body)
self.env["MAM_INSTALLER_URL"] = "file://" + str(p)
def helper(self, *argv, **kw):
p = subprocess.Popen(list(argv), start_new_session=True, **kw)
self.helpers.append(p)
return p
def stub_monitor(self):
p = self.ws / MONITOR_REL
p.write_text(STUB_MONITOR)
def start_monitor(self, env=None):
"""The way create and resume start it: lib.sh start_watchdog (detached, so
not a child of this test that would linger as a zombie)."""
res = self.lib('start_watchdog test "$WORKSPACE_ROOT"', env=dict(env or {}))
assert res.returncode == 0 and res.stdout.strip(), res.stdout + res.stderr
_until(lambda: _lock_held(self.mam / "monitor.lock"))
def monitor_pids(self):
r = subprocess.run(["pgrep", "-f", "bash %s --subscribe" % (self.ws / MONITOR_REL)],
capture_output=True, text=True)
return r.stdout.split()
def monitor_starts(self):
p = self.mam / "stub-monitor.log"
return p.read_text().count("start ") if p.exists() else 0
def _until(cond, timeout=15.0):
end = time.monotonic() + timeout
while time.monotonic() < end:
if cond():
return True
time.sleep(0.1)
raise AssertionError("condition not met within %ss" % timeout)
def _lock_held(path: Path) -> bool:
if not path.exists():
return False
with open(path, "a") as fh:
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fh, fcntl.LOCK_UN)
return False
except BlockingIOError:
return True
@pytest.fixture
def box(template, tmp_path):
root = Path(os.path.realpath(tmp_path))
ws = root / "ws"
shutil.copytree(template, ws, symlinks=True)
(root / "bin").mkdir()
herdr = root / "bin" / "herdr"
herdr.write_text(FAKE_HERDR)
herdr.chmod(0o755)
b = Box(root, ws)
try:
yield b
finally:
for p in b.helpers:
try:
os.killpg(p.pid, signal.SIGKILL)
except OSError:
pass
subprocess.run(["pkill", "-KILL", "-f", str(ws)], capture_output=True)
# --- helpers ---------------------------------------------------------------------
SKIP_SNAPSHOT = ("shim/", "update-admission.lock", "agent-sessions.db.lock", "monitor.lock", "stub-monitor.log")
def snapshot(ws: Path) -> dict:
out = {}
for dp, dns, fns in os.walk(ws):
dns[:] = [d for d in dns if d != "__pycache__"]
for f in fns:
p = Path(dp) / f
rel = str(p.relative_to(ws))
if rel.startswith(".mam/") and any(s in rel for s in SKIP_SNAPSHOT):
continue
out[rel] = "L:" + os.readlink(p) if p.is_symlink() else hashlib.sha256(p.read_bytes()).hexdigest()
return out
def write_rows(ws: Path, rows, **top):
d = dict(top, herdr_sessions=rows)
(ws / ".mam" / "agent-sessions.yaml").write_text(yaml.safe_dump(d, sort_keys=False))
def row(name, status="running", server="default", cwd=None, **extra):
r = {"name": name, "status": status, "role": "creator", "herdr_session": server, "herdr_server": server,
"pane": {"cwd": cwd or "/elsewhere", "cmd": "claude"}}
r.update(extra)
return r
def write_job(d: Path, jid, status="running", session="herdr:x", **extra):
d.mkdir(parents=True, exist_ok=True)
rec = dict({"job_id": jid, "status": status, "agent_session": session}, **extra)
(d / (jid + ".json")).write_text(json.dumps(rec))
def live_marker(path: Path, box: Box):
"""A marker owned by a live process (pid + lstart as loop_lock.sh writes them)."""
p = box.helper("sleep", "120")
ls = subprocess.run(["ps", "-p", str(p.pid), "-o", "lstart="], capture_output=True, text=True).stdout
ls = " ".join(ls.split())
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("pid=%d\nlstart=%s\nstarted=x\n" % (p.pid, ls))
return p
def dead_marker(path: Path):
p = subprocess.Popen(["true"])
p.wait()
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("pid=%d\nlstart=Thu Jan 1 00:00:00 1970\nstarted=x\n" % p.pid)
def assert_refused_unchanged(box: Box, before: dict, res, *needles):
assert res.returncode == 1, res.stdout + res.stderr
assert "update refused; nothing was changed" in res.stderr, res.stderr
for n in needles:
assert n in res.stderr, (n, res.stderr)
assert snapshot(box.ws) == before
assert not (box.mam / "update-in-progress").exists(), "the refusing update releases its marker"
assert not (box.mam / "update-backup").exists()
# --- success and the #8 state rules ------------------------------------------------
def test_update_in_place_keeps_mam_and_merges_install_state(box):
mam = box.mam
(mam / "version.txt").write_text("source=old\ncommit=stale0000\nfetched_at=then\nmethod=git\n")
(mam / "herdr_workspace_id").write_text("w9\n")
for d in ("briefs", "diagnostics", "resumelogs", "stoplogs", "jobs", "delegate_job_logs/j1"):
(mam / d).mkdir(parents=True, exist_ok=True)
(mam / d / "keep.txt").write_text(d)
write_job(mam / "jobs", "done1", status="completed")
(mam / "multi-agent-mux-monitor.log").write_text("log\n")
(box.ws / ".mam.env").write_text("MQTT_BROKER=example\n")
with open(mam / "install_state", "a") as fh:
fh.write("custom_marker=kept\n")
write_rows(box.ws, [row("a", status="stopped")], orchestrator_uuids=["u1"])
keep = {k: v for k, v in snapshot(box.ws).items()
if k.startswith((".mam/herdr_workspace_id", ".mam/briefs", ".mam/diagnostics", ".mam/resumelogs",
".mam/stoplogs", ".mam/jobs", ".mam/delegate_job_logs", ".mam/multi-agent-mux-monitor.log"))
or k == ".mam.env"}
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
after = snapshot(box.ws)
assert {k: after.get(k) for k in keep} == keep, "everything that is not the installer's is kept"
assert "stale0000" not in (mam / "version.txt").read_text()
state = (mam / "install_state").read_text().splitlines()
assert "custom_marker=kept" in state and state.count("gitignore_created=1") == 1, state
assert not (box.ws / ".mam_deploy" / "update.sh").exists()
assert not (mam / "update-backup").exists() and not (mam / "update-in-progress").exists()
assert "Update complete." in res.stdout
def test_old_format_registry_is_converted_by_the_new_code(box):
old = row("old-one", status="stopped", start_command="HERDR_SESSION_NAME=x herdr new-session",
kill_command="HERDR_SESSION_NAME=x herdr kill-session", attach_command="HERDR_SESSION_NAME=x herdr agent attach old-one")
write_rows(box.ws, [old])
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry-format: 1 -> 2, rows changed: 1" in res.stdout
y = yaml.safe_load((box.mam / "agent-sessions.yaml").read_text())
assert y["registry_format"] == 2
conn = sqlite3.connect(str(box.mam / "agent-sessions.db"))
try:
db = {r["name"]: r for r in (json.loads(x[0]) for x in conn.execute("SELECT data FROM sessions"))}
st = json.loads(conn.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
finally:
conn.close()
assert {r["name"]: r for r in y["herdr_sessions"]} == db and st["registry_format"] == 2
assert "kill_command" not in db["old-one"] and "start_command" not in db["old-one"]
def test_no_registry_is_nothing_to_convert(box):
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry: nothing to convert (no registry)" in res.stdout
assert not (box.mam / "agent-sessions.yaml").exists()
def test_a_failed_conversion_keeps_the_install_and_exits_nonzero(box, tmp_path):
src = tmp_path / "src"
shutil.copytree(REPO_ROOT, src, symlinks=True,
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", "__pycache__", "tests", "nats-docker"))
fmt = src / ".agents" / "skills" / "lib_py" / "registry_format.py"
fmt.write_text("import sys\nif __name__ == '__main__':\n sys.exit(3)\n" + fmt.read_text())
box.env["MAM_REPO_URL"] = str(src)
box.stub_monitor()
box.start_monitor()
write_rows(box.ws, [row("a", status="stopped")])
res = box.update()
assert res.returncode == 3, res.stdout + res.stderr
assert "the registry was not converted" in res.stderr
assert (box.ws / ".agents/skills/lib_py/registry_format.py").read_text() == fmt.read_text(), "install kept"
assert box.monitor_pids() and box.monitor_starts() == 2, "the monitor is started again anyway"
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
def test_refuses_outside_an_install_and_without_yes(box):
(box.ws / ".mam_deploy" / "remove.sh").unlink()
res = box.update()
assert res.returncode == 1 and "not an installed MAM workspace" in res.stderr
shutil.copy(REPO_ROOT / "deploy" / "remove.sh", box.ws / ".mam_deploy" / "remove.sh")
res = subprocess.run(["bash", str(box.ws / UPDATE_REL)], env=box.env, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=60)
assert res.returncode == 1 and "-y/--yes" in res.stderr
res = box.update("--force")
assert res.returncode == 2, "--force is not an alias of --yes"
# --- refusals (decision 1) ---------------------------------------------------------
def test_refuses_a_running_row(box):
write_rows(box.ws, [row("busy-creator", cwd="/somewhere/else")])
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "session busy-creator is running", "stop_session.sh --session busy-creator")
def test_refuses_an_unreadable_registry(box):
(box.mam / "agent-sessions.db").write_text("not a database")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "cannot be read")
def test_refuses_a_rowless_agent_on_any_running_server(box):
box.herdr(sessions={"default": True, "picked-by-hand": True, "down": False},
agents={"picked-by-hand": [{"name": "stray", "pane_id": "w3:p1", "cwd": str(box.ws / "sub")}],
"down": [{"name": "ghost", "pane_id": "w4:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "herdr agent stray (pane w3:p1, server picked-by-hand)")
assert "ghost" not in res.stderr, "a server that is not running is not asked"
calls = (box.root / "herdr.json.calls").read_text()
assert '"down"' not in calls
def test_the_rowless_agent_hint_stops_it_on_the_server_it_was_found_on(box):
"""R3: the printed stop command names the server; without it stop looks on
the workspace's server, where a same-named agent runs outside the workspace."""
write_rows(box.ws, [])
box.herdr(sessions={"default": True, "unrecorded-server": True},
agents={"unrecorded-server": [{"name": "stray", "pane_id": "w3:p1", "cwd": str(box.ws / "sub")}],
"default": [{"name": "stray", "pane_id": "w1:p1", "cwd": str(box.root / "elsewhere")}]})
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
line = next(ln for ln in res.stderr.splitlines() if "herdr agent stray" in ln)
cmd = line.split("stop it: ", 1)[1]
assert "--herdr-session unrecorded-server" in cmd, cmd
stop = subprocess.run(["bash", "-c", cmd], cwd=str(box.ws), env=box.env, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=120)
assert stop.returncode == 0, stop.stdout + stop.stderr
agents = json.loads(box.spec.read_text())["agents"]
assert agents["unrecorded-server"] == []
assert [x["pane_id"] for x in agents["default"]] == ["w1:p1"], "the same name elsewhere is left alone"
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
def test_an_agent_outside_the_workspace_does_not_refuse(box):
box.herdr(agents={"default": [{"name": "other", "pane_id": "w1:p2", "cwd": str(box.root / "elsewhere")}]})
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
@pytest.mark.parametrize("fail", ["session-list", "agent-list:default"])
def test_refuses_when_herdr_cannot_answer(box, fail):
box.herdr(fail=fail)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "cannot list the live agents")
def test_open_jobs_refuse_and_terminal_or_dry_run_do_not(box, tmp_path):
jobs = box.mam / "jobs"
write_job(jobs, "done1", status="completed")
write_job(jobs, "err1", status="error")
write_job(jobs, "can1", status="cancelled")
write_job(jobs, "dry1", status="pending", dry_run=True)
assert box.update().returncode == 0
write_job(jobs, "open1", status="pending", session="remote:x")
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "job open1", "close --job open1 --request cancelled")
(jobs / "open1.json").unlink()
(jobs / "broken.json").write_text("{not json")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job record %s cannot be read" % (jobs / "broken.json"))
(jobs / "broken.json").unlink()
extra = tmp_path / "mam-jobs-dir"
write_job(extra, "env1", status="running")
box.env["MAM_JOBS_DIR"] = str(extra)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job env1")
del box.env["MAM_JOBS_DIR"]
@pytest.mark.skipif(os.geteuid() == 0, reason="root reads a mode-000 directory")
def test_a_jobs_directory_that_cannot_be_listed_refuses(box, tmp_path):
"""R2: an unreadable .mam/jobs (or MAM_JOBS_DIR under an unreadable parent)
is not an empty registry."""
jobs = box.mam / "jobs"
write_job(jobs, "hidden1", status="running")
before = snapshot(box.ws)
jobs.chmod(0)
try:
res = box.update()
finally:
jobs.chmod(0o755)
assert_refused_unchanged(box, before, res, "jobs directory %s cannot be listed" % jobs)
(jobs / "hidden1.json").unlink()
parent = tmp_path / "locked"
write_job(parent / "jobs", "env1", status="running")
box.env["MAM_JOBS_DIR"] = str(parent / "jobs")
before = snapshot(box.ws)
parent.chmod(0)
try:
res = box.update()
finally:
parent.chmod(0o755)
assert_refused_unchanged(box, before, res, "jobs directory %s cannot be listed" % (parent / "jobs"))
del box.env["MAM_JOBS_DIR"]
def test_other_control_planes_count_only_jobs_addressed_to_this_registry(box, tmp_path):
other = tmp_path / "other-plane" / "jobs"
write_job(other, "theirs", status="running", session="herdr:not-ours")
write_rows(box.ws, [row("wt-creator", status="stopped", control_plane={"mam_jobs_dir": str(other)})])
assert box.update().returncode == 0, "a job of another registry's session is not ours"
write_job(other, "ours", status="running", session="herdr:wt-creator")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "job ours")
def test_loop_marker_live_refuses_stale_does_not(box):
live_marker(box.mam / "loop-guard-active", box)
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(), "a review loop is running")
dead_marker(box.mam / "loop-guard-active")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "stale loop marker" in res.stdout
# --- the calling agent (decision 7, N2) ----------------------------------------------
def _me_env(server="mam-x", pane="w5:p1"):
return {"HERDR_SESSION": server, "HERDR_PANE_ID": pane}
def test_the_calling_agent_its_row_and_its_running_job_are_excluded(box):
me = {"name": "me-creator", "pane_id": "w5:p1", "cwd": str(box.ws)}
box.herdr(sessions={"default": True, "mam-x": True}, agents={"mam-x": [me]})
write_rows(box.ws, [row("me-creator", server="mam-x", cwd=str(box.ws))])
write_job(box.mam / "jobs", "myjob", status="running", session="herdr:me-creator")
res = box.update(env=_me_env())
assert res.returncode == 0, res.stdout + res.stderr
write_job(box.mam / "jobs", "myjob2", status="pending", session="herdr:me-creator")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job myjob2")
def test_same_pane_id_on_another_server_is_not_excluded(box):
box.herdr(sessions={"default": True, "mam-x": True, "mam-y": True},
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}],
"mam-y": [{"name": "twin", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update(env=_me_env())
assert_refused_unchanged(box, before, res, "herdr agent twin (pane w5:p1, server mam-y)")
assert "agent me " not in res.stderr
def test_a_nameless_caller_pane_is_excluded_by_pane_only(box):
box.herdr(sessions={"default": True, "mam-x": True},
agents={"mam-x": [{"name": None, "pane_id": "w5:p1", "cwd": str(box.ws)}]})
assert box.update(env=_me_env()).returncode == 0
write_job(box.mam / "jobs", "nameless", status="running", session="herdr:")
write_job(box.mam / "jobs", "nonename", status="running", session="herdr:None")
before = snapshot(box.ws)
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job nameless", "job nonename")
@pytest.mark.parametrize("env", [{}, {"HERDR_PANE_ID": "w5:p1"}, {"HERDR_SESSION": "mam-x"}])
def test_nothing_is_excluded_when_the_caller_is_not_identified(box, env):
box.herdr(sessions={"default": True, "mam-x": True},
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
before = snapshot(box.ws)
res = box.update(env=env)
assert_refused_unchanged(box, before, res, "herdr agent me", "nothing was excluded as the calling agent")
# --- the monitor (decision 2, U3) ----------------------------------------------------
def test_a_running_monitor_is_stopped_and_started_again(box):
box.stub_monitor()
box.start_monitor()
first = set(box.monitor_pids())
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "monitor: stopping the daemon" in res.stdout and "monitor: restarted" in res.stdout
now = set(box.monitor_pids())
assert now and not (now & first) and box.monitor_starts() == 2
def test_no_monitor_before_means_none_after(box):
box.stub_monitor()
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert not box.monitor_pids() and box.monitor_starts() == 0
def test_no_subscriber_daemon_flag_wins(box):
box.stub_monitor()
box.start_monitor()
res = box.update(env={"MAM_NO_SUBSCRIBER_DAEMON": "1"})
assert res.returncode == 0, res.stdout + res.stderr
assert "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)" in res.stdout
assert not box.monitor_pids()
def test_a_monitor_that_ignores_term_is_killed_restarted_and_nothing_installed(box):
box.stub_monitor()
box.start_monitor(env={"STUB_MONITOR_IGNORE_TERM": "1"})
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
before = snapshot(box.ws)
res = box.update(env={"MAM_UPDATE_MONITOR_STOP_S": "1"})
assert res.returncode == 1, res.stdout + res.stderr
assert "it was killed and is started again" in res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert snapshot(box.ws) == before
assert box.monitor_pids() and box.monitor_starts() == 2
assert not (box.mam / "update-in-progress").exists()
def test_a_monitor_lock_held_by_an_unknown_process_refuses(box):
lock = box.mam / "monitor.lock"
with open(lock, "w") as fh:
fcntl.flock(fh, fcntl.LOCK_EX)
before = snapshot(box.ws)
res = box.update()
assert_refused_unchanged(box, before, res, "monitor.lock is held by a monitor the skills did not start")
# --- rollback (decisions 3, 10; U4) --------------------------------------------------
FAILING_INSTALL = """
echo broken >> "$ws/.agents/skills/lib.sh"
rm -f "$ws/.mam_deploy/remove.sh"
echo new > "$ws/MESSAGING.md"
mkdir -p "$ws/.venv/lib" && echo junk > "$ws/.venv/lib/junk"
echo junk > "$ws/.mam/version.txt"
echo junk > "$ws/.mam/asset_hashes.txt"
mkdir -p "$ws/.agents/skills/half-installed" && echo x > "$ws/.agents/skills/half-installed/f"
python3 -c 'import yaml,sys; p=sys.argv[1]; d=yaml.safe_load(open(p)); d["herdr_sessions"]=[]; yaml.safe_dump(d, open(p, "w"))' "$ws/.mam/agent-sessions.yaml"
exit 7
"""
def test_a_failed_install_rolls_everything_back(box):
write_rows(box.ws, [row("a", status="stopped")])
db = box.mam / "agent-sessions.db"
conn = sqlite3.connect(str(db))
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA wal_autocheckpoint=0")
conn.execute("CREATE TABLE state (id INTEGER PRIMARY KEY, data TEXT)")
conn.execute("CREATE TABLE sessions (name TEXT PRIMARY KEY, status TEXT, pane_cwd TEXT, data JSON)")
conn.execute("INSERT INTO state VALUES (1, ?)", (json.dumps({"orchestrator_uuids": ["in-wal"]}),))
conn.execute("INSERT INTO sessions VALUES ('a', 'stopped', '/x', ?)", (json.dumps(row("a", status="stopped")),))
conn.commit()
assert (box.mam / "agent-sessions.db-wal").stat().st_size > 0, "the commit is still in the WAL"
box.stub_monitor()
box.start_monitor()
box.installer(FAILING_INSTALL)
before = snapshot(box.ws)
try:
res = box.update()
finally:
conn.close()
assert res.returncode == 1, res.stdout + res.stderr
assert "the install failed (rc 7); rolling back" in res.stderr and "rolled back" in res.stdout
after = snapshot(box.ws)
after.pop(".mam/agent-sessions.db", None)
before.pop(".mam/agent-sessions.db", None)
before.pop(".mam/agent-sessions.db-wal", None)
before.pop(".mam/agent-sessions.db-shm", None)
assert after == before, sorted(set(after.items()) ^ set(before.items()))[:10]
assert not (box.ws / "MESSAGING.md").exists() and not (box.ws / ".venv").exists()
check = sqlite3.connect(str(db))
try:
st = json.loads(check.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
finally:
check.close()
assert st == {"orchestrator_uuids": ["in-wal"]}, "the backup carried the WAL's commit"
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
assert box.monitor_pids() and box.monitor_starts() == 2, "the restored tree starts the monitor again"
box.env["MAM_INSTALLER_URL"] = "file://" + str(REPO_ROOT / "deploy" / "install.sh")
res = box.update()
assert res.returncode == 0, "the skill runs again after a rollback: " + res.stdout + res.stderr
# --- leftovers (U4) ------------------------------------------------------------------
def _leftover(box: Box, state):
bk = box.mam / "update-backup"
(bk / "files" / ".agents" / "skills").mkdir(parents=True)
shutil.copy2(box.ws / ".agents/skills/lib.sh", bk / "files/.agents/skills/lib.sh")
(bk / "items").write_text(".agents/skills/lib.sh\nMESSAGING.md\n")
(bk / "absent").write_text("MESSAGING.md\n")
(bk / "registry").mkdir()
(bk / "registry" / "agent-sessions.yaml").write_text("herdr_sessions: []\n")
if state is not None:
(bk / "state").write_text(state + "\n")
return bk
@pytest.mark.parametrize("state", [None, "backing-up", "installed"])
def test_leftover_that_changed_nothing_or_finished_is_removed(box, state):
_leftover(box, state)
(box.ws / "MESSAGING.md").write_text("must survive: nothing is restored\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "removing the backup a previous update left" in res.stdout
assert (box.ws / "MESSAGING.md").exists()
assert not (box.mam / "update-backup").exists()
def test_leftover_installing_restores_files_but_not_the_registry(box):
_leftover(box, "installing")
pristine = (box.ws / ".agents/skills/lib.sh").read_bytes()
with open(box.ws / ".agents/skills/lib.sh", "a") as fh:
fh.write("# half installed\n")
(box.ws / "MESSAGING.md").write_text("made by the dead install\n")
write_rows(box.ws, [row("written-after-crash", status="stopped")])
box.installer("grep -q 'half installed' \"$ws/.agents/skills/lib.sh\" && exit 9\n"
"test -e \"$ws/MESSAGING.md\" && exit 8\n"
"cp \"$ws/.mam/agent-sessions.yaml\" \"$ws/../seen-registry.yaml\"\nexit 0\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "the previous update was rolled back" in res.stdout
assert (box.ws / ".agents/skills/lib.sh").read_bytes() == pristine
assert "written-after-crash" in (box.root / "seen-registry.yaml").read_text(), "registry left as it was"
assert not (box.mam / "update-backup").exists()
def test_leftover_whose_restore_fails_is_kept_and_nothing_more_changes(box):
bk = _leftover(box, "installing")
(bk / "items").write_text(".agents/skills/lib.sh\n.agents/skills/missing-from-backup\n")
box.stub_monitor()
box.start_monitor()
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
assert "restoring the previous update's backup failed" in res.stderr
assert bk.exists() and (bk / "state").read_text().strip() == "installing"
assert not (box.ws / "INSTALLER-RAN").exists()
assert not box.monitor_pids(), "the monitor stays stopped"
assert not (box.mam / "update-in-progress").exists()
def test_a_state_write_failure_before_the_install_changes_nothing_and_restarts_the_monitor(box):
real_mv = shutil.which("mv")
fake = box.bin / "mv"
fake.write_text("#!/usr/bin/env bash\n"
"src=\"${@: -2:1}\"\n"
"case \"$src\" in */update-backup/state.tmp) grep -qx installing \"$src\" && exit 1 ;; esac\n"
"exec %s \"$@\"\n" % real_mv)
fake.chmod(0o755)
box.stub_monitor()
box.start_monitor()
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update()
assert res.returncode == 1, res.stdout + res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert box.monitor_starts() == 2, "the monitor it stopped is started again"
_until(lambda: bool(box.monitor_pids()))
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
assert "nothing was changed" in res.stderr
def test_the_backup_lives_in_the_workspace_and_is_gone_after_success(box):
box.installer("test -f \"$ws/.mam/update-backup/state\" && grep -qx installing \"$ws/.mam/update-backup/state\" "
"&& test -f \"$ws/.mam/update-backup/files/.agents/skills/lib.sh\" && touch \"$ws/../saw-backup\"\n")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert (box.root / "saw-backup").exists()
assert not (box.mam / "update-backup").exists()
# --- the marker and the admission lock (decision 8) -----------------------------------
def test_a_live_update_marker_refuses_a_second_update_and_is_left_alone(box):
live_marker(box.mam / "update-in-progress", box)
text = (box.mam / "update-in-progress").read_text()
before = snapshot(box.ws)
res = box.update()
assert res.returncode == 1 and "another update is in progress" in res.stderr, res.stderr
assert (box.mam / "update-in-progress").read_text() == text
assert snapshot(box.ws) == before
def test_a_stale_update_marker_is_taken_over(box):
dead_marker(box.mam / "update-in-progress")
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "stale update marker" in res.stdout
assert not (box.mam / "update-in-progress").exists()
def test_two_updates_at_once_one_proceeds(box):
box.installer("touch \"$ws/../first-installing\"; sleep 4; exit 0\n")
first = box.helper("bash", str(box.ws / UPDATE_REL), "--yes", cwd=str(box.ws), env=box.env,
stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
_until(lambda: (box.root / "first-installing").exists())
marker = (box.mam / "update-in-progress").read_text()
second = box.update()
assert second.returncode == 1 and "another update is in progress" in second.stderr, second.stderr
assert (box.mam / "update-in-progress").read_text() == marker, "the running update's marker is untouched"
out, err = first.communicate(timeout=120)
assert first.returncode == 0, out + err
def test_entries_are_refused_while_the_copy_installs(box):
box.installer('if bash -c \'source "$1/.agents/skills/lib.sh" && mam_update_admit\' _ "$ws" 2> "$ws/../admit.err"; '
'then echo admitted > "$ws/../admit"; else echo refused > "$ws/../admit"; fi\n')
res = box.update()
assert res.returncode == 0, res.stdout + res.stderr
assert (box.root / "admit").read_text().strip() == "refused"
assert "an update of %s is in progress" % box.ws in (box.root / "admit.err").read_text()
@pytest.mark.parametrize("wait,needle", [("30", "session late-row is running"), ("1", "admission lock past")])
def test_an_entry_holding_the_admission_lock_is_waited_for_or_refuses(box, wait, needle):
script = ('mam_update_admit || exit 1\necho admitted\nsleep 3\n'
'printf "herdr_sessions:\\n- name: late-row\\n status: running\\n" > "$WORKSPACE_ROOT/.mam/agent-sessions.yaml"\n'
'mam_update_admit_release\nsleep 30\n')
entry = box.helper("bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(box.ws),
env=dict(box.env, WORKSPACE_ROOT=str(box.ws)), stdout=subprocess.PIPE, text=True)
assert entry.stdout.readline().strip() == "admitted"
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
res = box.update(env={"MAM_UPDATE_ADMISSION_WAIT_S": wait})
assert res.returncode == 1, res.stdout + res.stderr
assert needle in res.stderr, res.stderr
assert not (box.ws / "INSTALLER-RAN").exists()
assert not (box.mam / "update-in-progress").exists()
# --- the first transition (decision 4, criterion 5) ------------------------------------
def test_the_old_updater_installs_the_skill_which_then_converts_the_registry(tmp_path):
"""A workspace installed by v4.11.0 still has .mam_deploy/update.sh. Run once,
it installs the new version - the skill, and no updater left behind - and
copies the registry unconverted; the skill run right after converts it."""
old_src = tmp_path / "v4.11.0"
old_src.mkdir()
arch = subprocess.run(["git", "-C", str(REPO_ROOT), "archive", "0e1e2e8"], capture_output=True)
if arch.returncode != 0:
pytest.skip("no git history to rebuild v4.11.0 from")
subprocess.run(["tar", "-x", "-C", str(old_src)], input=arch.stdout, check=True)
root = Path(os.path.realpath(tmp_path))
ws = root / "ws"
ws.mkdir()
(root / "bin").mkdir()
(root / "bin" / "herdr").write_text(FAKE_HERDR)
(root / "bin" / "herdr").chmod(0o755)
b = Box(root, ws)
res = subprocess.run(["bash", str(old_src / "deploy" / "install.sh"), str(ws)],
env=dict(b.env, MAM_REPO_URL=str(old_src)), capture_output=True, text=True, timeout=600)
assert res.returncode == 0, res.stdout + res.stderr
assert (ws / ".mam_deploy" / "update.sh").exists()
write_rows(ws, [row("old-one", status="stopped", kill_command="x", start_command="y")])
res = subprocess.run(["bash", str(ws / ".mam_deploy" / "update.sh"), "--force"], cwd=str(ws), env=b.env,
capture_output=True, text=True, timeout=600, stdin=subprocess.DEVNULL)
assert res.returncode == 0, res.stdout + res.stderr
assert (ws / UPDATE_REL).exists() and not (ws / ".mam_deploy" / "update.sh").exists()
assert "registry_format" not in yaml.safe_load((ws / ".mam" / "agent-sessions.yaml").read_text())
res = b.update()
assert res.returncode == 0, res.stdout + res.stderr
assert "registry-format: 1 -> 2, rows changed: 1" in res.stdout
assert yaml.safe_load((ws / ".mam" / "agent-sessions.yaml").read_text())["registry_format"] == 2
+18 -4
View File
@@ -1,6 +1,8 @@
#!/usr/bin/env python3
"""Issue #8 — update.sh must keep the installer's fresh version.txt and
asset_hashes.txt instead of restoring the pre-update copies.
"""Issue #8 — an update must keep the installer's fresh version.txt and
asset_hashes.txt instead of restoring the pre-update copies. Since Issue #72
the update is the multi-agent-mux-update skill (in place, no remove.sh) and
no .mam_deploy/update.sh is installed.
Restoring them left version.txt naming the previous commit and made remove.sh
compare against stale hashes, so every updated-but-pristine file was backed up
@@ -51,8 +53,19 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
with open(mam / "install_state", "a") as fh:
fh.write("custom_marker=kept\n")
res = subprocess.run(["bash", str(ws / ".mam_deploy" / "update.sh"), "--force"],
cwd=str(ws), env=env, capture_output=True, text=True, timeout=600)
assert not (ws / ".mam_deploy" / "update.sh").exists()
upd_env = {k: v for k, v in env.items() if not k.startswith("HERDR_")
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
upd_env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
# The update lists the live herdr agents; never ask a real server here.
fake = ws.parent / "fake-herdr-bin"
fake.mkdir()
(fake / "herdr").write_text('#!/bin/sh\necho \'{"sessions": []}\'\n')
(fake / "herdr").chmod(0o755)
upd_env["PATH"] = "%s:%s" % (fake, upd_env["PATH"])
res = subprocess.run(["bash", str(ws / ".agents/skills/multi-agent-mux-update/scripts/update.sh"), "--yes"],
cwd=str(ws), env=upd_env, capture_output=True, text=True, timeout=600,
stdin=subprocess.DEVNULL)
assert res.returncode == 0, res.stderr + res.stdout
version = (mam / "version.txt").read_text()
@@ -64,3 +77,4 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
state = (mam / "install_state").read_text()
assert "custom_marker=kept" in state, state
assert state.count("gitignore_created=1") == 1, "merged keys must not be duplicated:\n" + state
assert not (ws / ".mam_deploy" / "update.sh").exists()