feat: multi-agent-mux-update skill and in-place install (Issue #72)
The update is a skill (decision 4); deploy/update.sh and the installed .mam_deploy/update.sh are gone. - Phase 1 (installed code, update.sh): publish the update marker under the exclusive admission lock; refuse before any change while a running row, a live herdr agent in the workspace on any running server, an open job (unreadable JSON included, dry_run not), a live loop marker or a monitor.lock held by a monitor the skills did not start (U3) is there - the calling agent (server + pane, its row and one running job) excepted (lib_py/update.py precheck, decisions 1 and 7). Stop the monitor (TERM, then KILL and restart it if it does not end; decision 2), handle a leftover backup by its state (U4), back up .agents, .mam_deploy, root files, .venv, the installer's .mam files and the registry (SQLite backup API under the registry lock) into .mam/update-backup/, then exec a copy of install_phase.sh outside the tree (decision 5). - The copy (no lib.sh/lib_py, bash 3.2) runs the installer in place; on failure it restores the backup and restarts the monitor from the restored tree; on success it runs phase 2, post_update.sh of the new tree, which converts the registry with lib_py.registry_format and restarts the monitor if it ran. The backup and the marker are removed at the end. - install.sh updates in place (decisions 3, 4, 9): install_state keys are set, one line each; no updater is installed and an old one (.mam_deploy/update.sh, or a root update.sh the manifest lists) is removed; framework files the new version no longer ships are deleted when unmodified, kept and listed when modified, and dropped from base/, the manifest and asset_hashes.txt; a local source's .mam is not staged. remove.sh knows the new skill; CI shellchecks its scripts; docs point at the skill. - test_issue8_update_state and test_deploy_layout td6/td27 run the skill. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
GodopuandClaude Opus 5.5 committed
1 parent
6359d65999
commit
47fe24e07c
19 files changed
+1820
-314
No files matched your search
@@ -0,0 +1,315 @@
|
||||
"""Decisions behind the multi-agent-mux-update skill (Issue #72).
|
||||
|
||||
update.sh keeps the orchestration in shell (phase 1: this tree's code) and asks
|
||||
this module what needs parsing. Standard library only, Python 3.9.
|
||||
|
||||
python -m lib_py.update precheck --workspace W --herdr H
|
||||
What refuses the update (comment 921, decisions 1 and 7): the running
|
||||
rows of the registry (MAM_STATE_JSON, from lib.sh `load_state_json
|
||||
strict`), every live herdr agent whose cwd is inside W on any server
|
||||
`herdr session list --json` reports running (read only: the real herdr H,
|
||||
never the shim, so no server is started), and the open jobs. The agent
|
||||
that called the update - by (server, pane): pane_lock.caller_server and
|
||||
HERDR_PANE_ID - is not counted, nor its row and its one running job.
|
||||
Prints one line per blocker with how to clear it; rc 0 nothing blocks,
|
||||
1 something does (a herdr or registry that cannot be read included).
|
||||
python -m lib_py.update monitor --workspace W
|
||||
JSON {"pids": [...], "lock": "free"|"held"}: the monitor daemons
|
||||
daemon.find_running recognises, and whether .mam/monitor.lock is held.
|
||||
python -m lib_py.update backup-registry --workspace W --dest D
|
||||
Copies the registry into D under the registry's own write lock
|
||||
(agent-sessions.db.lock, as atomic_yaml takes it): the DB through the
|
||||
SQLite backup API (commits still in -wal included), the YAML and its
|
||||
.bak as files. Prints the names it copied.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import fcntl
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Dict, Iterable, List, Mapping, Optional, Tuple
|
||||
|
||||
from lib_py.agents.sanitize import sanitize_herdr_agent_name
|
||||
from lib_py.delegation import daemon
|
||||
from lib_py.delegation.engine import TERMINAL_STATUSES
|
||||
from lib_py.pane_lock import caller_server
|
||||
|
||||
REGISTRY_FILES = ("agent-sessions.yaml", "agent-sessions.yaml.bak", "agent-sessions.db")
|
||||
HERDR_TIMEOUT_S = 20
|
||||
|
||||
|
||||
def _real(p: str) -> str:
|
||||
return os.path.realpath(p)
|
||||
|
||||
|
||||
def _inside(path: str, root: str) -> bool:
|
||||
if not path:
|
||||
return False
|
||||
p, r = _real(path), _real(root).rstrip(os.sep)
|
||||
return p == r or p.startswith(r + os.sep)
|
||||
|
||||
|
||||
def row_server(row: dict) -> str:
|
||||
return row.get("herdr_session") or row.get("herdr_server") or "default"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- herdr
|
||||
|
||||
class HerdrError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _herdr_json(herdr: str, args: List[str], env: Mapping[str, str]) -> dict:
|
||||
# A pane's own scope ids would narrow the answer to the caller's workspace.
|
||||
cenv = {k: v for k, v in env.items()
|
||||
if k not in ("HERDR_WORKSPACE_ID", "HERDR_PANE_ID", "HERDR_TAB_ID")}
|
||||
try:
|
||||
res = subprocess.run([herdr, *args], env=cenv, stdin=subprocess.DEVNULL,
|
||||
capture_output=True, text=True, timeout=HERDR_TIMEOUT_S)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
raise HerdrError("`herdr %s` failed: %s" % (" ".join(args), exc))
|
||||
if res.returncode != 0:
|
||||
raise HerdrError("`herdr %s` exited %d: %s" % (" ".join(args), res.returncode,
|
||||
(res.stderr or "").strip()[:200]))
|
||||
try:
|
||||
out = res.stdout
|
||||
return json.loads(out[out.index("{"):])
|
||||
except ValueError:
|
||||
raise HerdrError("`herdr %s` printed no JSON" % " ".join(args))
|
||||
|
||||
|
||||
def running_servers(herdr: str, env: Mapping[str, str]) -> List[str]:
|
||||
d = _herdr_json(herdr, ["session", "list", "--json"], env)
|
||||
sessions = d.get("sessions")
|
||||
if not isinstance(sessions, list):
|
||||
raise HerdrError("`herdr session list --json` has no sessions list")
|
||||
return [s["name"] for s in sessions
|
||||
if isinstance(s, dict) and s.get("running") and s.get("name")]
|
||||
|
||||
|
||||
def server_agents(herdr: str, server: str, env: Mapping[str, str]) -> List[dict]:
|
||||
d = _herdr_json(herdr, ["--session", server, "agent", "list"], env)
|
||||
agents = (d.get("result") or {}).get("agents")
|
||||
if not isinstance(agents, list):
|
||||
raise HerdrError("`herdr --session %s agent list` has no agents list" % server)
|
||||
return [a for a in agents if isinstance(a, dict)]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- jobs
|
||||
|
||||
def _job_records(d: str) -> Iterable[Tuple[str, Optional[dict]]]:
|
||||
for p in sorted(glob.glob(os.path.join(d, "*.json"))):
|
||||
try:
|
||||
with open(p, encoding="utf-8") as fh:
|
||||
rec = json.load(fh)
|
||||
except (OSError, ValueError):
|
||||
yield p, None
|
||||
continue
|
||||
yield p, rec if isinstance(rec, dict) else None
|
||||
|
||||
|
||||
def _is_open(rec: dict) -> bool:
|
||||
return rec.get("status") not in TERMINAL_STATUSES and rec.get("dry_run") is not True
|
||||
|
||||
|
||||
def open_jobs(workspace: str, state: dict, env: Mapping[str, str]) -> List[dict]:
|
||||
"""Every open record (not completed/error/cancelled, not dry_run) in this
|
||||
workspace's .mam/jobs and MAM_JOBS_DIR; in another row's
|
||||
control_plane.mam_jobs_dir only those addressed to a row of this registry.
|
||||
A record that cannot be read is open (path only)."""
|
||||
own, seen = [], set()
|
||||
for c in (os.path.join(workspace, ".mam", "jobs"), env.get("MAM_JOBS_DIR") or ""):
|
||||
if c and os.path.isdir(c) and _real(c) not in seen:
|
||||
seen.add(_real(c))
|
||||
own.append(c)
|
||||
rows = state.get("herdr_sessions") or []
|
||||
names = {"herdr:%s" % s.get("name") for s in rows if s.get("name")}
|
||||
other = []
|
||||
for s in rows:
|
||||
c = (s.get("control_plane") or {}).get("mam_jobs_dir") or ""
|
||||
if c and os.path.isdir(c) and _real(c) not in seen:
|
||||
seen.add(_real(c))
|
||||
other.append(c)
|
||||
out = []
|
||||
for dirs, addressed_only in ((own, False), (other, True)):
|
||||
for d in dirs:
|
||||
for path, rec in _job_records(d):
|
||||
if rec is None:
|
||||
out.append({"path": path, "unreadable": True})
|
||||
elif _is_open(rec) and (not addressed_only or rec.get("agent_session") in names):
|
||||
out.append({"path": path, "job_id": rec.get("job_id") or "",
|
||||
"agent_session": rec.get("agent_session") or "",
|
||||
"status": rec.get("status"), "registry_dir": d})
|
||||
return out
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- precheck
|
||||
|
||||
def precheck(workspace: str, state: dict, herdr: str, env: Mapping[str, str]) -> List[str]:
|
||||
"""The lines that refuse the update; [] when nothing does."""
|
||||
blockers: List[str] = []
|
||||
stop = ("bash .agents/skills/multi-agent-mux-stop/scripts/stop_session.sh --session %s"
|
||||
" (resume it after the update with multi-agent-mux-resume)")
|
||||
|
||||
# The caller (decision 7): its server and pane, both or nothing.
|
||||
me_server, me_pane = caller_server(env), env.get("HERDR_PANE_ID") or ""
|
||||
me_name: Optional[str] = None
|
||||
me_found = False
|
||||
|
||||
agents_by_server: Dict[str, List[dict]] = {}
|
||||
try:
|
||||
for server in running_servers(herdr, env):
|
||||
agents_by_server[server] = server_agents(herdr, server, env)
|
||||
except HerdrError as exc:
|
||||
blockers.append("herdr: cannot list the live agents (%s); fix herdr and try again" % exc)
|
||||
|
||||
if me_server and me_pane:
|
||||
for a in agents_by_server.get(me_server, []):
|
||||
if a.get("pane_id") == me_pane:
|
||||
me_found = True
|
||||
me_name = a.get("name") or None # a nameless pane is excluded by pane only
|
||||
break
|
||||
|
||||
rows = state.get("herdr_sessions") or []
|
||||
me_row = None
|
||||
if me_name:
|
||||
for s in rows:
|
||||
if (s.get("status") == "running" and row_server(s) == me_server
|
||||
and s.get("name") and sanitize_herdr_agent_name(s["name"]) == me_name):
|
||||
me_row = s
|
||||
break
|
||||
|
||||
for s in rows:
|
||||
if s.get("status") == "running" and s is not me_row:
|
||||
blockers.append("session %s is running (server %s); stop it: %s"
|
||||
% (s.get("name"), row_server(s), stop % s.get("name")))
|
||||
|
||||
row_keys = {(row_server(s), sanitize_herdr_agent_name(s["name"]))
|
||||
for s in rows if s.get("status") == "running" and s.get("name")}
|
||||
for server, agents in agents_by_server.items():
|
||||
for a in agents:
|
||||
if me_found and server == me_server and a.get("pane_id") == me_pane:
|
||||
continue
|
||||
if not (_inside(a.get("cwd") or "", workspace)
|
||||
or _inside(a.get("foreground_cwd") or "", workspace)):
|
||||
continue
|
||||
name = a.get("name") or ""
|
||||
if name and (server, name) in row_keys:
|
||||
continue # already listed as its running row
|
||||
blockers.append("herdr agent %s (pane %s, server %s) runs in %s; stop it: %s"
|
||||
% (name or "<unnamed>", a.get("pane_id"), server, a.get("cwd"),
|
||||
stop % name if name else
|
||||
"herdr --session %s pane close %s" % (server, a.get("pane_id"))))
|
||||
|
||||
jobs = open_jobs(workspace, state, env)
|
||||
if me_row is not None:
|
||||
mine = [j for j in jobs if not j.get("unreadable")
|
||||
and j.get("agent_session") == "herdr:%s" % me_row.get("name")
|
||||
and j.get("status") == "running"]
|
||||
if len(mine) == 1:
|
||||
jobs = [j for j in jobs if j is not mine[0]]
|
||||
for j in jobs:
|
||||
if j.get("unreadable"):
|
||||
blockers.append("job record %s cannot be read; check it, then move or delete it by hand"
|
||||
% j["path"])
|
||||
else:
|
||||
blockers.append("job %s (%s, %s) is open; wait for it to end or close it: python "
|
||||
".agents/skills/multi-agent-mux-delegate-job/scripts/registry.py"
|
||||
" --registry-dir %s close --job %s --request cancelled"
|
||||
% (j["job_id"], j["agent_session"] or "-", j["status"],
|
||||
j["registry_dir"], j["job_id"]))
|
||||
|
||||
if blockers and not me_found:
|
||||
blockers.append("note: no herdr agent was found at the caller's server %r and pane %r, so"
|
||||
" nothing was excluded as the calling agent" % (me_server, me_pane))
|
||||
return blockers
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- monitor
|
||||
|
||||
def monitor_state(workspace: str) -> dict:
|
||||
pids = daemon.find_running(daemon.monitor_script_path(workspace))
|
||||
lock = os.path.join(workspace, ".mam", "monitor.lock")
|
||||
held = False
|
||||
if os.path.exists(lock):
|
||||
with open(lock, "a") as fh:
|
||||
try:
|
||||
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
fcntl.flock(fh, fcntl.LOCK_UN)
|
||||
except BlockingIOError:
|
||||
held = True
|
||||
return {"pids": pids, "lock": "held" if held else "free"}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- registry backup
|
||||
|
||||
def backup_registry(workspace: str, dest: str) -> List[str]:
|
||||
mam = os.path.join(workspace, ".mam")
|
||||
db = os.path.join(mam, "agent-sessions.db")
|
||||
os.makedirs(dest, exist_ok=True)
|
||||
copied = []
|
||||
with open(db + ".lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
try:
|
||||
if os.path.exists(db):
|
||||
src = sqlite3.connect(db, timeout=60.0)
|
||||
dst = sqlite3.connect(os.path.join(dest, "agent-sessions.db"))
|
||||
try:
|
||||
src.backup(dst)
|
||||
finally:
|
||||
dst.close()
|
||||
src.close()
|
||||
copied.append("agent-sessions.db")
|
||||
for name in REGISTRY_FILES:
|
||||
p = os.path.join(mam, name)
|
||||
if name != "agent-sessions.db" and os.path.exists(p):
|
||||
shutil.copy2(p, os.path.join(dest, name))
|
||||
copied.append(name)
|
||||
finally:
|
||||
fcntl.flock(lk, fcntl.LOCK_UN)
|
||||
return copied
|
||||
|
||||
|
||||
def _state() -> dict:
|
||||
return json.loads(os.environ.get("MAM_STATE_JSON") or "{}") or {}
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
p = argparse.ArgumentParser(prog="lib_py.update")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
s = sub.add_parser("precheck")
|
||||
s.add_argument("--workspace", required=True)
|
||||
s.add_argument("--herdr", required=True)
|
||||
s = sub.add_parser("monitor")
|
||||
s.add_argument("--workspace", required=True)
|
||||
s = sub.add_parser("backup-registry")
|
||||
s.add_argument("--workspace", required=True)
|
||||
s.add_argument("--dest", required=True)
|
||||
a = p.parse_args(argv)
|
||||
ws = _real(a.workspace)
|
||||
if a.cmd == "precheck":
|
||||
try:
|
||||
state = _state()
|
||||
except ValueError as exc:
|
||||
print("registry: cannot parse MAM_STATE_JSON (%s)" % exc)
|
||||
return 1
|
||||
lines = precheck(ws, state, a.herdr, os.environ)
|
||||
for line in lines:
|
||||
print(line)
|
||||
return 1 if lines else 0
|
||||
if a.cmd == "monitor":
|
||||
print(json.dumps(monitor_state(ws)))
|
||||
return 0
|
||||
for name in backup_registry(ws, a.dest):
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: multi-agent-mux-update
|
||||
description: "Update an installed MAM workspace in place to the latest version. Refuses while a session, an open job, a review loop or an unknown monitor is live; stops the monitor daemon and starts it again afterwards; rolls everything back if the install fails; converts the registry with the newly installed code. Use instead of the removed .mam_deploy/update.sh."
|
||||
version: 4.11.0
|
||||
author: godopu
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
environments: [terminal, herdr]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [update, install, registry, migration]
|
||||
related_skills: [multi-agent-mux-stop, multi-agent-mux-resume, multi-agent-mux-monitor]
|
||||
---
|
||||
|
||||
# Multi-Agent Update
|
||||
|
||||
```bash
|
||||
bash .agents/skills/multi-agent-mux-update/scripts/update.sh --yes
|
||||
```
|
||||
|
||||
| Item | Behaviour |
|
||||
|---|---|
|
||||
| Where | An installed workspace only (`.mam_deploy/remove.sh` present). A git checkout updates with `git pull`. |
|
||||
| Confirmation | `[y/N]` prompt; without a terminal `-y`/`--yes` is required. There is no force option. |
|
||||
| Refused when | a running registry row, a live herdr agent whose cwd is in the workspace (any running server), an open job (unreadable job JSON included, `dry_run` excluded), a live loop marker, `.mam/monitor.lock` held by a monitor the skills did not start, or another update. The calling agent (its herdr server + pane, row and one running job) is not counted. Each refusal lists the cleanup; nothing is changed. |
|
||||
| During the update | create, resume, `run_loop.sh`, delegate submit, stop and orc-onboard refuse (`.mam/update-in-progress`, `.mam/update-admission.lock`). |
|
||||
| Monitor | stopped first, started again afterwards only if it ran (`MAM_NO_SUBSCRIBER_DAEMON=1`: not started). |
|
||||
| Install | the installer at `MAM_INSTALLER_URL` (default: Gitea `main`) runs in place; `.mam` is kept. On failure `.agents/`, `.mam_deploy/`, root files, `.venv`, the installer's `.mam` files and the registry are restored from `.mam/update-backup/`. |
|
||||
| Registry | converted by the new code (`lib_py.registry_format`); a failed conversion keeps the install and exits nonzero. |
|
||||
| Exit | `0` updated, `1` refused or failed, `2` bad arguments. |
|
||||
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env bash
|
||||
# install_phase.sh — multi-agent-mux-update, the part that changes the tree (Issue #72, decision 5).
|
||||
#
|
||||
# install_phase.sh run <workspace> <monitor_was_running 0|1> <tmpdir>
|
||||
# install_phase.sh restore <workspace> [--skip-registry]
|
||||
#
|
||||
# update.sh copies this file outside the workspace and execs the copy (same
|
||||
# pid and start time, so its .mam/update-in-progress marker stays live). From
|
||||
# here on the tree is being replaced, so this calls no lib.sh or lib_py: it runs
|
||||
# the installer in place, and on failure restores .mam/update-backup/ and starts
|
||||
# the monitor again with the restored (old, whole) tree's start_watchdog. On
|
||||
# success phase 2 is the NEW tree's post_update.sh, in a child process. It ends
|
||||
# by removing the backup and the marker (only its own: pid + lstart).
|
||||
# Must run on macOS bash 3.2.
|
||||
set -uo pipefail
|
||||
|
||||
MODE="${1:-}"
|
||||
WS="${2:-}"
|
||||
[ -n "$MODE" ] && [ -d "$WS" ] || { echo "usage: $0 run|restore <workspace> ..." >&2; exit 2; }
|
||||
BK="$WS/.mam/update-backup"
|
||||
|
||||
set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
|
||||
|
||||
# Puts back every backed-up item, removes the ones that did not exist before,
|
||||
# then (unless --skip-registry) the registry: the DB from the backup, its
|
||||
# -wal/-shm removed (macOS keeps them after close).
|
||||
restore() {
|
||||
local rel n
|
||||
[ -f "$BK/items" ] || { echo "ERROR: $BK/items is missing" >&2; return 1; }
|
||||
while IFS= read -r rel; do
|
||||
[ -n "$rel" ] || continue
|
||||
rm -rf "${WS:?}/$rel" || return 1
|
||||
grep -Fqx -- "$rel" "$BK/absent" 2>/dev/null && continue
|
||||
mkdir -p "$(dirname "$WS/$rel")" && cp -pPR "$BK/files/$rel" "$WS/$rel" || return 1
|
||||
done < "$BK/items"
|
||||
[ "${1:-}" = "--skip-registry" ] && return 0
|
||||
for n in agent-sessions.yaml agent-sessions.yaml.bak agent-sessions.db; do
|
||||
rm -f "$WS/.mam/$n" || return 1
|
||||
if [ -f "$BK/registry/$n" ]; then
|
||||
cp -p "$BK/registry/$n" "$WS/.mam/$n" || return 1
|
||||
fi
|
||||
done
|
||||
rm -f "$WS/.mam/agent-sessions.db-wal" "$WS/.mam/agent-sessions.db-shm"
|
||||
}
|
||||
|
||||
release_marker() {
|
||||
local m="$WS/.mam/update-in-progress" pid lstart cur
|
||||
[ -f "$m" ] || return 0
|
||||
pid="$(grep '^pid=' "$m" | head -n 1 | cut -d= -f2-)"
|
||||
lstart="$(grep '^lstart=' "$m" | head -n 1 | cut -d= -f2-)"
|
||||
cur="$(ps -p $$ -o lstart= 2>/dev/null | tr -s ' ' | sed 's/^ *//;s/ *$//')"
|
||||
if [ "$pid" = "$$" ] && [ "$lstart" = "$cur" ]; then
|
||||
rm -f "$m"
|
||||
fi
|
||||
}
|
||||
|
||||
restart_old_monitor() {
|
||||
[ "$1" = "1" ] || return 0
|
||||
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
|
||||
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
|
||||
return 0
|
||||
fi
|
||||
local pid
|
||||
if pid="$(bash -c 'source "$1/.agents/skills/lib.sh" && start_watchdog update "$1"' _ "$WS")"; then
|
||||
echo "monitor: restarted (PID $pid)"
|
||||
else
|
||||
echo "ERROR: could not restart the monitor daemon" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
case "$MODE" in
|
||||
restore)
|
||||
restore "${3:-}"
|
||||
exit $?
|
||||
;;
|
||||
run) ;;
|
||||
*) echo "usage: $0 run|restore <workspace> ..." >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
MON="${3:-0}"
|
||||
TMPD="${4:-}"
|
||||
trap '[ -z "$TMPD" ] || rm -rf "$TMPD"' EXIT
|
||||
|
||||
set_state installing || { echo "ERROR: cannot write $BK/state" >&2; release_marker; exit 1; }
|
||||
URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
|
||||
echo "install: $URL"
|
||||
irc=0
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
curl -fsSL "$URL" | bash -s -- "$WS" || irc=$?
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -qO- "$URL" | bash -s -- "$WS" || irc=$?
|
||||
else
|
||||
echo "ERROR: neither curl nor wget is available to fetch the installer" >&2
|
||||
irc=1
|
||||
fi
|
||||
|
||||
if [ "$irc" -ne 0 ]; then
|
||||
echo "ERROR: the install failed (rc $irc); rolling back" >&2
|
||||
if restore; then
|
||||
rm -rf "$BK"
|
||||
echo "rolled back: the workspace is at the version it had before the update"
|
||||
restart_old_monitor "$MON"
|
||||
else
|
||||
echo "ERROR: the rollback failed; the backup is kept at $BK (the next update restores it)" >&2
|
||||
fi
|
||||
release_marker
|
||||
exit 1
|
||||
fi
|
||||
set_state installed
|
||||
|
||||
POST="$WS/.agents/skills/multi-agent-mux-update/scripts/post_update.sh"
|
||||
prc=0
|
||||
if [ -f "$POST" ]; then
|
||||
bash "$POST" --monitor "$MON" || prc=$?
|
||||
else
|
||||
echo "ERROR: the installed version has no $POST; the registry converts on its next write and the monitor was not restarted" >&2
|
||||
prc=1
|
||||
fi
|
||||
rm -rf "$BK"
|
||||
release_marker
|
||||
[ "$prc" -eq 0 ] && echo "Update complete."
|
||||
exit "$prc"
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env bash
|
||||
# post_update.sh — multi-agent-mux-update phase 2 (Issue #72, decision 5).
|
||||
# post_update.sh --monitor <0|1>
|
||||
#
|
||||
# Run by install_phase.sh from the NEWLY installed tree, in its own process:
|
||||
# converts the registry with the new code's entry point (lib_py.registry_format,
|
||||
# #71) and starts the monitor again if it ran before the update. Exit 0, or
|
||||
# nonzero when the conversion or the monitor restart failed (the install is
|
||||
# kept either way: the registry converts again on its next write).
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
|
||||
export WORKSPACE_ROOT="$WS"
|
||||
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
|
||||
# shellcheck source=../../lib.sh
|
||||
source "$WS/.agents/skills/lib.sh"
|
||||
|
||||
MON=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--monitor) MON="$2"; shift 2 ;;
|
||||
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
rc=0
|
||||
if [ ! -f "$AGENT_SESSIONS_YAML" ] && [ ! -f "$WS/.mam/agent-sessions.db" ]; then
|
||||
echo "registry: nothing to convert (no registry)"
|
||||
else
|
||||
"$(_delegate_py_bin)" -m lib_py.registry_format --yaml "$AGENT_SESSIONS_YAML" || rc=$?
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
echo "ERROR: the registry was not converted (rc $rc); the install is kept and the next registry write converts it" >&2
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$MON" = "1" ]; then
|
||||
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
|
||||
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
|
||||
elif pid="$(start_watchdog update "$WS")"; then
|
||||
echo "monitor: restarted (PID $pid)"
|
||||
else
|
||||
echo "ERROR: could not restart the monitor daemon" >&2
|
||||
rc=1
|
||||
fi
|
||||
fi
|
||||
exit "$rc"
|
||||
+257
@@ -0,0 +1,257 @@
|
||||
#!/usr/bin/env bash
|
||||
# update.sh — multi-agent-mux-update (Issue #72): update an installed workspace in place.
|
||||
# Usage:
|
||||
# bash .agents/skills/multi-agent-mux-update/scripts/update.sh [-y|--yes]
|
||||
#
|
||||
# Phase 1 (this script, the installed code): publish .mam/update-in-progress
|
||||
# under the admission lock (lib.sh mam_update_admit is the other side), refuse
|
||||
# while anything live could see the tree change (lib_py/update.py precheck, a
|
||||
# live loop marker, a monitor.lock held by a monitor the skills did not start),
|
||||
# stop the monitor daemon, back up what the install can change into
|
||||
# .mam/update-backup/, then exec a copy of install_phase.sh outside the tree.
|
||||
# That copy runs the installer (MAM_INSTALLER_URL) in place, rolls back on
|
||||
# failure, and runs phase 2 (post_update.sh of the NEW tree) on success.
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 = updated (registry converted or nothing to convert)
|
||||
# 1 = refused or failed; what was changed and restored is printed
|
||||
# 2 = invalid args
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WS="$(cd -P "$SCRIPT_DIR/../../../.." && pwd -P)"
|
||||
export WORKSPACE_ROOT="$WS"
|
||||
export AGENT_SESSIONS_YAML="$WS/.mam/agent-sessions.yaml"
|
||||
# shellcheck source=../../lib.sh
|
||||
source "$WS/.agents/skills/lib.sh"
|
||||
# shellcheck source=../../multi-agent-mux-loop/scripts/loop_lock.sh
|
||||
source "$SKILL_DIR/multi-agent-mux-loop/scripts/loop_lock.sh"
|
||||
|
||||
YES=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-y|--yes) YES=1; shift ;;
|
||||
-h|--help) sed -n '2,4p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
||||
*) echo "ERROR: unknown arg: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
MAM="$WS/.mam"
|
||||
MARKER="$MAM/update-in-progress"
|
||||
BK="$MAM/update-backup"
|
||||
PY="$(_delegate_py_bin)"
|
||||
|
||||
if [ ! -f "$WS/.mam_deploy/remove.sh" ] && [ ! -f "$WS/remove.sh" ]; then
|
||||
echo "ERROR: $WS is not an installed MAM workspace (no .mam_deploy/remove.sh); a git checkout updates with git pull" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$YES" != "1" ]; then
|
||||
if [ ! -t 0 ]; then
|
||||
echo "ERROR: not a terminal; run with -y/--yes" >&2
|
||||
exit 1
|
||||
fi
|
||||
ans=""
|
||||
read -r -p "Update MAM in $WS to the latest version? [y/N]: " ans || ans=""
|
||||
case "$ans" in
|
||||
y|Y|yes|Yes|YES) ;;
|
||||
*) echo "Update cancelled."; exit 0 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# --- the marker, published under the exclusive admission lock (decision 8) ---
|
||||
mkdir -p "$MAM"
|
||||
exec 7>>"$MAM/update-admission.lock"
|
||||
case "$("$PY" -m lib_py.pane_lock acquire --fd 7 --wait-env MAM_UPDATE_ADMISSION_WAIT_S)" in
|
||||
locked) ;;
|
||||
timeout)
|
||||
echo "ERROR: a create, resume, loop, delegate submit, stop or orc-onboard kept the admission lock past MAM_UPDATE_ADMISSION_WAIT_S; nothing was changed, try again" >&2
|
||||
exit 1 ;;
|
||||
*)
|
||||
echo "ERROR: the admission lock $MAM/update-admission.lock could not be decided (lib_py.pane_lock); nothing was changed" >&2
|
||||
exit 1 ;;
|
||||
esac
|
||||
if [ -f "$MARKER" ] && [ "$(_mam_lock_holder_state "$MARKER")" = "stale" ]; then
|
||||
echo "NOTE: stale update marker (pid $(_mam_lock_field "$MARKER" pid)) taken over; its update is gone"
|
||||
fi
|
||||
marker_rc=0
|
||||
mam_acquire_loop_lock "$MARKER" || marker_rc=$?
|
||||
exec 7>&-
|
||||
case "$marker_rc" in
|
||||
0) ;;
|
||||
3) echo "ERROR: another update is in progress (pid $(_mam_lock_field "$MARKER" pid)); nothing was changed" >&2; exit 1 ;;
|
||||
*) echo "ERROR: cannot tell whether another update is in progress (ps unavailable?); nothing was changed" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
TMPD=""
|
||||
MONITOR_WAS_RUNNING=0
|
||||
RESTART_MONITOR=0
|
||||
_phase1_exit() {
|
||||
local rc=$?
|
||||
if [ "$RESTART_MONITOR" = "1" ]; then
|
||||
_restart_monitor || true
|
||||
fi
|
||||
mam_release_loop_lock "$MARKER" || true
|
||||
[ -z "$TMPD" ] || rm -rf "$TMPD"
|
||||
exit "$rc"
|
||||
}
|
||||
trap _phase1_exit EXIT
|
||||
|
||||
_restart_monitor() {
|
||||
if [ "${MAM_NO_SUBSCRIBER_DAEMON:-0}" = "1" ]; then
|
||||
echo "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)"
|
||||
return 0
|
||||
fi
|
||||
local pid
|
||||
pid=$(start_watchdog update "$WS") || { echo "ERROR: could not restart the monitor daemon" >&2; return 1; }
|
||||
echo "monitor: restarted (PID $pid)"
|
||||
}
|
||||
|
||||
_monitor_json() {
|
||||
"$PY" -m lib_py.update monitor --workspace "$WS"
|
||||
}
|
||||
_json_field() {
|
||||
"$PY" -c 'import json,sys; v=json.load(sys.stdin)[sys.argv[1]]; print(" ".join(v) if isinstance(v, list) else v)' "$1"
|
||||
}
|
||||
|
||||
# --- pre-check: refuse before any change (decisions 1, 7; U3) ---
|
||||
BLOCKERS=""
|
||||
_block() { BLOCKERS="${BLOCKERS} - $1
|
||||
"; }
|
||||
|
||||
if [ -f "$MAM/loop-guard-active" ]; then
|
||||
loop_st="$(_mam_lock_holder_state "$MAM/loop-guard-active")"
|
||||
case "$loop_st" in
|
||||
stale) echo "NOTE: stale loop marker $MAM/loop-guard-active (its loop is gone); not a reason to refuse" ;;
|
||||
*) _block "a review loop is running (pid $(_mam_lock_field "$MAM/loop-guard-active" pid), holder $loop_st); wait for it to end" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
STATE_JSON=""
|
||||
if ! STATE_JSON="$(load_state_json strict)"; then
|
||||
_block "the registry $AGENT_SESSIONS_YAML cannot be read; fix it first"
|
||||
STATE_JSON='{}'
|
||||
fi
|
||||
REAL_HERDR="$(_resolve_real_herdr_path || true)"
|
||||
if [ -z "$REAL_HERDR" ]; then
|
||||
_block "no herdr binary found on PATH, so the live agents cannot be listed"
|
||||
else
|
||||
pre_out=""
|
||||
pre_rc=0
|
||||
pre_out="$(MAM_STATE_JSON="$STATE_JSON" "$PY" -m lib_py.update precheck --workspace "$WS" --herdr "$REAL_HERDR")" || pre_rc=$?
|
||||
if [ "$pre_rc" -ne 0 ]; then
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] && _block "$line"
|
||||
done <<< "${pre_out:-lib_py.update precheck failed (rc $pre_rc)}"
|
||||
fi
|
||||
fi
|
||||
|
||||
MON_JSON="$(_monitor_json)" || { _block "cannot inspect the monitor daemon (lib_py.update monitor)"; MON_JSON='{"pids": [], "lock": "free"}'; }
|
||||
MON_PIDS="$(printf '%s' "$MON_JSON" | _json_field pids)"
|
||||
MON_LOCK="$(printf '%s' "$MON_JSON" | _json_field lock)"
|
||||
if [ -z "$MON_PIDS" ] && [ "$MON_LOCK" = "held" ]; then
|
||||
_block "$MAM/monitor.lock is held by a monitor the skills did not start (not 'bash $WS/.agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh --subscribe'); stop it (find it with: pgrep -af reconcile.sh) and try again"
|
||||
fi
|
||||
|
||||
if [ -n "$BLOCKERS" ]; then
|
||||
echo "ERROR: update refused; nothing was changed:" >&2
|
||||
printf '%s' "$BLOCKERS" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- stop the monitor daemon (decision 2) ---
|
||||
_monitor_gone() {
|
||||
local p
|
||||
for p in "$@"; do
|
||||
kill -0 "$p" 2>/dev/null && return 1
|
||||
done
|
||||
[ "$(_monitor_json | _json_field lock)" = "free" ]
|
||||
}
|
||||
_wait_monitor_gone() {
|
||||
local deadline=$(( $(date +%s) + $1 ))
|
||||
shift
|
||||
while :; do
|
||||
_monitor_gone "$@" && return 0
|
||||
[ "$(date +%s)" -lt "$deadline" ] || return 1
|
||||
sleep 0.5
|
||||
done
|
||||
}
|
||||
if [ -n "$MON_PIDS" ]; then
|
||||
MONITOR_WAS_RUNNING=1
|
||||
# shellcheck disable=SC2086
|
||||
set -- $MON_PIDS
|
||||
kids=""
|
||||
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
|
||||
echo "monitor: stopping the daemon (PID $MON_PIDS)"
|
||||
kill -TERM "$@" 2>/dev/null || true
|
||||
if ! _wait_monitor_gone "$(mam_int_knob MAM_UPDATE_MONITOR_STOP_S 30 1)" "$@"; then
|
||||
for p in "$@"; do kids="$kids $(pgrep -P "$p" 2>/dev/null | tr '\n' ' ' || true)"; done
|
||||
echo "WARNING: the monitor did not stop within MAM_UPDATE_MONITOR_STOP_S; sending SIGKILL to $* $kids" >&2
|
||||
# shellcheck disable=SC2086
|
||||
kill -KILL "$@" $kids 2>/dev/null || true
|
||||
# shellcheck disable=SC2086
|
||||
if _wait_monitor_gone 5 "$@" $kids; then
|
||||
echo "ERROR: the monitor did not stop in time; it was killed and is started again. Nothing else was changed." >&2
|
||||
_restart_monitor || true
|
||||
exit 1
|
||||
fi
|
||||
echo "ERROR: could not restore the monitor state: PIDs still alive or monitor.lock still held ($*$kids; lock $(_monitor_json | _json_field lock)). Nothing else was changed." >&2
|
||||
exit 1
|
||||
fi
|
||||
RESTART_MONITOR=1
|
||||
fi
|
||||
|
||||
TMPD="$(mktemp -d "${TMPDIR:-/tmp}/mam-update.XXXXXX")"
|
||||
cp "$SCRIPT_DIR/install_phase.sh" "$TMPD/install_phase.sh"
|
||||
|
||||
# --- a leftover backup of an update that died (U4) ---
|
||||
if [ -e "$BK" ]; then
|
||||
left_st="$(cat "$BK/state" 2>/dev/null || true)"
|
||||
case "$left_st" in
|
||||
""|backing-up|installed)
|
||||
echo "NOTE: removing the backup a previous update left (state ${left_st:-none}); nothing to restore"
|
||||
rm -rf "$BK" ;;
|
||||
installing)
|
||||
echo "NOTE: a previous update died during its install; restoring its backup (the registry is left as it is)"
|
||||
if ! bash "$TMPD/install_phase.sh" restore "$WS" --skip-registry; then
|
||||
RESTART_MONITOR=0
|
||||
echo "ERROR: restoring the previous update's backup failed; it is kept at $BK. Nothing more was changed; the monitor stays stopped." >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -rf "$BK"
|
||||
echo "the previous update was rolled back" ;;
|
||||
*)
|
||||
echo "ERROR: $BK has an unknown state '$left_st'; it is kept and nothing was changed. Check it and remove it by hand." >&2
|
||||
exit 1 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# --- the rollback backup (decisions 3, 10; U4) ---
|
||||
_set_state() { printf '%s\n' "$1" > "$BK/state.tmp" && mv -f "$BK/state.tmp" "$BK/state"; }
|
||||
ITEMS=".agents .mam_deploy AGENTS.md MESSAGING.md BOOTSTRAP.md BOOTSTRAP.ko.md .mam.env.example .mam.env .gitignore remove.sh update.sh .venv .mam/install_manifest.txt .mam/install_state .mam/asset_hashes.txt .mam/version.txt .mam/base .mam/skill-backups"
|
||||
_backup() {
|
||||
mkdir -p "$BK/files" || return 1
|
||||
_set_state backing-up || return 1
|
||||
: > "$BK/items" && : > "$BK/absent" || return 1
|
||||
local rel
|
||||
for rel in $ITEMS; do
|
||||
printf '%s\n' "$rel" >> "$BK/items" || return 1
|
||||
if [ -e "$WS/$rel" ] || [ -L "$WS/$rel" ]; then
|
||||
mkdir -p "$BK/files/$(dirname "$rel")" && cp -pPR "$WS/$rel" "$BK/files/$rel" || return 1
|
||||
else
|
||||
printf '%s\n' "$rel" >> "$BK/absent" || return 1
|
||||
fi
|
||||
done
|
||||
"$PY" -m lib_py.update backup-registry --workspace "$WS" --dest "$BK/registry" >/dev/null
|
||||
}
|
||||
echo "backup: $BK"
|
||||
if ! _backup; then
|
||||
rm -rf "$BK"
|
||||
echo "ERROR: the backup could not be made; nothing was changed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- hand over to the copy (decision 5): same pid, so the marker stays live ---
|
||||
RESTART_MONITOR=0
|
||||
trap - EXIT
|
||||
exec bash "$TMPD/install_phase.sh" run "$WS" "$MONITOR_WAS_RUNNING" "$TMPD"
|
||||
+3
-2
@@ -38,7 +38,7 @@ bash deploy/install.sh
|
||||
- `MQTT_BROKER=localhost` 로 된 초기 `.mam.env` 를 씁니다. 이미 있는 `.mam.env` 는 그대로 둡니다. 예전 `.env` 만 있으면 그것을 `.mam.env` 로 복사하고 원본은 남깁니다.
|
||||
- `.venv`, `.mam/`, `.mam.env` 를 `.gitignore` 에 추가합니다.
|
||||
|
||||
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트와 제거는 `deploy/update.sh`, `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
|
||||
`.mam.env` 의 `MQTT_BROKER` 를 직접 운영하는 브로커로 바꾸십시오(사설 브로커는 [MESSAGING.md](MESSAGING.md) 참고). `MQTT_BROKER` 가 아예 설정되지 않으면 코드는 공개 브로커 `broker.hivemq.com` 을 씁니다. 시험용으로는 괜찮지만 실제 작업용은 아닙니다. `deploy/generate-env.sh` 는 `.mam.env` 가 없을 때 `.mam.env.example` 로 만듭니다(`--force` 는 `.mam.env.bak` 을 남기고 덮어씁니다). 업데이트는 `multi-agent-mux-update` 스킬, 제거는 `deploy/remove.sh` 입니다. `deploy/install_mam.sh` 는 로컬 클론에서 다른 디렉터리로 복사해 설치하는 별도 설치기입니다.
|
||||
|
||||
---
|
||||
|
||||
@@ -56,6 +56,7 @@ bash deploy/install.sh
|
||||
| **`multi-agent-mux-delegate-job`** | 세션(`herdr:<name>`)이나 다른 호스트의 러너(`remote:<name>`)에 잡을 위임하고 결과를 기다립니다. [위임 계약](#-위임-계약)을 보십시오. |
|
||||
| **`multi-agent-mux-loop`** | 계획·구현·리뷰: 선택적인 planner 와 creator, 그다음 선택한 리뷰어(`--reviewer A,B`, 또는 `--all-reviewer` 로 creator 를 뺀 실행 중인 리뷰어 전원)가 모두 `PASS` 를 낼 때까지(또는 `--max-loop` 까지). 둘 다 없으면 creator 가 스스로 리뷰합니다. 루프 밖에서 끝난 작업은 `--review-only` 로 리뷰합니다. 통과한 리포트는 `.agents/reports/<session>/` 으로 옮겨 보관합니다. |
|
||||
| **`multi-agent-mux-orc-onboard`** | 오케스트레이터 자신의 세션 ID 를 등록해, 세션 탐색이 그것을 하위 에이전트로 착각하지 않게 합니다. |
|
||||
| **`multi-agent-mux-update`** | 설치된 작업 공간을 제자리에서 업데이트합니다. 실행 중인 세션·열린 잡·루프가 있으면 거부하고, 모니터를 멈췄다가 원래대로 되돌리며, 설치가 실패하면 되돌리고, 새 코드로 레지스트리를 변환합니다. |
|
||||
|
||||
---
|
||||
|
||||
@@ -195,7 +196,7 @@ sequenceDiagram
|
||||
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
|
||||
│ └── multi-agent-mux-*/ # 스킬마다 디렉터리 하나 (SKILL.md + scripts/)
|
||||
├── .mam/ # 런타임 상태 (git 제외): 세션, 잡, 감사 로그, shim
|
||||
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
|
||||
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
|
||||
├── docs/ # contracts/, design/, 에이전트 통합·브리지 가이드
|
||||
├── nats-docker/ # 사설 NATS 브로커 배포 (서브모듈)
|
||||
├── tests/ # pytest 스위트
|
||||
|
||||
@@ -38,7 +38,7 @@ bash deploy/install.sh
|
||||
- writes a starter `.mam.env` with `MQTT_BROKER=localhost`. An existing `.mam.env` is kept. If only a legacy `.env` exists, it is copied to `.mam.env` and the original stays;
|
||||
- adds `.venv`, `.mam/` and `.mam.env` to `.gitignore`.
|
||||
|
||||
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). `deploy/update.sh` and `deploy/remove.sh` update and uninstall. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
|
||||
Point `MQTT_BROKER` in `.mam.env` at a broker you run (see [MESSAGING.md](MESSAGING.md) for a private broker). If `MQTT_BROKER` is not set at all, the code falls back to the public `broker.hivemq.com`. That is fine for a trial, but not for real work. `deploy/generate-env.sh` creates `.mam.env` from `.mam.env.example` when it is missing (`--force` overwrites it, keeping a `.mam.env.bak`). The `multi-agent-mux-update` skill updates an installed workspace, and `deploy/remove.sh` uninstalls. `deploy/install_mam.sh` is a separate installer that copies from a local clone into another directory.
|
||||
|
||||
---
|
||||
|
||||
@@ -56,6 +56,7 @@ Every skill lives under `.agents/skills/` with a `SKILL.md` that documents its o
|
||||
| **`multi-agent-mux-delegate-job`** | Delegates a job to a session (`herdr:<name>`) or to a runner on another host (`remote:<name>`) and waits for its result. See [Delegation contract](#-delegation-contract). |
|
||||
| **`multi-agent-mux-loop`** | Plan, implement, review: an optional planner and a creator, then the selected reviewers (`--reviewer A,B`, or `--all-reviewer` for every running reviewer except the creator) until all of them `PASS` or `--max-loop` is reached. With neither option the creator reviews its own work. `--review-only` reviews work finished outside the loop. Passing reports are promoted to `.agents/reports/<session>/`. |
|
||||
| **`multi-agent-mux-orc-onboard`** | Registers the orchestrator's own session id, so session discovery never mistakes it for a sub-agent. |
|
||||
| **`multi-agent-mux-update`** | Updates an installed workspace in place: refuses while sessions, open jobs or a loop are live; stops and restores the monitor; rolls back on a failed install; converts the registry with the new code. |
|
||||
|
||||
---
|
||||
|
||||
@@ -195,7 +196,7 @@ sequenceDiagram
|
||||
│ │ └── *.py # pane_lock, stall_watch, worktree, herdr_config, ...
|
||||
│ └── multi-agent-mux-*/ # One directory per skill (SKILL.md + scripts/)
|
||||
├── .mam/ # Runtime state (git-ignored): sessions, jobs, audit logs, shim
|
||||
├── deploy/ # install.sh, install_mam.sh, update.sh, remove.sh, generate-env.sh
|
||||
├── deploy/ # install.sh, install_mam.sh, remove.sh, generate-env.sh
|
||||
├── docs/ # contracts/, design/, agent integration and bridge guides
|
||||
├── nats-docker/ # Private NATS broker deployment (submodule)
|
||||
├── tests/ # pytest suite
|
||||
|
||||
+3
-3
@@ -9,7 +9,7 @@ This directory contains packaging templates and installation scripts to deploy t
|
||||
* **`install.sh`**: A self-contained, idempotent remote shell installer (via curl) that checks system requirements (`herdr`, `python3`), sets up a local python virtual environment (`.venv`), and performs 3-way refresh with key-level registry merge (Rev.2).
|
||||
* **`install_mam.sh`**: A local-clone installer that copies rules/skills (`.agents/`), `AGENTS.md`, manifests, and asset hashes into target projects.
|
||||
* **`lib_ownership.sh`**: Single source of truth for framework-owned files and key-level registry files.
|
||||
* **`update.sh`**: In-place updater script installed into target `.mam_deploy/update.sh`.
|
||||
* **Updates**: installed workspaces update with the `multi-agent-mux-update` skill (`bash .agents/skills/multi-agent-mux-update/scripts/update.sh`).
|
||||
* **`remove.sh`**: Clean uninstaller script installed into target `.mam_deploy/remove.sh`.
|
||||
* **`generate-env.sh`**: Environment configuration bootstrap helper.
|
||||
* **`INSTALL.md`**: Detailed installation and quick-start user manual.
|
||||
@@ -40,7 +40,7 @@ bash deploy/install_mam.sh --target /path/to/your/project
|
||||
Refer to **`INSTALL.md`** inside this directory for the full instructions and workflows.
|
||||
|
||||
> [!NOTE]
|
||||
> `install_mam.sh` automatically deploys `.mam_deploy/update.sh` and `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
|
||||
> `install_mam.sh` automatically deploys `.mam_deploy/remove.sh` into target workspaces, recording install manifests and asset hashes to enable clean updates and uninstalls.
|
||||
|
||||
### 3. Custom Fork / Private Mirror Installations
|
||||
If you run a private mirror or fork, you can override the source URLs during installation using environment variables:
|
||||
@@ -53,7 +53,7 @@ curl -fsSL https://my-mirror.example.com/.../install.sh \
|
||||
bash
|
||||
|
||||
# Updating an existing workspace against a mirror
|
||||
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash deploy/update.sh
|
||||
MAM_INSTALLER_URL=https://my-mirror.example.com/.../install.sh bash .agents/skills/multi-agent-mux-update/scripts/update.sh
|
||||
```
|
||||
|
||||
### 4. Registering as a Workspace Plugin
|
||||
|
||||
+1
-1
@@ -41,7 +41,7 @@ jobs:
|
||||
shellcheck deploy/install.sh
|
||||
shellcheck deploy/install_mam.sh
|
||||
shellcheck deploy/generate-env.sh
|
||||
shellcheck deploy/update.sh
|
||||
shellcheck .agents/skills/multi-agent-mux-update/scripts/*.sh
|
||||
shellcheck deploy/remove.sh
|
||||
echo "✅ ShellCheck completed successfully."
|
||||
|
||||
|
||||
+67
-18
@@ -125,7 +125,9 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
|
||||
FETCH_METHOD="archive"
|
||||
if [ -d "$REPO_URL" ]; then
|
||||
echo "🌐 Copying local working tree into a staging area..."
|
||||
cp -R "$REPO_URL/." "$STAGE_DIR/"
|
||||
# Everything but .mam/: runtime state is never staged, and it may hold the
|
||||
# update skill's rollback copy (.mam/update-backup/, Issue #72).
|
||||
find "$REPO_URL/." -mindepth 1 -maxdepth 1 ! -name .mam -exec cp -R {} "$STAGE_DIR/" \;
|
||||
FETCH_METHOD="local"
|
||||
elif command -v git &>/dev/null; then
|
||||
echo "🌐 Cloning repository (shallow) into a staging area..."
|
||||
@@ -157,9 +159,15 @@ if [ "$REFRESH" -eq 1 ] || [ "${MAM_SKIP_VENV:-0}" -eq 1 ] || ! check_assets_pre
|
||||
MANIFEST_FILE=".mam/install_manifest.txt"
|
||||
touch "$MANIFEST_FILE"
|
||||
|
||||
# Migrate legacy root layout for remove.sh and update.sh into .mam_deploy/ if owned
|
||||
# Drop every line naming this path from the manifest
|
||||
manifest_drop() {
|
||||
grep -Fvx -- "$1" "$MANIFEST_FILE" > "$MANIFEST_FILE.tmp" || true
|
||||
mv -f "$MANIFEST_FILE.tmp" "$MANIFEST_FILE"
|
||||
}
|
||||
|
||||
# Migrate legacy root layout for remove.sh into .mam_deploy/ if owned
|
||||
mkdir -p .mam_deploy
|
||||
for legacy_script in remove.sh update.sh; do
|
||||
for legacy_script in remove.sh; do
|
||||
if [ -f "$legacy_script" ] && grep -Fqx "$legacy_script" "$MANIFEST_FILE" 2>/dev/null; then
|
||||
mv -f "$legacy_script" ".mam_deploy/$legacy_script"
|
||||
python3 -c '
|
||||
@@ -179,6 +187,13 @@ with open(path, "w") as f:
|
||||
fi
|
||||
done
|
||||
|
||||
# update.sh is no longer shipped (Issue #72; the multi-agent-mux-update skill
|
||||
# updates). A legacy root update.sh goes only if the manifest says we put it there.
|
||||
if [ -f update.sh ] && grep -Fqx update.sh "$MANIFEST_FILE" 2>/dev/null; then
|
||||
rm -f update.sh
|
||||
manifest_drop update.sh
|
||||
fi
|
||||
|
||||
# Safe refresh & fingerprint checking logic
|
||||
TS=$(date -u +%Y%m%dT%H%M%SZ)
|
||||
FRAMEWORK_LEDGER=".mam/.framework_ledger.tmp"
|
||||
@@ -374,7 +389,7 @@ MERGEREG
|
||||
fi
|
||||
done
|
||||
|
||||
# Install remove.sh and update.sh into .mam_deploy/
|
||||
# Install remove.sh into .mam_deploy/
|
||||
mkdir -p .mam_deploy
|
||||
if [ -f "$STAGE_DIR/deploy/remove.sh" ]; then
|
||||
cp "$STAGE_DIR/deploy/remove.sh" .mam_deploy/remove.sh
|
||||
@@ -384,13 +399,9 @@ MERGEREG
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "$STAGE_DIR/deploy/update.sh" ]; then
|
||||
cp "$STAGE_DIR/deploy/update.sh" .mam_deploy/update.sh
|
||||
chmod 0755 .mam_deploy/update.sh
|
||||
if ! grep -Fqx ".mam_deploy/update.sh" "$MANIFEST_FILE" 2>/dev/null; then
|
||||
echo ".mam_deploy/update.sh" >> "$MANIFEST_FILE"
|
||||
fi
|
||||
fi
|
||||
# An older install's updater is not shipped any more (Issue #72)
|
||||
rm -f .mam_deploy/update.sh
|
||||
manifest_drop .mam_deploy/update.sh
|
||||
|
||||
if [ -f "$STAGE_DIR/.mam.env.example" ] && [ ! -e ".mam.env.example" ]; then
|
||||
cp "$STAGE_DIR/.mam.env.example" .
|
||||
@@ -407,6 +418,34 @@ MERGEREG
|
||||
fi
|
||||
fi
|
||||
|
||||
# Framework-owned files the previous install recorded (manifest and
|
||||
# asset_hashes.txt) that this version no longer ships (Issue #72). Unmodified
|
||||
# ones are deleted. Modified ones are kept and become the user's file: they
|
||||
# leave the manifest, so remove.sh leaves them alone, and asset_hashes.txt
|
||||
# (rebuilt below from the shipped files only).
|
||||
if [ -f .mam/asset_hashes.txt ]; then
|
||||
SHIPPED_LIST=".mam/.shipped_paths.tmp"
|
||||
cut -f2 "$FRAMEWORK_LEDGER" > "$SHIPPED_LIST"
|
||||
while read -r _old_sha _old_path; do
|
||||
[ -n "${_old_path:-}" ] || continue
|
||||
is_framework_owned "$_old_path" || continue
|
||||
grep -Fqx -- "$_old_path" "$MANIFEST_FILE" || continue
|
||||
if grep -Fqx -- "$_old_path" "$SHIPPED_LIST"; then continue; fi
|
||||
if [ -f "$_old_path" ] && [ "$(python3 -c 'import hashlib, sys; print(hashlib.sha256(open(sys.argv[1], "rb").read()).hexdigest())' "$_old_path")" != "$_old_sha" ]; then
|
||||
echo "ℹ️ Local modification detected (no longer shipped, kept as yours): $_old_path" >&2
|
||||
else
|
||||
rm -f "$_old_path"
|
||||
_old_dir="$(dirname "$_old_path")"
|
||||
while [ "$_old_dir" != ".agents" ] && rmdir "$_old_dir" 2>/dev/null; do
|
||||
_old_dir="$(dirname "$_old_dir")"
|
||||
done
|
||||
fi
|
||||
rm -f ".mam/base/$_old_path"
|
||||
manifest_drop "$_old_path"
|
||||
done < .mam/asset_hashes.txt
|
||||
rm -f "$SHIPPED_LIST"
|
||||
fi
|
||||
|
||||
# Re-build asset_hashes.txt from the UPSTREAM copy of every framework-owned file
|
||||
python3 - .mam/asset_hashes.txt "$FRAMEWORK_LEDGER" <<'HASHDB'
|
||||
import os, hashlib, sys
|
||||
@@ -590,15 +629,25 @@ with open(gi_path, "w") as f:
|
||||
PY
|
||||
|
||||
if [ "${MAM_SKIP_GITIGNORE:-0}" -eq 0 ]; then
|
||||
# Set the key: overwrite it in place, one line per key, other keys kept
|
||||
python3 -c '
|
||||
import os
|
||||
os.makedirs(".mam", exist_ok=True)
|
||||
lines = []
|
||||
if os.path.exists(".mam/install_state"):
|
||||
with open(".mam/install_state", "a") as f:
|
||||
f.write("gitignore_created=1\n")
|
||||
else:
|
||||
os.makedirs(".mam", exist_ok=True)
|
||||
with open(".mam/install_state", "w") as f:
|
||||
f.write("gitignore_created=1\n")
|
||||
with open(".mam/install_state") as f:
|
||||
lines = f.read().splitlines()
|
||||
out, done = [], False
|
||||
for line in lines:
|
||||
if line.startswith("gitignore_created="):
|
||||
if done:
|
||||
continue
|
||||
line, done = "gitignore_created=1", True
|
||||
out.append(line)
|
||||
if not done:
|
||||
out.append("gitignore_created=1")
|
||||
with open(".mam/install_state", "w") as f:
|
||||
f.write("\n".join(out) + "\n")
|
||||
'
|
||||
fi
|
||||
|
||||
@@ -617,6 +666,6 @@ cat <<EOF
|
||||
$ bash .agents/skills/multi-agent-mux-status/scripts/status.sh
|
||||
|
||||
3. Update installed skills in the future:
|
||||
$ bash .mam_deploy/update.sh
|
||||
$ bash .agents/skills/multi-agent-mux-update/scripts/update.sh
|
||||
--------------------------------------------------------------------------------
|
||||
EOF
|
||||
@@ -132,16 +132,12 @@ if [ -f "$SRC_DIR/.mam.env.example" ]; then
|
||||
log_ok "Copied .mam.env.example configuration template"
|
||||
fi
|
||||
|
||||
# Deploy remove.sh and update.sh into .mam_deploy/ (R-3)
|
||||
# Deploy remove.sh into .mam_deploy/ (R-3)
|
||||
mkdir -p "$TARGET_DIR/.mam_deploy"
|
||||
if [ -f "$SRC_DIR/deploy/remove.sh" ]; then
|
||||
cp "$SRC_DIR/deploy/remove.sh" "$TARGET_DIR/.mam_deploy/remove.sh"
|
||||
chmod 0755 "$TARGET_DIR/.mam_deploy/remove.sh"
|
||||
fi
|
||||
if [ -f "$SRC_DIR/deploy/update.sh" ]; then
|
||||
cp "$SRC_DIR/deploy/update.sh" "$TARGET_DIR/.mam_deploy/update.sh"
|
||||
chmod 0755 "$TARGET_DIR/.mam_deploy/update.sh"
|
||||
fi
|
||||
|
||||
if [ -f "$SRC_DIR/deploy/generate-env.sh" ]; then
|
||||
mkdir -p "$TARGET_DIR/scripts"
|
||||
@@ -161,7 +157,7 @@ mkdir -p "$TARGET_DIR/.mam"
|
||||
MANIFEST_FILE="$TARGET_DIR/.mam/install_manifest.txt"
|
||||
: > "$MANIFEST_FILE"
|
||||
( cd "$TARGET_DIR" && find .agents -type f -print ) >> "$MANIFEST_FILE"
|
||||
for extra in ".mam.env.example" ".mam_deploy/remove.sh" ".mam_deploy/update.sh" \
|
||||
for extra in ".mam.env.example" ".mam_deploy/remove.sh" \
|
||||
"scripts/generate-env.sh"; do
|
||||
if [ -e "$TARGET_DIR/$extra" ]; then
|
||||
echo "$extra" >> "$MANIFEST_FILE"
|
||||
|
||||
@@ -92,6 +92,7 @@ else
|
||||
".agents/skills/multi-agent-mux-resume"
|
||||
".agents/skills/multi-agent-mux-status"
|
||||
".agents/skills/multi-agent-mux-stop"
|
||||
".agents/skills/multi-agent-mux-update"
|
||||
".agents/hooks"
|
||||
".agents/hooks.json"
|
||||
".agents/MULTI_AGENT_RULES.md"
|
||||
|
||||
@@ -1,269 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# ==============================================================================
|
||||
# update.sh — Multi-Agent Mux (MAM) Orchestration Updater
|
||||
# ==============================================================================
|
||||
# Safely updates MAM skills, virtual environment, and docs to the latest version.
|
||||
# Preserves user configuration (.env) and local metadata/jobs database (.mam).
|
||||
# ==============================================================================
|
||||
set -euo pipefail
|
||||
|
||||
TARGET_DIR=""
|
||||
FORCE=0
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-y|--yes|--force)
|
||||
FORCE=1
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
TARGET_DIR="$1"
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "$TARGET_DIR" ]; then
|
||||
if [ "$(basename "$SCRIPT_DIR")" = ".mam_deploy" ]; then
|
||||
TARGET_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
else
|
||||
TARGET_DIR="$(pwd)"
|
||||
fi
|
||||
else
|
||||
if [ "$(basename "$TARGET_DIR")" = ".mam_deploy" ]; then
|
||||
TARGET_DIR="$(dirname "$TARGET_DIR")"
|
||||
fi
|
||||
if [ ! -d "$TARGET_DIR" ]; then
|
||||
echo "❌ Error: Target directory '$TARGET_DIR' does not exist." >&2
|
||||
exit 1
|
||||
fi
|
||||
TARGET_DIR="$(cd "$TARGET_DIR" && pwd)"
|
||||
fi
|
||||
|
||||
echo "===================================================================="
|
||||
echo "⚡ Starting Multi-Agent Mux (MAM) Update"
|
||||
echo "📂 Target Workspace: $TARGET_DIR"
|
||||
echo "===================================================================="
|
||||
|
||||
cd "$TARGET_DIR"
|
||||
|
||||
# 1. Verification of existing install (B-2: dual resolution)
|
||||
REMOVER=""
|
||||
for cand in ".mam_deploy/remove.sh" "remove.sh"; do
|
||||
if [ -f "$cand" ]; then
|
||||
REMOVER="$cand"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -z "$REMOVER" ]; then
|
||||
echo "❌ Error: No MAM installation (remove.sh) found in '$TARGET_DIR'." >&2
|
||||
echo " Please run install.sh first to set up the workspace." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Request confirmation if not forced
|
||||
if [ $FORCE -eq 0 ]; then
|
||||
echo "⚠️ WARNING: This will update MAM orchestration skills, virtual environment, "
|
||||
echo " and docs to the latest version."
|
||||
echo " (Your configuration, job history, and custom skills will be preserved)."
|
||||
if [ ! -t 0 ]; then
|
||||
echo "❌ Error: Non-interactive terminal detected. Please run with -y/--yes/--force." >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! read -p "❓ Proceed with update? [y/N]: " -r response; then
|
||||
response="n"
|
||||
fi
|
||||
if [[ ! "$response" =~ ^[yY](es)?$ ]]; then
|
||||
echo "❌ Update cancelled by user."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 1.5 Pre-capture legacy ownership before remove.sh deletes the manifest (M-4)
|
||||
MAM_LEGACY_ENV_OWNED=0
|
||||
if [ -f ".mam/install_manifest.txt" ] && grep -Fqx ".env" ".mam/install_manifest.txt" 2>/dev/null; then
|
||||
MAM_LEGACY_ENV_OWNED=1
|
||||
fi
|
||||
export MAM_LEGACY_ENV_OWNED
|
||||
|
||||
# 2. Stage backups of user configurations and metadata to prevent deletion
|
||||
echo "💾 Backing up configuration and database..."
|
||||
HAS_ENV=0
|
||||
ENV_BACKUP_SRC=""
|
||||
ENV_BACKUP_TMP=""
|
||||
|
||||
if [ -f ".mam.env" ]; then
|
||||
HAS_ENV=1
|
||||
ENV_BACKUP_SRC=".mam.env"
|
||||
ENV_BACKUP_TMP=".mam.env.update-tmp"
|
||||
mv ".mam.env" "$ENV_BACKUP_TMP"
|
||||
elif [ -f ".env" ]; then
|
||||
HAS_ENV=1
|
||||
ENV_BACKUP_SRC=".env"
|
||||
ENV_BACKUP_TMP=".env.update-tmp"
|
||||
mv ".env" "$ENV_BACKUP_TMP"
|
||||
fi
|
||||
|
||||
HAS_MAM=0
|
||||
if [ -d ".mam" ]; then
|
||||
HAS_MAM=1
|
||||
# Copy database and jobs to temporary backup outside of .mam.
|
||||
# We do NOT move the .mam folder away so that remove.sh can still read .mam/install_manifest.txt!
|
||||
mkdir -p .mam.update-tmp
|
||||
# Copy SQLite databases and session files
|
||||
for db in .mam/agent-sessions.*; do
|
||||
if [ -f "$db" ]; then
|
||||
cp -f "$db" .mam.update-tmp/
|
||||
fi
|
||||
done
|
||||
# Copy jobs history
|
||||
if [ -d ".mam/jobs" ] && [ "$(ls -A .mam/jobs 2>/dev/null)" ]; then
|
||||
mkdir -p .mam.update-tmp/jobs
|
||||
cp -rf .mam/jobs/* .mam.update-tmp/jobs/
|
||||
fi
|
||||
# Copy delegate logs
|
||||
if [ -d ".mam/delegate_job_logs" ] && [ "$(ls -A .mam/delegate_job_logs 2>/dev/null)" ]; then
|
||||
mkdir -p .mam.update-tmp/delegate_job_logs
|
||||
cp -rf .mam/delegate_job_logs/* .mam.update-tmp/delegate_job_logs/
|
||||
fi
|
||||
# Copy manifest so we have a backup
|
||||
if [ -f ".mam/install_manifest.txt" ]; then
|
||||
cp -f .mam/install_manifest.txt .mam.update-tmp/
|
||||
fi
|
||||
# C12: Copy MAM state files across update cycle
|
||||
for st in install_state asset_hashes.txt version.txt; do
|
||||
if [ -f ".mam/$st" ]; then
|
||||
cp -f ".mam/$st" .mam.update-tmp/
|
||||
fi
|
||||
done
|
||||
if [ -d ".mam/skill-backups" ]; then
|
||||
mkdir -p .mam.update-tmp/skill-backups
|
||||
cp -rf .mam/skill-backups/* .mam.update-tmp/skill-backups/ 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
|
||||
# Define trap to restore backup files on failure
|
||||
restore_on_failure() {
|
||||
echo "❌ Update failed. Reverting configuration and database to previous state..."
|
||||
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
|
||||
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC" 2>/dev/null || true
|
||||
fi
|
||||
if [ $HAS_MAM -eq 1 ] && [ -d ".mam.update-tmp" ]; then
|
||||
# Revert to old database/jobs backup by restoring .mam directory
|
||||
rm -rf .mam 2>/dev/null || true
|
||||
mkdir -p .mam
|
||||
cp -f .mam.update-tmp/agent-sessions.* .mam/ 2>/dev/null || true
|
||||
if [ -d ".mam.update-tmp/jobs" ]; then
|
||||
cp -rf .mam.update-tmp/jobs .mam/ 2>/dev/null || true
|
||||
fi
|
||||
if [ -d ".mam.update-tmp/delegate_job_logs" ]; then
|
||||
cp -rf .mam.update-tmp/delegate_job_logs .mam/ 2>/dev/null || true
|
||||
fi
|
||||
if [ -f ".mam.update-tmp/install_manifest.txt" ]; then
|
||||
cp -f .mam.update-tmp/install_manifest.txt .mam/ 2>/dev/null || true
|
||||
fi
|
||||
for st in install_state asset_hashes.txt version.txt; do
|
||||
if [ -f ".mam.update-tmp/$st" ]; then
|
||||
cp -f ".mam.update-tmp/$st" .mam/ 2>/dev/null || true
|
||||
fi
|
||||
done
|
||||
if [ -d ".mam.update-tmp/skill-backups" ]; then
|
||||
mkdir -p .mam/skill-backups
|
||||
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
|
||||
fi
|
||||
rm -rf .mam.update-tmp 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
trap restore_on_failure EXIT
|
||||
|
||||
# 3. Perform uninstallation of existing files
|
||||
echo "🗑️ Removing existing installation..."
|
||||
# remove.sh will run in manifest mode because .mam/install_manifest.txt is still present.
|
||||
# It will delete .agents/, documents, scripts, .venv, and .mam folder.
|
||||
bash "$REMOVER" --force "$TARGET_DIR"
|
||||
|
||||
# 4. Fetch and run the latest installer from Gitea
|
||||
echo "📥 Fetching and running the latest installer..."
|
||||
# Note: MAM_*_URL environment variables are automatically inherited by the child installer bash process.
|
||||
INSTALLER_URL="${MAM_INSTALLER_URL:-https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh}"
|
||||
if command -v curl &>/dev/null; then
|
||||
curl -fsSL "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
|
||||
elif command -v wget &>/dev/null; then
|
||||
wget -qO- "$INSTALLER_URL" | bash -s -- "$TARGET_DIR"
|
||||
else
|
||||
echo "❌ Error: Neither 'curl' nor 'wget' is available to fetch the installer." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Disable failure trap since installation succeeded
|
||||
trap - EXIT
|
||||
|
||||
# 5. Restore backups of configuration and database
|
||||
echo "🔄 Restoring configuration and database..."
|
||||
if [ $HAS_ENV -eq 1 ] && [ -n "$ENV_BACKUP_TMP" ] && [ -f "$ENV_BACKUP_TMP" ]; then
|
||||
if [ "$MAM_LEGACY_ENV_OWNED" = "1" ] && [ "$ENV_BACKUP_SRC" = ".env" ] && [ ! -f ".mam.env" ]; then
|
||||
mv -f "$ENV_BACKUP_TMP" ".mam.env"
|
||||
chmod 0600 ".mam.env" 2>/dev/null || true
|
||||
else
|
||||
mv -f "$ENV_BACKUP_TMP" "$ENV_BACKUP_SRC"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ $HAS_MAM -eq 1 ]; then
|
||||
if [ -d ".mam.update-tmp" ]; then
|
||||
# The installer created a new .mam directory with a new manifest.
|
||||
# We want to merge the old .mam database/jobs back while keeping the new manifest.
|
||||
for db in .mam.update-tmp/agent-sessions.*; do
|
||||
if [ -f "$db" ]; then
|
||||
cp -f "$db" .mam/
|
||||
fi
|
||||
done
|
||||
if [ -d ".mam.update-tmp/jobs" ] && [ "$(ls -A .mam.update-tmp/jobs 2>/dev/null)" ]; then
|
||||
mkdir -p .mam/jobs
|
||||
cp -rf .mam.update-tmp/jobs/* .mam/jobs/
|
||||
fi
|
||||
if [ -d ".mam.update-tmp/delegate_job_logs" ] && [ "$(ls -A .mam.update-tmp/delegate_job_logs 2>/dev/null)" ]; then
|
||||
mkdir -p .mam/delegate_job_logs
|
||||
cp -rf .mam.update-tmp/delegate_job_logs/* .mam/delegate_job_logs/
|
||||
fi
|
||||
# Issue #8: asset_hashes.txt and version.txt describe the tree the
|
||||
# installer just wrote, so the pre-update copies must NOT come back here -
|
||||
# restoring them made version.txt name the previous commit and remove.sh
|
||||
# treat every updated file as locally modified (spurious backups). Only
|
||||
# install_state carries history worth keeping: merge in any key the fresh
|
||||
# install did not write (e.g. gitignore_created from the original install).
|
||||
# The failure path (restore_on_failure) still puts all three back verbatim.
|
||||
if [ -f ".mam.update-tmp/install_state" ]; then
|
||||
touch .mam/install_state
|
||||
# A fresh install_state without a final newline would glue the first
|
||||
# merged key onto its last line.
|
||||
if [ -s .mam/install_state ] && [ -n "$(tail -c1 .mam/install_state)" ]; then
|
||||
printf '\n' >> .mam/install_state
|
||||
fi
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
case "$line" in
|
||||
*=*) ;;
|
||||
*) continue ;; # not a key=value line
|
||||
esac
|
||||
key="${line%%=*}"
|
||||
[ -n "$key" ] || continue
|
||||
# exact key match, no regex interpretation of the key
|
||||
if ! awk -F= -v k="$key" '$1 == k { found = 1 } END { exit !found }' .mam/install_state; then
|
||||
printf '%s\n' "$line" >> .mam/install_state
|
||||
fi
|
||||
done < .mam.update-tmp/install_state
|
||||
fi
|
||||
if [ -d ".mam.update-tmp/skill-backups" ]; then
|
||||
mkdir -p .mam/skill-backups
|
||||
cp -rf .mam.update-tmp/skill-backups/* .mam/skill-backups/ 2>/dev/null || true
|
||||
fi
|
||||
rm -rf ".mam.update-tmp"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "===================================================================="
|
||||
echo "🎉 Update complete!"
|
||||
echo "===================================================================="
|
||||
@@ -463,6 +463,20 @@
|
||||
"evidence": [],
|
||||
"mismatch": null,
|
||||
"note": "post-stop checks; no script offers them as a command - stage 3"
|
||||
},
|
||||
{
|
||||
"skill": "multi-agent-mux-update",
|
||||
"start": 17,
|
||||
"end": 19,
|
||||
"lang": "bash",
|
||||
"sha": "63f73ade58d3",
|
||||
"class": "c",
|
||||
"kind": "invocation",
|
||||
"evidence": [
|
||||
"multi-agent-mux-update/scripts/update.sh:3"
|
||||
],
|
||||
"mismatch": null,
|
||||
"note": "entry point call (Issue #72)"
|
||||
}
|
||||
],
|
||||
"removed": [
|
||||
|
||||
@@ -48,7 +48,7 @@ class TestDeployLayout(unittest.TestCase):
|
||||
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".agents", "skills", "lib.sh")))
|
||||
|
||||
def test_td6_td7_td8_mam_deploy_layout_and_removal(self):
|
||||
"""T-D6~D8: Verify remove.sh/update.sh installed in .mam_deploy and work properly."""
|
||||
"""T-D6~D8: Verify remove.sh installed in .mam_deploy (update.sh not, Issue #72) and works properly."""
|
||||
subprocess.run(
|
||||
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
|
||||
env=self.env,
|
||||
@@ -59,7 +59,7 @@ class TestDeployLayout(unittest.TestCase):
|
||||
remover = os.path.join(self.test_dir, ".mam_deploy", "remove.sh")
|
||||
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
|
||||
self.assertTrue(os.path.exists(remover))
|
||||
self.assertTrue(os.path.exists(updater))
|
||||
self.assertFalse(os.path.exists(updater))
|
||||
self.assertTrue(os.access(remover, os.X_OK))
|
||||
|
||||
# T-D8: Execution from inside .mam_deploy directory targets parent workspace
|
||||
@@ -129,17 +129,29 @@ class TestDeployLayout(unittest.TestCase):
|
||||
self.assertTrue(os.path.exists(backups_dir))
|
||||
|
||||
def test_td27_td28_update_preserves_mam_state(self):
|
||||
"""T-D27~D28: Verify update.sh preserves .mam state files across cycles (B-5 gate)."""
|
||||
"""T-D27~D28: Verify the update skill preserves .mam state files across cycles (B-5 gate)."""
|
||||
subprocess.run(
|
||||
["bash", os.path.join(self.repo_root, "deploy/install.sh"), self.test_dir],
|
||||
env=self.env,
|
||||
check=True
|
||||
)
|
||||
|
||||
# Run update.sh
|
||||
updater = os.path.join(self.test_dir, ".mam_deploy", "update.sh")
|
||||
res = subprocess.run(["bash", updater, "--force"], cwd=self.test_dir, env=self.env, capture_output=True, text=True)
|
||||
self.assertEqual(res.returncode, 0, f"update.sh failed: {res.stderr}")
|
||||
# Run the update skill (Issue #72: .mam_deploy/update.sh is gone)
|
||||
updater = os.path.join(self.test_dir, ".agents", "skills", "multi-agent-mux-update", "scripts", "update.sh")
|
||||
env = {k: v for k, v in self.env.items() if not k.startswith("HERDR_")
|
||||
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
|
||||
env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
|
||||
# The update lists the live herdr agents; never ask a real server here.
|
||||
fake = os.path.join(self.temp_dir, "fake-herdr-bin")
|
||||
os.makedirs(fake)
|
||||
with open(os.path.join(fake, "herdr"), "w") as f:
|
||||
f.write('#!/bin/sh\necho \'{"sessions": []}\'\n')
|
||||
os.chmod(os.path.join(fake, "herdr"), 0o755)
|
||||
env["PATH"] = fake + ":" + env["PATH"]
|
||||
res = subprocess.run(["bash", updater, "--yes"], cwd=self.test_dir, env=env, capture_output=True, text=True,
|
||||
stdin=subprocess.DEVNULL)
|
||||
self.assertEqual(res.returncode, 0, f"the update skill failed: {res.stdout}{res.stderr}")
|
||||
self.assertFalse(os.path.exists(os.path.join(self.test_dir, ".mam_deploy", "update.sh")))
|
||||
|
||||
# Verify asset_hashes.txt and version.txt still exist
|
||||
self.assertTrue(os.path.exists(os.path.join(self.test_dir, ".mam", "asset_hashes.txt")))
|
||||
|
||||
@@ -115,8 +115,8 @@ def test_inventory_matches_skill_md(skill):
|
||||
f"actual (start, end, lang, sha): {json.dumps(actual)}")
|
||||
|
||||
|
||||
def test_inventory_covers_eight_skills():
|
||||
assert len(SKILL_NAMES) == 8
|
||||
def test_inventory_covers_nine_skills():
|
||||
assert len(SKILL_NAMES) == 9 # Issue #72: + multi-agent-mux-update
|
||||
assert {b["skill"] for b in DATA["blocks"]} == set(SKILL_NAMES)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Issue #72 — the installer side of the multi-agent-mux-update skill.
|
||||
|
||||
The update skill runs the new install.sh in place over an existing install
|
||||
(no remove.sh, .mam kept). These pin what that in-place run must do:
|
||||
install_state keeps one line per key, no updater is shipped any more and an
|
||||
old one is cleaned up, and framework files the new version no longer ships
|
||||
are deleted unless the user modified them.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
# Extra framework-owned files only the "old" source ships.
|
||||
RETIRED_SKILL = ".agents/skills/multi-agent-mux-retired72"
|
||||
RETIRED_UNMODIFIED = [RETIRED_SKILL + "/SKILL.md", RETIRED_SKILL + "/scripts/old.sh"]
|
||||
RETIRED_SHARED_DIR = ".agents/skills/multi-agent-mux-retired72b"
|
||||
RETIRED_SHARED = RETIRED_SHARED_DIR + "/SKILL.md"
|
||||
RETIRED_MODIFIED = ".agents/hooks/retired72_hook.sh"
|
||||
|
||||
|
||||
def _copy_tree(dest):
|
||||
shutil.copytree(REPO_ROOT, dest,
|
||||
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", ".mam_deploy",
|
||||
"__pycache__", ".pytest_cache"),
|
||||
symlinks=True)
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def sources():
|
||||
"""(new, old): the tree as is, and the same tree plus files it no longer ships."""
|
||||
tmp = tempfile.mkdtemp(prefix="mam_issue72_src_")
|
||||
new, old = os.path.join(tmp, "new"), os.path.join(tmp, "old")
|
||||
_copy_tree(new)
|
||||
_copy_tree(old)
|
||||
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
|
||||
path = os.path.join(old, rel)
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w") as f:
|
||||
f.write("# shipped by the old version only: %s\n" % rel)
|
||||
try:
|
||||
yield new, old
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws():
|
||||
tmp = tempfile.mkdtemp(prefix="mam_issue72_ws_")
|
||||
try:
|
||||
yield os.path.join(tmp, "workspace")
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
|
||||
|
||||
def _install(src, ws):
|
||||
env = dict(os.environ)
|
||||
env.update({"MAM_REPO_URL": src, "MAM_SKIP_VENV": "1"})
|
||||
res = subprocess.run(["bash", os.path.join(src, "deploy", "install.sh"), ws],
|
||||
env=env, capture_output=True, text=True, timeout=600)
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
return res
|
||||
|
||||
|
||||
def _remove(ws):
|
||||
res = subprocess.run(["bash", os.path.join(ws, ".mam_deploy", "remove.sh"), "--force"],
|
||||
cwd=ws, capture_output=True, text=True, timeout=600)
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
return res
|
||||
|
||||
|
||||
def _read(ws, rel):
|
||||
with open(os.path.join(ws, rel)) as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
def _manifest(ws):
|
||||
return _read(ws, ".mam/install_manifest.txt").splitlines()
|
||||
|
||||
|
||||
def _hashed_paths(ws):
|
||||
return [line.split(None, 1)[1] for line in _read(ws, ".mam/asset_hashes.txt").splitlines()
|
||||
if line.strip()]
|
||||
|
||||
|
||||
def test_in_place_reinstall_keeps_one_line_per_install_state_key(sources, ws):
|
||||
new, _ = sources
|
||||
_install(new, ws)
|
||||
with open(os.path.join(ws, ".mam", "install_state"), "w") as f:
|
||||
f.write("first=a\ngitignore_created=1\nlast=b\n")
|
||||
|
||||
_install(new, ws)
|
||||
|
||||
assert _read(ws, ".mam/install_state").splitlines() == [
|
||||
"first=a", "gitignore_created=1", "last=b"]
|
||||
# remove.sh must read the key as 1: the .gitignore it created goes away.
|
||||
_remove(ws)
|
||||
assert not os.path.exists(os.path.join(ws, ".gitignore"))
|
||||
|
||||
|
||||
def test_fresh_install_ships_no_updater(sources, ws):
|
||||
new, _ = sources
|
||||
res = _install(new, ws)
|
||||
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
|
||||
manifest = _manifest(ws)
|
||||
assert ".mam_deploy/update.sh" not in manifest and "update.sh" not in manifest
|
||||
assert "bash .mam_deploy/update.sh" not in res.stdout
|
||||
assert "multi-agent-mux-update/scripts/update.sh" in res.stdout
|
||||
|
||||
|
||||
def test_in_place_reinstall_deletes_an_old_mam_deploy_updater(sources, ws):
|
||||
new, _ = sources
|
||||
_install(new, ws)
|
||||
with open(os.path.join(ws, ".mam_deploy", "update.sh"), "w") as f:
|
||||
f.write("#!/usr/bin/env bash\n# old updater\n")
|
||||
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
|
||||
f.write(".mam_deploy/update.sh\n")
|
||||
|
||||
_install(new, ws)
|
||||
|
||||
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
|
||||
assert ".mam_deploy/update.sh" not in _manifest(ws)
|
||||
|
||||
|
||||
def test_legacy_root_updater_goes_only_when_the_manifest_lists_it(sources, ws):
|
||||
new, _ = sources
|
||||
_install(new, ws)
|
||||
root_updater = os.path.join(ws, "update.sh")
|
||||
|
||||
# Listed: the installer put it there (very old layout).
|
||||
with open(root_updater, "w") as f:
|
||||
f.write("# legacy updater\n")
|
||||
with open(os.path.join(ws, ".mam", "install_manifest.txt"), "a") as f:
|
||||
f.write("update.sh\n")
|
||||
_install(new, ws)
|
||||
assert not os.path.exists(root_updater)
|
||||
assert not os.path.exists(os.path.join(ws, ".mam_deploy", "update.sh"))
|
||||
manifest = _manifest(ws)
|
||||
assert "update.sh" not in manifest and ".mam_deploy/update.sh" not in manifest
|
||||
|
||||
# Not listed: the user's own file stays untouched.
|
||||
with open(root_updater, "w") as f:
|
||||
f.write("# user's own script\n")
|
||||
_install(new, ws)
|
||||
assert _read(ws, "update.sh") == "# user's own script\n"
|
||||
|
||||
|
||||
def test_files_the_new_version_no_longer_ships(sources, ws):
|
||||
new, old = sources
|
||||
_install(old, ws)
|
||||
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
|
||||
assert rel in _manifest(ws) and rel in _hashed_paths(ws), rel
|
||||
# The user edits one retired file and adds their own file next to another.
|
||||
with open(os.path.join(ws, RETIRED_MODIFIED), "a") as f:
|
||||
f.write("# my change\n")
|
||||
with open(os.path.join(ws, RETIRED_SHARED_DIR, "notes.txt"), "w") as f:
|
||||
f.write("mine\n")
|
||||
# A base/ copy of a retired file must go with it.
|
||||
base_copy = os.path.join(ws, ".mam", "base", RETIRED_UNMODIFIED[0])
|
||||
os.makedirs(os.path.dirname(base_copy))
|
||||
shutil.copy(os.path.join(ws, RETIRED_UNMODIFIED[0]), base_copy)
|
||||
|
||||
res = _install(new, ws)
|
||||
|
||||
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED]:
|
||||
assert not os.path.exists(os.path.join(ws, rel)), rel
|
||||
assert not os.path.exists(os.path.join(ws, RETIRED_SKILL)), "emptied directory left behind"
|
||||
assert _read(ws, RETIRED_SHARED_DIR + "/notes.txt") == "mine\n"
|
||||
assert "# my change" in _read(ws, RETIRED_MODIFIED)
|
||||
assert "Local modification detected" in res.stderr and RETIRED_MODIFIED in res.stderr
|
||||
for rel in RETIRED_UNMODIFIED:
|
||||
assert rel not in res.stderr, rel
|
||||
assert not os.path.exists(base_copy)
|
||||
manifest, hashed = _manifest(ws), _hashed_paths(ws)
|
||||
for rel in RETIRED_UNMODIFIED + [RETIRED_SHARED, RETIRED_MODIFIED]:
|
||||
assert rel not in manifest and rel not in hashed, rel
|
||||
# Still-shipped framework files are untouched by the cleanup.
|
||||
assert ".agents/skills/lib.sh" in manifest and ".agents/skills/lib.sh" in hashed
|
||||
|
||||
# The kept file is the user's now: uninstalling leaves it in place.
|
||||
_remove(ws)
|
||||
assert "# my change" in _read(ws, RETIRED_MODIFIED)
|
||||
|
||||
|
||||
@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0,
|
||||
reason="root reads a mode-000 file")
|
||||
def test_local_source_update_backup_is_not_staged(sources, ws):
|
||||
new, _ = sources
|
||||
backup = os.path.join(new, ".mam", "update-backup")
|
||||
os.makedirs(backup)
|
||||
unreadable = os.path.join(backup, "unreadable")
|
||||
with open(unreadable, "w") as f:
|
||||
f.write("x")
|
||||
os.chmod(unreadable, 0)
|
||||
try:
|
||||
_install(new, ws)
|
||||
finally:
|
||||
os.chmod(unreadable, 0o600)
|
||||
shutil.rmtree(os.path.join(new, ".mam"))
|
||||
@@ -0,0 +1,710 @@
|
||||
"""Issue #72 - the multi-agent-mux-update skill.
|
||||
|
||||
Every test updates a copy of one workspace installed from this checkout
|
||||
(MAM_REPO_URL, MAM_SKIP_VENV=1). herdr is a fake that answers `session list
|
||||
--json` and `--session S agent list` from a JSON spec, so the pre-check's view
|
||||
of the live agents is set per test; the monitor daemon is a stub reconcile.sh
|
||||
that holds .mam/monitor.lock like the real one. Nothing here touches a real
|
||||
herdr server or the control root's monitor.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
UPDATE_REL = ".agents/skills/multi-agent-mux-update/scripts/update.sh"
|
||||
MONITOR_REL = ".agents/skills/multi-agent-mux-monitor/scripts/reconcile.sh"
|
||||
|
||||
FAKE_HERDR = r'''#!/usr/bin/env python3
|
||||
import json, os, sys
|
||||
spec = json.load(open(os.environ["FAKE_HERDR_SPEC"]))
|
||||
a = sys.argv[1:]
|
||||
server = "default"
|
||||
if a[:1] == ["--session"]:
|
||||
server, a = a[1], a[2:]
|
||||
with open(os.environ["FAKE_HERDR_SPEC"] + ".calls", "a") as f:
|
||||
f.write(json.dumps([server] + a) + "\n")
|
||||
if a[:2] == ["session", "list"]:
|
||||
if spec.get("fail") == "session-list":
|
||||
sys.exit(1)
|
||||
print(json.dumps({"sessions": [{"name": n, "running": r} for n, r in spec.get("sessions", {"default": True}).items()]}))
|
||||
elif a[:2] == ["agent", "list"]:
|
||||
if spec.get("fail") == "agent-list:" + server:
|
||||
sys.exit(1)
|
||||
print(json.dumps({"result": {"agents": spec.get("agents", {}).get(server, [])}}))
|
||||
'''
|
||||
|
||||
STUB_MONITOR = r'''#!/usr/bin/env bash
|
||||
# Issue #72 test stub of reconcile.sh: a wrapper whose python child holds monitor.lock.
|
||||
d="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd -P)"
|
||||
echo "start $$ $*" >> "$d/.mam/stub-monitor.log"
|
||||
python3 -c '
|
||||
import fcntl, os, signal, sys, time
|
||||
if os.environ.get("STUB_MONITOR_IGNORE_TERM") == "1":
|
||||
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||
f = open(sys.argv[1], "w")
|
||||
fcntl.flock(f, fcntl.LOCK_EX)
|
||||
time.sleep(600)
|
||||
' "$d/.mam/monitor.lock" &
|
||||
child=$!
|
||||
if [ "${STUB_MONITOR_IGNORE_TERM:-0}" = "1" ]; then
|
||||
trap '' TERM
|
||||
else
|
||||
trap 'kill $child 2>/dev/null; exit 143' TERM
|
||||
fi
|
||||
while kill -0 "$child" 2>/dev/null; do wait "$child"; done
|
||||
'''
|
||||
|
||||
|
||||
def _clean_env():
|
||||
env = {k: v for k, v in os.environ.items()
|
||||
if not k.startswith(("HERDR_", "XDG_", "MAM_", "STUB_"))
|
||||
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "DELEGATE_JOB_LOGS_DIR", "PYTHONPATH")}
|
||||
env.update(MAM_REPO_URL=str(REPO_ROOT), MAM_SKIP_VENV="1",
|
||||
MAM_INSTALLER_URL="file://" + str(REPO_ROOT / "deploy" / "install.sh"),
|
||||
MAM_ENV_FILE="/nonexistent/.mam.env", PYTHONDONTWRITEBYTECODE="1")
|
||||
return env
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def template(tmp_path_factory):
|
||||
ws = tmp_path_factory.mktemp("tpl72") / "ws"
|
||||
ws.mkdir()
|
||||
res = subprocess.run(["bash", str(REPO_ROOT / "deploy" / "install.sh"), str(ws)], env=_clean_env(),
|
||||
capture_output=True, text=True, timeout=600)
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
return ws
|
||||
|
||||
|
||||
class Box:
|
||||
def __init__(self, root: Path, ws: Path):
|
||||
self.root, self.ws = root, ws
|
||||
self.mam = ws / ".mam"
|
||||
self.bin = root / "bin"
|
||||
self.spec = root / "herdr.json"
|
||||
self.env = _clean_env()
|
||||
self.env["PATH"] = "%s:%s" % (self.bin, os.environ["PATH"])
|
||||
self.env["FAKE_HERDR_SPEC"] = str(self.spec)
|
||||
self.helpers = []
|
||||
self.herdr()
|
||||
|
||||
def herdr(self, sessions=None, agents=None, fail=None):
|
||||
self.spec.write_text(json.dumps({"sessions": sessions or {"default": True},
|
||||
"agents": agents or {}, "fail": fail}))
|
||||
|
||||
def update(self, *args, env=None, timeout=300):
|
||||
e = dict(self.env, **(env or {}))
|
||||
return subprocess.run(["bash", str(self.ws / UPDATE_REL), "--yes", *args], cwd=str(self.ws), env=e,
|
||||
stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=timeout)
|
||||
|
||||
def lib(self, script, env=None, timeout=120):
|
||||
e = dict(self.env, WORKSPACE_ROOT=str(self.ws), **(env or {}))
|
||||
return subprocess.run(["bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(self.ws)],
|
||||
env=e, capture_output=True, text=True, timeout=timeout)
|
||||
|
||||
def installer(self, body):
|
||||
p = self.root / "installer.sh"
|
||||
p.write_text("#!/usr/bin/env bash\nws=\"$1\"\n" + body)
|
||||
self.env["MAM_INSTALLER_URL"] = "file://" + str(p)
|
||||
|
||||
def helper(self, *argv, **kw):
|
||||
p = subprocess.Popen(list(argv), start_new_session=True, **kw)
|
||||
self.helpers.append(p)
|
||||
return p
|
||||
|
||||
def stub_monitor(self):
|
||||
p = self.ws / MONITOR_REL
|
||||
p.write_text(STUB_MONITOR)
|
||||
|
||||
def start_monitor(self, env=None):
|
||||
"""The way create and resume start it: lib.sh start_watchdog (detached, so
|
||||
not a child of this test that would linger as a zombie)."""
|
||||
res = self.lib('start_watchdog test "$WORKSPACE_ROOT"', env=dict(env or {}))
|
||||
assert res.returncode == 0 and res.stdout.strip(), res.stdout + res.stderr
|
||||
_until(lambda: _lock_held(self.mam / "monitor.lock"))
|
||||
|
||||
def monitor_pids(self):
|
||||
r = subprocess.run(["pgrep", "-f", "bash %s --subscribe" % (self.ws / MONITOR_REL)],
|
||||
capture_output=True, text=True)
|
||||
return r.stdout.split()
|
||||
|
||||
def monitor_starts(self):
|
||||
p = self.mam / "stub-monitor.log"
|
||||
return p.read_text().count("start ") if p.exists() else 0
|
||||
|
||||
|
||||
def _until(cond, timeout=15.0):
|
||||
end = time.monotonic() + timeout
|
||||
while time.monotonic() < end:
|
||||
if cond():
|
||||
return True
|
||||
time.sleep(0.1)
|
||||
raise AssertionError("condition not met within %ss" % timeout)
|
||||
|
||||
|
||||
def _lock_held(path: Path) -> bool:
|
||||
if not path.exists():
|
||||
return False
|
||||
with open(path, "a") as fh:
|
||||
try:
|
||||
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
fcntl.flock(fh, fcntl.LOCK_UN)
|
||||
return False
|
||||
except BlockingIOError:
|
||||
return True
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def box(template, tmp_path):
|
||||
root = Path(os.path.realpath(tmp_path))
|
||||
ws = root / "ws"
|
||||
shutil.copytree(template, ws, symlinks=True)
|
||||
(root / "bin").mkdir()
|
||||
herdr = root / "bin" / "herdr"
|
||||
herdr.write_text(FAKE_HERDR)
|
||||
herdr.chmod(0o755)
|
||||
b = Box(root, ws)
|
||||
try:
|
||||
yield b
|
||||
finally:
|
||||
for p in b.helpers:
|
||||
try:
|
||||
os.killpg(p.pid, signal.SIGKILL)
|
||||
except OSError:
|
||||
pass
|
||||
subprocess.run(["pkill", "-KILL", "-f", str(ws)], capture_output=True)
|
||||
|
||||
|
||||
# --- helpers ---------------------------------------------------------------------
|
||||
|
||||
SKIP_SNAPSHOT = ("shim/", "update-admission.lock", "agent-sessions.db.lock", "monitor.lock", "stub-monitor.log")
|
||||
|
||||
|
||||
def snapshot(ws: Path) -> dict:
|
||||
out = {}
|
||||
for dp, dns, fns in os.walk(ws):
|
||||
dns[:] = [d for d in dns if d != "__pycache__"]
|
||||
for f in fns:
|
||||
p = Path(dp) / f
|
||||
rel = str(p.relative_to(ws))
|
||||
if rel.startswith(".mam/") and any(s in rel for s in SKIP_SNAPSHOT):
|
||||
continue
|
||||
out[rel] = "L:" + os.readlink(p) if p.is_symlink() else hashlib.sha256(p.read_bytes()).hexdigest()
|
||||
return out
|
||||
|
||||
|
||||
def write_rows(ws: Path, rows, **top):
|
||||
d = dict(top, herdr_sessions=rows)
|
||||
(ws / ".mam" / "agent-sessions.yaml").write_text(yaml.safe_dump(d, sort_keys=False))
|
||||
|
||||
|
||||
def row(name, status="running", server="default", cwd=None, **extra):
|
||||
r = {"name": name, "status": status, "role": "creator", "herdr_session": server, "herdr_server": server,
|
||||
"pane": {"cwd": cwd or "/elsewhere", "cmd": "claude"}}
|
||||
r.update(extra)
|
||||
return r
|
||||
|
||||
|
||||
def write_job(d: Path, jid, status="running", session="herdr:x", **extra):
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
rec = dict({"job_id": jid, "status": status, "agent_session": session}, **extra)
|
||||
(d / (jid + ".json")).write_text(json.dumps(rec))
|
||||
|
||||
|
||||
def live_marker(path: Path, box: Box):
|
||||
"""A marker owned by a live process (pid + lstart as loop_lock.sh writes them)."""
|
||||
p = box.helper("sleep", "120")
|
||||
ls = subprocess.run(["ps", "-p", str(p.pid), "-o", "lstart="], capture_output=True, text=True).stdout
|
||||
ls = " ".join(ls.split())
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("pid=%d\nlstart=%s\nstarted=x\n" % (p.pid, ls))
|
||||
return p
|
||||
|
||||
|
||||
def dead_marker(path: Path):
|
||||
p = subprocess.Popen(["true"])
|
||||
p.wait()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("pid=%d\nlstart=Thu Jan 1 00:00:00 1970\nstarted=x\n" % p.pid)
|
||||
|
||||
|
||||
def assert_refused_unchanged(box: Box, before: dict, res, *needles):
|
||||
assert res.returncode == 1, res.stdout + res.stderr
|
||||
assert "update refused; nothing was changed" in res.stderr, res.stderr
|
||||
for n in needles:
|
||||
assert n in res.stderr, (n, res.stderr)
|
||||
assert snapshot(box.ws) == before
|
||||
assert not (box.mam / "update-in-progress").exists(), "the refusing update releases its marker"
|
||||
assert not (box.mam / "update-backup").exists()
|
||||
|
||||
|
||||
# --- success and the #8 state rules ------------------------------------------------
|
||||
|
||||
def test_update_in_place_keeps_mam_and_merges_install_state(box):
|
||||
mam = box.mam
|
||||
(mam / "version.txt").write_text("source=old\ncommit=stale0000\nfetched_at=then\nmethod=git\n")
|
||||
(mam / "herdr_workspace_id").write_text("w9\n")
|
||||
for d in ("briefs", "diagnostics", "resumelogs", "stoplogs", "jobs", "delegate_job_logs/j1"):
|
||||
(mam / d).mkdir(parents=True, exist_ok=True)
|
||||
(mam / d / "keep.txt").write_text(d)
|
||||
write_job(mam / "jobs", "done1", status="completed")
|
||||
(mam / "multi-agent-mux-monitor.log").write_text("log\n")
|
||||
(box.ws / ".mam.env").write_text("MQTT_BROKER=example\n")
|
||||
with open(mam / "install_state", "a") as fh:
|
||||
fh.write("custom_marker=kept\n")
|
||||
write_rows(box.ws, [row("a", status="stopped")], orchestrator_uuids=["u1"])
|
||||
keep = {k: v for k, v in snapshot(box.ws).items()
|
||||
if k.startswith((".mam/herdr_workspace_id", ".mam/briefs", ".mam/diagnostics", ".mam/resumelogs",
|
||||
".mam/stoplogs", ".mam/jobs", ".mam/delegate_job_logs", ".mam/multi-agent-mux-monitor.log"))
|
||||
or k == ".mam.env"}
|
||||
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
after = snapshot(box.ws)
|
||||
assert {k: after.get(k) for k in keep} == keep, "everything that is not the installer's is kept"
|
||||
assert "stale0000" not in (mam / "version.txt").read_text()
|
||||
state = (mam / "install_state").read_text().splitlines()
|
||||
assert "custom_marker=kept" in state and state.count("gitignore_created=1") == 1, state
|
||||
assert not (box.ws / ".mam_deploy" / "update.sh").exists()
|
||||
assert not (mam / "update-backup").exists() and not (mam / "update-in-progress").exists()
|
||||
assert "Update complete." in res.stdout
|
||||
|
||||
|
||||
def test_old_format_registry_is_converted_by_the_new_code(box):
|
||||
old = row("old-one", status="stopped", start_command="HERDR_SESSION_NAME=x herdr new-session",
|
||||
kill_command="HERDR_SESSION_NAME=x herdr kill-session", attach_command="HERDR_SESSION_NAME=x herdr agent attach old-one")
|
||||
write_rows(box.ws, [old])
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "registry-format: 1 -> 2, rows changed: 1" in res.stdout
|
||||
y = yaml.safe_load((box.mam / "agent-sessions.yaml").read_text())
|
||||
assert y["registry_format"] == 2
|
||||
conn = sqlite3.connect(str(box.mam / "agent-sessions.db"))
|
||||
try:
|
||||
db = {r["name"]: r for r in (json.loads(x[0]) for x in conn.execute("SELECT data FROM sessions"))}
|
||||
st = json.loads(conn.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
|
||||
finally:
|
||||
conn.close()
|
||||
assert {r["name"]: r for r in y["herdr_sessions"]} == db and st["registry_format"] == 2
|
||||
assert "kill_command" not in db["old-one"] and "start_command" not in db["old-one"]
|
||||
|
||||
|
||||
def test_no_registry_is_nothing_to_convert(box):
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "registry: nothing to convert (no registry)" in res.stdout
|
||||
assert not (box.mam / "agent-sessions.yaml").exists()
|
||||
|
||||
|
||||
def test_a_failed_conversion_keeps_the_install_and_exits_nonzero(box, tmp_path):
|
||||
src = tmp_path / "src"
|
||||
shutil.copytree(REPO_ROOT, src, symlinks=True,
|
||||
ignore=shutil.ignore_patterns(".git", ".venv", ".mam", "__pycache__", "tests", "nats-docker"))
|
||||
fmt = src / ".agents" / "skills" / "lib_py" / "registry_format.py"
|
||||
fmt.write_text("import sys\nif __name__ == '__main__':\n sys.exit(3)\n" + fmt.read_text())
|
||||
box.env["MAM_REPO_URL"] = str(src)
|
||||
box.stub_monitor()
|
||||
box.start_monitor()
|
||||
write_rows(box.ws, [row("a", status="stopped")])
|
||||
res = box.update()
|
||||
assert res.returncode == 3, res.stdout + res.stderr
|
||||
assert "the registry was not converted" in res.stderr
|
||||
assert (box.ws / ".agents/skills/lib_py/registry_format.py").read_text() == fmt.read_text(), "install kept"
|
||||
assert box.monitor_pids() and box.monitor_starts() == 2, "the monitor is started again anyway"
|
||||
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
|
||||
|
||||
|
||||
def test_refuses_outside_an_install_and_without_yes(box):
|
||||
(box.ws / ".mam_deploy" / "remove.sh").unlink()
|
||||
res = box.update()
|
||||
assert res.returncode == 1 and "not an installed MAM workspace" in res.stderr
|
||||
shutil.copy(REPO_ROOT / "deploy" / "remove.sh", box.ws / ".mam_deploy" / "remove.sh")
|
||||
res = subprocess.run(["bash", str(box.ws / UPDATE_REL)], env=box.env, stdin=subprocess.DEVNULL,
|
||||
capture_output=True, text=True, timeout=60)
|
||||
assert res.returncode == 1 and "-y/--yes" in res.stderr
|
||||
res = box.update("--force")
|
||||
assert res.returncode == 2, "--force is not an alias of --yes"
|
||||
|
||||
|
||||
# --- refusals (decision 1) ---------------------------------------------------------
|
||||
|
||||
def test_refuses_a_running_row(box):
|
||||
write_rows(box.ws, [row("busy-creator", cwd="/somewhere/else")])
|
||||
before = snapshot(box.ws)
|
||||
res = box.update()
|
||||
assert_refused_unchanged(box, before, res, "session busy-creator is running", "stop_session.sh --session busy-creator")
|
||||
|
||||
|
||||
def test_refuses_an_unreadable_registry(box):
|
||||
(box.mam / "agent-sessions.db").write_text("not a database")
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "cannot be read")
|
||||
|
||||
|
||||
def test_refuses_a_rowless_agent_on_any_running_server(box):
|
||||
box.herdr(sessions={"default": True, "picked-by-hand": True, "down": False},
|
||||
agents={"picked-by-hand": [{"name": "stray", "pane_id": "w3:p1", "cwd": str(box.ws / "sub")}],
|
||||
"down": [{"name": "ghost", "pane_id": "w4:p1", "cwd": str(box.ws)}]})
|
||||
before = snapshot(box.ws)
|
||||
res = box.update()
|
||||
assert_refused_unchanged(box, before, res, "herdr agent stray (pane w3:p1, server picked-by-hand)")
|
||||
assert "ghost" not in res.stderr, "a server that is not running is not asked"
|
||||
calls = (box.root / "herdr.json.calls").read_text()
|
||||
assert '"down"' not in calls
|
||||
|
||||
|
||||
def test_an_agent_outside_the_workspace_does_not_refuse(box):
|
||||
box.herdr(agents={"default": [{"name": "other", "pane_id": "w1:p2", "cwd": str(box.root / "elsewhere")}]})
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fail", ["session-list", "agent-list:default"])
|
||||
def test_refuses_when_herdr_cannot_answer(box, fail):
|
||||
box.herdr(fail=fail)
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "cannot list the live agents")
|
||||
|
||||
|
||||
def test_open_jobs_refuse_and_terminal_or_dry_run_do_not(box, tmp_path):
|
||||
jobs = box.mam / "jobs"
|
||||
write_job(jobs, "done1", status="completed")
|
||||
write_job(jobs, "err1", status="error")
|
||||
write_job(jobs, "can1", status="cancelled")
|
||||
write_job(jobs, "dry1", status="pending", dry_run=True)
|
||||
assert box.update().returncode == 0
|
||||
|
||||
write_job(jobs, "open1", status="pending", session="remote:x")
|
||||
before = snapshot(box.ws)
|
||||
res = box.update()
|
||||
assert_refused_unchanged(box, before, res, "job open1", "close --job open1 --request cancelled")
|
||||
(jobs / "open1.json").unlink()
|
||||
|
||||
(jobs / "broken.json").write_text("{not json")
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "job record %s cannot be read" % (jobs / "broken.json"))
|
||||
(jobs / "broken.json").unlink()
|
||||
|
||||
extra = tmp_path / "mam-jobs-dir"
|
||||
write_job(extra, "env1", status="running")
|
||||
box.env["MAM_JOBS_DIR"] = str(extra)
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "job env1")
|
||||
del box.env["MAM_JOBS_DIR"]
|
||||
|
||||
|
||||
def test_other_control_planes_count_only_jobs_addressed_to_this_registry(box, tmp_path):
|
||||
other = tmp_path / "other-plane" / "jobs"
|
||||
write_job(other, "theirs", status="running", session="herdr:not-ours")
|
||||
write_rows(box.ws, [row("wt-creator", status="stopped", control_plane={"mam_jobs_dir": str(other)})])
|
||||
assert box.update().returncode == 0, "a job of another registry's session is not ours"
|
||||
write_job(other, "ours", status="running", session="herdr:wt-creator")
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "job ours")
|
||||
|
||||
|
||||
def test_loop_marker_live_refuses_stale_does_not(box):
|
||||
live_marker(box.mam / "loop-guard-active", box)
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(), "a review loop is running")
|
||||
dead_marker(box.mam / "loop-guard-active")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "stale loop marker" in res.stdout
|
||||
|
||||
|
||||
# --- the calling agent (decision 7, N2) ----------------------------------------------
|
||||
|
||||
def _me_env(server="mam-x", pane="w5:p1"):
|
||||
return {"HERDR_SESSION": server, "HERDR_PANE_ID": pane}
|
||||
|
||||
|
||||
def test_the_calling_agent_its_row_and_its_running_job_are_excluded(box):
|
||||
me = {"name": "me-creator", "pane_id": "w5:p1", "cwd": str(box.ws)}
|
||||
box.herdr(sessions={"default": True, "mam-x": True}, agents={"mam-x": [me]})
|
||||
write_rows(box.ws, [row("me-creator", server="mam-x", cwd=str(box.ws))])
|
||||
write_job(box.mam / "jobs", "myjob", status="running", session="herdr:me-creator")
|
||||
res = box.update(env=_me_env())
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
|
||||
write_job(box.mam / "jobs", "myjob2", status="pending", session="herdr:me-creator")
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job myjob2")
|
||||
|
||||
|
||||
def test_same_pane_id_on_another_server_is_not_excluded(box):
|
||||
box.herdr(sessions={"default": True, "mam-x": True, "mam-y": True},
|
||||
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}],
|
||||
"mam-y": [{"name": "twin", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
|
||||
before = snapshot(box.ws)
|
||||
res = box.update(env=_me_env())
|
||||
assert_refused_unchanged(box, before, res, "herdr agent twin (pane w5:p1, server mam-y)")
|
||||
assert "agent me " not in res.stderr
|
||||
|
||||
|
||||
def test_a_nameless_caller_pane_is_excluded_by_pane_only(box):
|
||||
box.herdr(sessions={"default": True, "mam-x": True},
|
||||
agents={"mam-x": [{"name": None, "pane_id": "w5:p1", "cwd": str(box.ws)}]})
|
||||
assert box.update(env=_me_env()).returncode == 0
|
||||
write_job(box.mam / "jobs", "nameless", status="running", session="herdr:")
|
||||
write_job(box.mam / "jobs", "nonename", status="running", session="herdr:None")
|
||||
before = snapshot(box.ws)
|
||||
assert_refused_unchanged(box, before, box.update(env=_me_env()), "job nameless", "job nonename")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("env", [{}, {"HERDR_PANE_ID": "w5:p1"}, {"HERDR_SESSION": "mam-x"}])
|
||||
def test_nothing_is_excluded_when_the_caller_is_not_identified(box, env):
|
||||
box.herdr(sessions={"default": True, "mam-x": True},
|
||||
agents={"mam-x": [{"name": "me", "pane_id": "w5:p1", "cwd": str(box.ws)}]})
|
||||
before = snapshot(box.ws)
|
||||
res = box.update(env=env)
|
||||
assert_refused_unchanged(box, before, res, "herdr agent me", "nothing was excluded as the calling agent")
|
||||
|
||||
|
||||
# --- the monitor (decision 2, U3) ----------------------------------------------------
|
||||
|
||||
def test_a_running_monitor_is_stopped_and_started_again(box):
|
||||
box.stub_monitor()
|
||||
box.start_monitor()
|
||||
first = set(box.monitor_pids())
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "monitor: stopping the daemon" in res.stdout and "monitor: restarted" in res.stdout
|
||||
now = set(box.monitor_pids())
|
||||
assert now and not (now & first) and box.monitor_starts() == 2
|
||||
|
||||
|
||||
def test_no_monitor_before_means_none_after(box):
|
||||
box.stub_monitor()
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert not box.monitor_pids() and box.monitor_starts() == 0
|
||||
|
||||
|
||||
def test_no_subscriber_daemon_flag_wins(box):
|
||||
box.stub_monitor()
|
||||
box.start_monitor()
|
||||
res = box.update(env={"MAM_NO_SUBSCRIBER_DAEMON": "1"})
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "monitor: not restarted (MAM_NO_SUBSCRIBER_DAEMON=1)" in res.stdout
|
||||
assert not box.monitor_pids()
|
||||
|
||||
|
||||
def test_a_monitor_that_ignores_term_is_killed_restarted_and_nothing_installed(box):
|
||||
box.stub_monitor()
|
||||
box.start_monitor(env={"STUB_MONITOR_IGNORE_TERM": "1"})
|
||||
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
|
||||
before = snapshot(box.ws)
|
||||
res = box.update(env={"MAM_UPDATE_MONITOR_STOP_S": "1"})
|
||||
assert res.returncode == 1, res.stdout + res.stderr
|
||||
assert "it was killed and is started again" in res.stderr
|
||||
assert not (box.ws / "INSTALLER-RAN").exists()
|
||||
assert snapshot(box.ws) == before
|
||||
assert box.monitor_pids() and box.monitor_starts() == 2
|
||||
assert not (box.mam / "update-in-progress").exists()
|
||||
|
||||
|
||||
def test_a_monitor_lock_held_by_an_unknown_process_refuses(box):
|
||||
lock = box.mam / "monitor.lock"
|
||||
with open(lock, "w") as fh:
|
||||
fcntl.flock(fh, fcntl.LOCK_EX)
|
||||
before = snapshot(box.ws)
|
||||
res = box.update()
|
||||
assert_refused_unchanged(box, before, res, "monitor.lock is held by a monitor the skills did not start")
|
||||
|
||||
|
||||
# --- rollback (decisions 3, 10; U4) --------------------------------------------------
|
||||
|
||||
FAILING_INSTALL = """
|
||||
echo broken >> "$ws/.agents/skills/lib.sh"
|
||||
rm -f "$ws/.mam_deploy/remove.sh"
|
||||
echo new > "$ws/MESSAGING.md"
|
||||
mkdir -p "$ws/.venv/lib" && echo junk > "$ws/.venv/lib/junk"
|
||||
echo junk > "$ws/.mam/version.txt"
|
||||
echo junk > "$ws/.mam/asset_hashes.txt"
|
||||
mkdir -p "$ws/.agents/skills/half-installed" && echo x > "$ws/.agents/skills/half-installed/f"
|
||||
python3 -c 'import yaml,sys; p=sys.argv[1]; d=yaml.safe_load(open(p)); d["herdr_sessions"]=[]; yaml.safe_dump(d, open(p, "w"))' "$ws/.mam/agent-sessions.yaml"
|
||||
exit 7
|
||||
"""
|
||||
|
||||
|
||||
def test_a_failed_install_rolls_everything_back(box):
|
||||
write_rows(box.ws, [row("a", status="stopped")])
|
||||
db = box.mam / "agent-sessions.db"
|
||||
conn = sqlite3.connect(str(db))
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA wal_autocheckpoint=0")
|
||||
conn.execute("CREATE TABLE state (id INTEGER PRIMARY KEY, data TEXT)")
|
||||
conn.execute("CREATE TABLE sessions (name TEXT PRIMARY KEY, status TEXT, pane_cwd TEXT, data JSON)")
|
||||
conn.execute("INSERT INTO state VALUES (1, ?)", (json.dumps({"orchestrator_uuids": ["in-wal"]}),))
|
||||
conn.execute("INSERT INTO sessions VALUES ('a', 'stopped', '/x', ?)", (json.dumps(row("a", status="stopped")),))
|
||||
conn.commit()
|
||||
assert (box.mam / "agent-sessions.db-wal").stat().st_size > 0, "the commit is still in the WAL"
|
||||
box.stub_monitor()
|
||||
box.start_monitor()
|
||||
box.installer(FAILING_INSTALL)
|
||||
before = snapshot(box.ws)
|
||||
try:
|
||||
res = box.update()
|
||||
finally:
|
||||
conn.close()
|
||||
assert res.returncode == 1, res.stdout + res.stderr
|
||||
assert "the install failed (rc 7); rolling back" in res.stderr and "rolled back" in res.stdout
|
||||
after = snapshot(box.ws)
|
||||
after.pop(".mam/agent-sessions.db", None)
|
||||
before.pop(".mam/agent-sessions.db", None)
|
||||
before.pop(".mam/agent-sessions.db-wal", None)
|
||||
before.pop(".mam/agent-sessions.db-shm", None)
|
||||
assert after == before, sorted(set(after.items()) ^ set(before.items()))[:10]
|
||||
assert not (box.ws / "MESSAGING.md").exists() and not (box.ws / ".venv").exists()
|
||||
check = sqlite3.connect(str(db))
|
||||
try:
|
||||
st = json.loads(check.execute("SELECT data FROM state WHERE id=1").fetchone()[0])
|
||||
finally:
|
||||
check.close()
|
||||
assert st == {"orchestrator_uuids": ["in-wal"]}, "the backup carried the WAL's commit"
|
||||
assert not (box.mam / "update-backup").exists() and not (box.mam / "update-in-progress").exists()
|
||||
assert box.monitor_pids() and box.monitor_starts() == 2, "the restored tree starts the monitor again"
|
||||
|
||||
box.env["MAM_INSTALLER_URL"] = "file://" + str(REPO_ROOT / "deploy" / "install.sh")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, "the skill runs again after a rollback: " + res.stdout + res.stderr
|
||||
|
||||
|
||||
# --- leftovers (U4) ------------------------------------------------------------------
|
||||
|
||||
def _leftover(box: Box, state):
|
||||
bk = box.mam / "update-backup"
|
||||
(bk / "files" / ".agents" / "skills").mkdir(parents=True)
|
||||
shutil.copy2(box.ws / ".agents/skills/lib.sh", bk / "files/.agents/skills/lib.sh")
|
||||
(bk / "items").write_text(".agents/skills/lib.sh\nMESSAGING.md\n")
|
||||
(bk / "absent").write_text("MESSAGING.md\n")
|
||||
(bk / "registry").mkdir()
|
||||
(bk / "registry" / "agent-sessions.yaml").write_text("herdr_sessions: []\n")
|
||||
if state is not None:
|
||||
(bk / "state").write_text(state + "\n")
|
||||
return bk
|
||||
|
||||
|
||||
@pytest.mark.parametrize("state", [None, "backing-up", "installed"])
|
||||
def test_leftover_that_changed_nothing_or_finished_is_removed(box, state):
|
||||
_leftover(box, state)
|
||||
(box.ws / "MESSAGING.md").write_text("must survive: nothing is restored\n")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "removing the backup a previous update left" in res.stdout
|
||||
assert (box.ws / "MESSAGING.md").exists()
|
||||
assert not (box.mam / "update-backup").exists()
|
||||
|
||||
|
||||
def test_leftover_installing_restores_files_but_not_the_registry(box):
|
||||
_leftover(box, "installing")
|
||||
pristine = (box.ws / ".agents/skills/lib.sh").read_bytes()
|
||||
with open(box.ws / ".agents/skills/lib.sh", "a") as fh:
|
||||
fh.write("# half installed\n")
|
||||
(box.ws / "MESSAGING.md").write_text("made by the dead install\n")
|
||||
write_rows(box.ws, [row("written-after-crash", status="stopped")])
|
||||
box.installer("grep -q 'half installed' \"$ws/.agents/skills/lib.sh\" && exit 9\n"
|
||||
"test -e \"$ws/MESSAGING.md\" && exit 8\n"
|
||||
"cp \"$ws/.mam/agent-sessions.yaml\" \"$ws/../seen-registry.yaml\"\nexit 0\n")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "the previous update was rolled back" in res.stdout
|
||||
assert (box.ws / ".agents/skills/lib.sh").read_bytes() == pristine
|
||||
assert "written-after-crash" in (box.root / "seen-registry.yaml").read_text(), "registry left as it was"
|
||||
assert not (box.mam / "update-backup").exists()
|
||||
|
||||
|
||||
def test_leftover_whose_restore_fails_is_kept_and_nothing_more_changes(box):
|
||||
bk = _leftover(box, "installing")
|
||||
(bk / "items").write_text(".agents/skills/lib.sh\n.agents/skills/missing-from-backup\n")
|
||||
box.stub_monitor()
|
||||
box.start_monitor()
|
||||
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
|
||||
res = box.update()
|
||||
assert res.returncode == 1, res.stdout + res.stderr
|
||||
assert "restoring the previous update's backup failed" in res.stderr
|
||||
assert bk.exists() and (bk / "state").read_text().strip() == "installing"
|
||||
assert not (box.ws / "INSTALLER-RAN").exists()
|
||||
assert not box.monitor_pids(), "the monitor stays stopped"
|
||||
assert not (box.mam / "update-in-progress").exists()
|
||||
|
||||
|
||||
def test_the_backup_lives_in_the_workspace_and_is_gone_after_success(box):
|
||||
box.installer("test -f \"$ws/.mam/update-backup/state\" && grep -qx installing \"$ws/.mam/update-backup/state\" "
|
||||
"&& test -f \"$ws/.mam/update-backup/files/.agents/skills/lib.sh\" && touch \"$ws/../saw-backup\"\n")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert (box.root / "saw-backup").exists()
|
||||
assert not (box.mam / "update-backup").exists()
|
||||
|
||||
|
||||
# --- the marker and the admission lock (decision 8) -----------------------------------
|
||||
|
||||
def test_a_live_update_marker_refuses_a_second_update_and_is_left_alone(box):
|
||||
live_marker(box.mam / "update-in-progress", box)
|
||||
text = (box.mam / "update-in-progress").read_text()
|
||||
before = snapshot(box.ws)
|
||||
res = box.update()
|
||||
assert res.returncode == 1 and "another update is in progress" in res.stderr, res.stderr
|
||||
assert (box.mam / "update-in-progress").read_text() == text
|
||||
assert snapshot(box.ws) == before
|
||||
|
||||
|
||||
def test_a_stale_update_marker_is_taken_over(box):
|
||||
dead_marker(box.mam / "update-in-progress")
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert "stale update marker" in res.stdout
|
||||
assert not (box.mam / "update-in-progress").exists()
|
||||
|
||||
|
||||
def test_two_updates_at_once_one_proceeds(box):
|
||||
box.installer("touch \"$ws/../first-installing\"; sleep 4; exit 0\n")
|
||||
first = box.helper("bash", str(box.ws / UPDATE_REL), "--yes", cwd=str(box.ws), env=box.env,
|
||||
stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||
_until(lambda: (box.root / "first-installing").exists())
|
||||
marker = (box.mam / "update-in-progress").read_text()
|
||||
second = box.update()
|
||||
assert second.returncode == 1 and "another update is in progress" in second.stderr, second.stderr
|
||||
assert (box.mam / "update-in-progress").read_text() == marker, "the running update's marker is untouched"
|
||||
out, err = first.communicate(timeout=120)
|
||||
assert first.returncode == 0, out + err
|
||||
|
||||
|
||||
def test_entries_are_refused_while_the_copy_installs(box):
|
||||
box.installer('if bash -c \'source "$1/.agents/skills/lib.sh" && mam_update_admit\' _ "$ws" 2> "$ws/../admit.err"; '
|
||||
'then echo admitted > "$ws/../admit"; else echo refused > "$ws/../admit"; fi\n')
|
||||
res = box.update()
|
||||
assert res.returncode == 0, res.stdout + res.stderr
|
||||
assert (box.root / "admit").read_text().strip() == "refused"
|
||||
assert "an update of %s is in progress" % box.ws in (box.root / "admit.err").read_text()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("wait,needle", [("30", "session late-row is running"), ("1", "admission lock past")])
|
||||
def test_an_entry_holding_the_admission_lock_is_waited_for_or_refuses(box, wait, needle):
|
||||
script = ('mam_update_admit || exit 1\necho admitted\nsleep 3\n'
|
||||
'printf "herdr_sessions:\\n- name: late-row\\n status: running\\n" > "$WORKSPACE_ROOT/.mam/agent-sessions.yaml"\n'
|
||||
'mam_update_admit_release\nsleep 30\n')
|
||||
entry = box.helper("bash", "-c", 'source "$1/.agents/skills/lib.sh"\n' + script, "_", str(box.ws),
|
||||
env=dict(box.env, WORKSPACE_ROOT=str(box.ws)), stdout=subprocess.PIPE, text=True)
|
||||
assert entry.stdout.readline().strip() == "admitted"
|
||||
box.installer("touch \"$ws/INSTALLER-RAN\"\n")
|
||||
res = box.update(env={"MAM_UPDATE_ADMISSION_WAIT_S": wait})
|
||||
assert res.returncode == 1, res.stdout + res.stderr
|
||||
assert needle in res.stderr, res.stderr
|
||||
assert not (box.ws / "INSTALLER-RAN").exists()
|
||||
assert not (box.mam / "update-in-progress").exists()
|
||||
@@ -1,6 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Issue #8 — update.sh must keep the installer's fresh version.txt and
|
||||
asset_hashes.txt instead of restoring the pre-update copies.
|
||||
"""Issue #8 — an update must keep the installer's fresh version.txt and
|
||||
asset_hashes.txt instead of restoring the pre-update copies. Since Issue #72
|
||||
the update is the multi-agent-mux-update skill (in place, no remove.sh) and
|
||||
no .mam_deploy/update.sh is installed.
|
||||
|
||||
Restoring them left version.txt naming the previous commit and made remove.sh
|
||||
compare against stale hashes, so every updated-but-pristine file was backed up
|
||||
@@ -51,8 +53,19 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
|
||||
with open(mam / "install_state", "a") as fh:
|
||||
fh.write("custom_marker=kept\n")
|
||||
|
||||
res = subprocess.run(["bash", str(ws / ".mam_deploy" / "update.sh"), "--force"],
|
||||
cwd=str(ws), env=env, capture_output=True, text=True, timeout=600)
|
||||
assert not (ws / ".mam_deploy" / "update.sh").exists()
|
||||
upd_env = {k: v for k, v in env.items() if not k.startswith("HERDR_")
|
||||
and k not in ("WORKSPACE_ROOT", "AGENT_SESSIONS_YAML", "MAM_JOBS_DIR")}
|
||||
upd_env["MAM_NO_SUBSCRIBER_DAEMON"] = "1"
|
||||
# The update lists the live herdr agents; never ask a real server here.
|
||||
fake = ws.parent / "fake-herdr-bin"
|
||||
fake.mkdir()
|
||||
(fake / "herdr").write_text('#!/bin/sh\necho \'{"sessions": []}\'\n')
|
||||
(fake / "herdr").chmod(0o755)
|
||||
upd_env["PATH"] = "%s:%s" % (fake, upd_env["PATH"])
|
||||
res = subprocess.run(["bash", str(ws / ".agents/skills/multi-agent-mux-update/scripts/update.sh"), "--yes"],
|
||||
cwd=str(ws), env=upd_env, capture_output=True, text=True, timeout=600,
|
||||
stdin=subprocess.DEVNULL)
|
||||
assert res.returncode == 0, res.stderr + res.stdout
|
||||
|
||||
version = (mam / "version.txt").read_text()
|
||||
@@ -64,3 +77,4 @@ def test_update_keeps_fresh_version_and_hashes_and_merges_install_state(installe
|
||||
state = (mam / "install_state").read_text()
|
||||
assert "custom_marker=kept" in state, state
|
||||
assert state.count("gitignore_created=1") == 1, "merged keys must not be duplicated:\n" + state
|
||||
assert not (ws / ".mam_deploy" / "update.sh").exists()
|
||||
Reference in new issue
Block a user