GodopuandClaude Opus 5
33b91fe0c8
Two defects in the started-based verdict, both found by the full suite.
The first was destructive. No `started` within T exited 1, which fired the
rollback trap and killed the session. That is worse than the bug being fixed:
create_session already has a rule for "the agent is not responding but the
process is alive" one step earlier - a TUI that never signalled ready is
preserved for diagnosis - and an onboarding that is merely slow is the same
situation. The job is marked `error` and warned about loudly; the session, its
row and the pane stay. MULTI_AGENT_RULES §6 has the GM gate on this job
anyway, so an unacknowledged onboarding still blocks real work.
It also broke 21 tests across 8 modules (create/sanity/tier2/tier3/tier4/
issue15/issue19/issue21) and added roughly an hour to the suite: no sandbox
has an agent to publish `started`, so every create_session test sat out the
full 180 s and then failed on the rollback. MAM_ONBOARD_ACK_S shortens that
wait, and the sandbox fixture sets 3 s. It is deliberately not the wrapper's
MAM_INJECT_ACK_GRACE_S - that is a different wait whose own tests assert on
the value they pass, and reusing it made issue18's acceptance2 read "within
3s" instead of "within 4s".
The second: the delivery failure published `error` and then the rollback
published `cancelled` on the same job, which the registry refuses
("conflicting cancelled after terminal error"). Terminal events are published
exactly once, so the trap now skips a job this run already closed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
672 lines
30 KiB
Python
672 lines
30 KiB
Python
"""Issue #23 + #24 — the create_session start path.
|
|
|
|
Four regressions live here, in the order of the issues:
|
|
|
|
1. a herdr call aimed at another server must not carry the caller pane's own
|
|
workspace/pane/tab ids (#23 comments: reproduced on Linux and macOS);
|
|
2. blocking start-up modals are agent-adapter knowledge and are answered while
|
|
waiting for the TUI (#24);
|
|
3. onboarding delivery is judged by the agent's own `started` event (#23);
|
|
4. a rollback verifies the kill and leaves no `running` row or dangling job (#23).
|
|
|
|
What is mocked and what is real is stated per test. The dialog tests fix OUR
|
|
contract - which pane string gets which keys - not that the keys drive a real
|
|
agy/grok/codex/opencode TUI; no such TUI can be started here.
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import subprocess
|
|
import textwrap
|
|
|
|
import pytest
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# helpers
|
|
# --------------------------------------------------------------------------
|
|
|
|
def _run_lib(tmp_path, body, env=None):
|
|
"""Source the sandbox lib.sh and run `body`. Unlike the shim-contract
|
|
helper this keeps HERDR_* exactly as `env` sets it: these tests are about
|
|
what the caller's pane leaks in."""
|
|
lib_path = tmp_path / ".agents" / "skills" / "lib.sh"
|
|
script = 'source "%s"\n_init_herdr_isolation\n%s\n' % (lib_path, body)
|
|
run_env = dict(os.environ)
|
|
for var in ("HERDR_SESSION", "HERDR_SESSION_NAME", "HERDR_WORKSPACE_ID",
|
|
"HERDR_PANE_ID", "HERDR_TAB_ID"):
|
|
run_env.pop(var, None)
|
|
run_env.update(env or {})
|
|
return subprocess.run(["bash", "-c", script], capture_output=True,
|
|
text=True, cwd=str(tmp_path), env=run_env)
|
|
|
|
|
|
def _mock_state(mock_herdr):
|
|
with open(mock_herdr) as f:
|
|
return json.load(f)
|
|
|
|
|
|
def _write_mock_state(mock_herdr, state):
|
|
with open(mock_herdr, "w") as f:
|
|
json.dump(state, f, indent=2)
|
|
|
|
|
|
def _seed_agent(mock_herdr, tmp_path, name, workspace_id, pane_id):
|
|
state = _mock_state(mock_herdr)
|
|
state.setdefault("workspaces", [])
|
|
if not any(w.get("workspace_id") == workspace_id for w in state["workspaces"]):
|
|
state["workspaces"].append(
|
|
{"workspace_id": workspace_id, "label": workspace_id, "cwd": str(tmp_path)})
|
|
state.setdefault("agents", {})[name] = {
|
|
"name": name, "agent": "agy", "status": "running", "cwd": str(tmp_path),
|
|
"workspace_id": workspace_id, "pane_id": pane_id, "pid": 9999,
|
|
"command": "agy", "buffer": "Ready",
|
|
}
|
|
state.setdefault("panes", []).append(
|
|
{"pane_id": pane_id, "workspace_id": workspace_id, "label": name, "buffer": "Ready"})
|
|
_write_mock_state(mock_herdr, state)
|
|
|
|
|
|
def _install_env_recorder(tmp_path, monkeypatch):
|
|
"""A herdr earlier in PATH than the mock that dumps the env it was called
|
|
with, then execs the mock. The shim resolves the first non-shim herdr on
|
|
PATH, so this observes exactly what reached the real binary."""
|
|
rec_dir = tmp_path / "recorder"
|
|
rec_dir.mkdir(parents=True, exist_ok=True)
|
|
env_log = tmp_path / "herdr_env.ndjson"
|
|
real = tmp_path / "bin" / "herdr"
|
|
script = rec_dir / "herdr"
|
|
script.write_text(textwrap.dedent('''\
|
|
#!/usr/bin/env python3
|
|
import json, os, sys
|
|
with open(%r, "a") as f:
|
|
f.write(json.dumps({
|
|
"argv": sys.argv[1:],
|
|
"HERDR_WORKSPACE_ID": os.environ.get("HERDR_WORKSPACE_ID"),
|
|
"HERDR_PANE_ID": os.environ.get("HERDR_PANE_ID"),
|
|
"HERDR_TAB_ID": os.environ.get("HERDR_TAB_ID"),
|
|
"HERDR_SESSION": os.environ.get("HERDR_SESSION"),
|
|
"HERDR_SESSION_NAME": os.environ.get("HERDR_SESSION_NAME"),
|
|
}) + "\\n")
|
|
os.execv(%r, [%r] + sys.argv[1:])
|
|
''') % (str(env_log), str(real), str(real)))
|
|
script.chmod(0o755)
|
|
monkeypatch.setenv("PATH", "%s:%s" % (rec_dir, os.environ["PATH"]))
|
|
return env_log
|
|
|
|
|
|
def _recorded(env_log):
|
|
if not os.path.exists(env_log):
|
|
return []
|
|
return [json.loads(l) for l in open(env_log) if l.strip()]
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# item 1 — caller pane scope must not leak across servers
|
|
# --------------------------------------------------------------------------
|
|
|
|
CROSS = {"HERDR_SESSION": "default", "HERDR_SESSION_NAME": "mam-isolated",
|
|
"HERDR_WORKSPACE_ID": "w5", "HERDR_PANE_ID": "w5:p2",
|
|
"HERDR_TAB_ID": "w5:t1"}
|
|
|
|
|
|
def test_s1_cross_server_call_drops_the_caller_panes_scope_ids(
|
|
mam_sandbox, mock_herdr, mock_agents, monkeypatch):
|
|
"""S1 (mock herdr): the caller sits in another server's pane. has-session,
|
|
kill-session and agent prompt must find the agent, and the three ids must
|
|
not reach herdr."""
|
|
tmp_path = mam_sandbox
|
|
env_log = _install_env_recorder(tmp_path, monkeypatch)
|
|
_seed_agent(mock_herdr, tmp_path, "creator-agy-01", "w1", "w1:p5")
|
|
|
|
res = _run_lib(tmp_path, 'herdr has-session -t creator-agy-01', env=CROSS)
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
|
|
res = _run_lib(tmp_path, 'herdr agent prompt creator-agy-01 "hello"', env=CROSS)
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
|
|
res = _run_lib(tmp_path, 'herdr kill-session -t creator-agy-01', env=CROSS)
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
|
|
calls = _recorded(env_log)
|
|
assert calls, "the recorder saw no herdr call"
|
|
for call in calls:
|
|
assert call["HERDR_WORKSPACE_ID"] is None, call
|
|
assert call["HERDR_PANE_ID"] is None, call
|
|
assert call["HERDR_TAB_ID"] is None, call
|
|
|
|
# the kill really happened, i.e. this is not a silent no-op
|
|
assert "creator-agy-01" not in _mock_state(mock_herdr).get("agents", {})
|
|
|
|
|
|
def test_s1b_the_delegate_job_wrappers_bare_herdr_is_the_lib_sh_function(
|
|
mam_sandbox, mock_herdr, mock_agents, monkeypatch):
|
|
"""S1b (mock herdr): the wrapper sources lib.sh and then calls a bare
|
|
`herdr` (wrapper :918 preflight). A shell function beats PATH, so that call
|
|
is mam_herdr and is fixed by item 1 too - this is why #23's preflight
|
|
failure is cured by the same change."""
|
|
tmp_path = mam_sandbox
|
|
_install_env_recorder(tmp_path, monkeypatch)
|
|
_seed_agent(mock_herdr, tmp_path, "creator-agy-01", "w1", "w1:p5")
|
|
|
|
res = _run_lib(tmp_path, 'echo "kind=$(type -t herdr)"\nherdr has-session -t creator-agy-01',
|
|
env=CROSS)
|
|
assert "kind=function" in res.stdout, res.stdout + res.stderr
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
|
|
|
|
def test_s2_same_server_keeps_the_explicit_workspace_scope(
|
|
mam_sandbox, mock_herdr, mock_agents, monkeypatch):
|
|
"""S2 (mock herdr): on the target's own server the exported scope still
|
|
scopes - a same-named agent in another workspace is not addressed (H-22)."""
|
|
tmp_path = mam_sandbox
|
|
_seed_agent(mock_herdr, tmp_path, "creator-agy-01", "w1", "w1:p5")
|
|
same = {"HERDR_SESSION": "mam-isolated", "HERDR_SESSION_NAME": "mam-isolated",
|
|
"HERDR_WORKSPACE_ID": "w2"}
|
|
|
|
before = len(_mock_state(mock_herdr).get("calls", []))
|
|
res = _run_lib(tmp_path, 'herdr agent prompt creator-agy-01 "hello from w2"', env=same)
|
|
assert res.returncode != 0, res.stdout + res.stderr
|
|
new = _mock_state(mock_herdr).get("calls", [])[before:]
|
|
prompts = [c for c in new if len(c) >= 2 and c[0] == "agent" and c[1] == "prompt"]
|
|
assert prompts == [], new
|
|
|
|
|
|
def test_s2b_a_caller_outside_any_pane_keeps_the_explicit_scope(
|
|
mam_sandbox, mock_herdr, mock_agents):
|
|
"""S2b (mock herdr): no HERDR_SESSION means the caller is not in a pane at
|
|
all, so an exported HERDR_WORKSPACE_ID is a deliberate scope, not a leak.
|
|
This is the case H-18/H-21/H-23 fix."""
|
|
tmp_path = mam_sandbox
|
|
_seed_agent(mock_herdr, tmp_path, "creator-agy-01", "w1", "w1:p5")
|
|
|
|
res = _run_lib(tmp_path, 'herdr has-session -t creator-agy-01',
|
|
env={"HERDR_SESSION_NAME": "mam-isolated", "HERDR_WORKSPACE_ID": "w2"})
|
|
assert res.returncode == 1, "an explicit out-of-scope id must still hide the agent"
|
|
|
|
res = _run_lib(tmp_path, 'herdr has-session -t creator-agy-01',
|
|
env={"HERDR_SESSION_NAME": "mam-isolated", "HERDR_WORKSPACE_ID": "w1"})
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
|
|
|
|
def test_s3_shim_contract_regressions_still_pass(mam_sandbox):
|
|
"""S3: H-18 / H-21 / H-23 are the tests that fix today's scoping meaning.
|
|
Run them explicitly as this change's regression guard."""
|
|
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
res = subprocess.run(
|
|
[os.path.join(repo, ".venv", "bin", "python"), "-m", "pytest", "-q",
|
|
"-p", "no:cacheprovider",
|
|
"tests/test_herdr_shim_contract.py", "-k",
|
|
"h18 or h21 or h22 or h23"],
|
|
capture_output=True, text=True, cwd=repo,
|
|
env=dict(os.environ, PYTHONDONTWRITEBYTECODE="1"))
|
|
assert res.returncode == 0, res.stdout[-4000:] + res.stderr[-2000:]
|
|
|
|
|
|
def test_s11_this_branch_does_not_change_the_generated_shim(mam_sandbox):
|
|
"""S11: the item-1 fix lives in mam_herdr, never in the heredoc that
|
|
lib.sh materialises into $WORKSPACE_ROOT/.mam/shim/herdr. That file is
|
|
shared by every checkout bound to one control plane, so keeping it
|
|
byte-identical to base is what makes "the control plane is not modified"
|
|
true at runtime."""
|
|
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
base = os.environ.get("MAM_BASE_SHA", "1e78dae")
|
|
show = subprocess.run(["git", "show", "%s:.agents/skills/lib.sh" % base],
|
|
capture_output=True, text=True, cwd=repo)
|
|
if show.returncode != 0:
|
|
pytest.skip("base %s not available in this checkout" % base)
|
|
with open(os.path.join(repo, ".agents", "skills", "lib.sh")) as f:
|
|
head = f.read()
|
|
assert _heredoc_body(head) == _heredoc_body(show.stdout), (
|
|
"the shim heredoc changed: the generated .mam/shim/herdr of this branch "
|
|
"would differ from the control root's, and the two would overwrite "
|
|
"each other on every herdr call")
|
|
|
|
|
|
def _heredoc_body(text):
|
|
lines = text.splitlines(True)
|
|
start = None
|
|
for i, line in enumerate(lines):
|
|
if line.strip() == "cat <<'EOF' > \"$tmp_file\"":
|
|
start = i + 1
|
|
break
|
|
assert start is not None, "shim heredoc not found in lib.sh"
|
|
for j in range(start, len(lines)):
|
|
if lines[j].rstrip("\n") == "EOF":
|
|
return "".join(lines[start:j])
|
|
raise AssertionError("unterminated shim heredoc")
|
|
|
|
|
|
@pytest.mark.parametrize("env,expected_rc", [(CROSS, 1), ({"HERDR_SESSION_NAME": "mam-isolated"}, 1)])
|
|
def test_s12_mam_herdr_never_aborts_its_caller_under_set_e(
|
|
mam_sandbox, mock_herdr, mock_agents, env, expected_rc):
|
|
"""S12 (mock herdr): mam_herdr runs inside scripts using `set -euo
|
|
pipefail`. Neither branch of the cross-server test may abort the shell,
|
|
and the child's rc must be the function's rc."""
|
|
tmp_path = mam_sandbox
|
|
body = ('set -euo pipefail\n'
|
|
'rc=0\n'
|
|
'mam_herdr has-session -t nobody-here || rc=$?\n'
|
|
'echo "missing_rc=$rc"\n'
|
|
'_seen=0\n'
|
|
'mam_herdr has-session -t creator-agy-01 || _seen=$?\n'
|
|
'echo "present_rc=$_seen"\n'
|
|
'echo REACHED_END\n')
|
|
_seed_agent(mock_herdr, tmp_path, "creator-agy-01", "w1", "w1:p5")
|
|
lib_path = tmp_path / ".agents" / "skills" / "lib.sh"
|
|
script = 'source "%s"\n%s' % (lib_path, body)
|
|
run_env = dict(os.environ)
|
|
for var in ("HERDR_SESSION", "HERDR_SESSION_NAME", "HERDR_WORKSPACE_ID",
|
|
"HERDR_PANE_ID", "HERDR_TAB_ID"):
|
|
run_env.pop(var, None)
|
|
run_env.update(env)
|
|
res = subprocess.run(["bash", "-c", script], capture_output=True, text=True,
|
|
cwd=str(tmp_path), env=run_env)
|
|
assert "REACHED_END" in res.stdout, res.stdout + res.stderr
|
|
assert "missing_rc=%d" % expected_rc in res.stdout, res.stdout + res.stderr
|
|
assert "present_rc=0" in res.stdout, res.stdout + res.stderr
|
|
|
|
|
|
def test_s13_the_loop_watcher_applies_the_same_rule(mam_sandbox):
|
|
"""S13 (pure unit): watcher.py builds its own herdr argv and never goes
|
|
through mam_herdr, so it needs the same rule."""
|
|
import sys
|
|
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
sys.path.insert(0, os.path.join(repo, ".agents", "skills"))
|
|
from lib_py.loop import watcher
|
|
|
|
caller_pane = {"HERDR_SESSION": "default", "HERDR_WORKSPACE_ID": "w5",
|
|
"HERDR_PANE_ID": "w5:p2", "HERDR_TAB_ID": "w5:t1", "PATH": "/usr/bin"}
|
|
|
|
cross = watcher.scoped_env("mam-isolated", environ=caller_pane)
|
|
assert "HERDR_WORKSPACE_ID" not in cross and "HERDR_PANE_ID" not in cross
|
|
assert "HERDR_TAB_ID" not in cross and cross["PATH"] == "/usr/bin"
|
|
|
|
same = watcher.scoped_env("default", environ=caller_pane)
|
|
assert same["HERDR_WORKSPACE_ID"] == "w5"
|
|
|
|
outside = watcher.scoped_env("mam-isolated", environ={"HERDR_WORKSPACE_ID": "w5"})
|
|
assert outside["HERDR_WORKSPACE_ID"] == "w5"
|
|
|
|
none_target = watcher.scoped_env(None, environ=caller_pane)
|
|
assert none_target["HERDR_WORKSPACE_ID"] == "w5"
|
|
|
|
seen = {}
|
|
|
|
def _run(argv, **kw):
|
|
seen["env"] = kw.get("env")
|
|
|
|
class P:
|
|
returncode, stdout, stderr = 0, "{}", ""
|
|
return P()
|
|
|
|
watcher.wait_once("agt", session="iso-x", run=_run)
|
|
assert seen["env"] is not None and "HERDR_SESSION" not in ("",)
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# item 2 — start-up dialogs come from the adapter (#24)
|
|
#
|
|
# MOCKED, and this is the limit of it: the pane text is a fixture string and
|
|
# send-keys is a shell stub, so these fix OUR contract - which screen signature
|
|
# gets which keys, once. They do NOT show that the keys drive a real agy, grok,
|
|
# codex or opencode TUI; no such TUI can be started in this environment. That
|
|
# check is a live herdr run after the loop.
|
|
# --------------------------------------------------------------------------
|
|
|
|
def _run_dialogs(tmp_path, agent, pane_text, timeout=1, env=None, prelude=""):
|
|
lib = tmp_path / ".agents" / "skills" / "lib.sh"
|
|
pane_file = tmp_path / "pane.txt"
|
|
pane_file.write_text(pane_text)
|
|
keys_log = tmp_path / "sent_keys.txt"
|
|
if keys_log.exists():
|
|
keys_log.unlink()
|
|
script = '''
|
|
source "%s"
|
|
_pane_tail() { cat "%s"; }
|
|
_sks_herdr() { printf '%%s\\n' "$*" >> "%s"; }
|
|
%s
|
|
handle_startup_dialogs test-sess %d %s
|
|
echo "rc=$?"
|
|
''' % (lib, pane_file, keys_log, prelude, timeout, agent)
|
|
run_env = dict(os.environ)
|
|
run_env.pop("MAM_AUTO_TRUST", None)
|
|
run_env.update(env or {})
|
|
res = subprocess.run(["bash", "-c", script], capture_output=True, text=True,
|
|
cwd=str(tmp_path), env=run_env)
|
|
sent = keys_log.read_text().splitlines() if keys_log.exists() else []
|
|
return res, sent
|
|
|
|
|
|
AGENT_DIALOGS = [
|
|
("agy", "Do you trust the contents of this project?\n > Yes, I trust this folder",
|
|
["send-keys -t test-sess Enter"]),
|
|
("grok", "Do you trust the contents of this directory?",
|
|
["send-keys -t test-sess y"]),
|
|
("codex", "Do you trust the contents of this directory?\n > 1. Yes, continue",
|
|
["send-keys -t test-sess Enter"]),
|
|
("codex", "Update available\n 1. Update now\n 2. Skip\n 3. Skip until next version",
|
|
["send-keys -t test-sess 2"]),
|
|
("claude", "Do you trust the files in this folder?\n Yes, I trust this folder",
|
|
["send-keys -t test-sess Enter"]),
|
|
("claude", "Try the new fullscreen renderer?\n Yes, try it",
|
|
["send-keys -t test-sess Escape"]),
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize("agent,pane,expected", AGENT_DIALOGS)
|
|
def test_s4_each_agents_dialog_signature_gets_its_own_keys(
|
|
mam_sandbox, agent, pane, expected):
|
|
"""S4 (stubbed pane + stubbed send-keys): the adapter's signature is
|
|
matched and exactly its keys are sent."""
|
|
res, sent = _run_dialogs(mam_sandbox, agent, pane)
|
|
assert sent == expected, (res.stdout, res.stderr, sent)
|
|
|
|
|
|
def test_s4b_every_adapter_publishes_the_fact(mam_sandbox):
|
|
"""S4b (real `lib_py.agents facts`): MAM_STARTUP_DIALOGS is part of the
|
|
facts contract for every agent, so the shell never has to know an agent
|
|
name. An agent with no known blocking modal publishes an empty list."""
|
|
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
skills = os.path.join(repo, ".agents", "skills")
|
|
seen = {}
|
|
for agent in ("claude", "agy", "grok", "codex", "opencode", "hermes"):
|
|
out = subprocess.run(
|
|
[os.path.join(repo, ".venv", "bin", "python"), "-m", "lib_py.agents",
|
|
"facts", agent],
|
|
capture_output=True, text=True, cwd=repo,
|
|
env=dict(os.environ, PYTHONPATH=skills)).stdout
|
|
line = [l for l in out.splitlines() if l.startswith("MAM_STARTUP_DIALOGS=")]
|
|
assert line, "no MAM_STARTUP_DIALOGS for %s" % agent
|
|
seen[agent] = json.loads(line[0].split("=", 1)[1].strip("'"))
|
|
for agent in ("claude", "agy", "grok", "codex"):
|
|
assert seen[agent], "%s must publish its start-up dialogs" % agent
|
|
for pattern, keys, kind in seen[agent]:
|
|
assert kind in ("trust", "notice"), (agent, kind)
|
|
assert keys, (agent, pattern)
|
|
assert seen["opencode"] == [] and seen["hermes"] == []
|
|
|
|
|
|
def test_s5_an_accepted_dialog_still_on_screen_is_not_resent(mam_sandbox):
|
|
"""S5 (stubbed pane + stubbed send-keys): #24's comment reports that
|
|
`--source visible` keeps showing the dialog for a while after it was
|
|
accepted, and _pane_capture hardcodes that source. The pane here never
|
|
changes, so a frame-count rule would send the keys again every poll; the
|
|
time cooldown must keep it at one."""
|
|
res, sent = _run_dialogs(
|
|
mam_sandbox, "grok", "Do you trust the contents of this directory?",
|
|
timeout=3, env={"MAM_DIALOG_RESEND_S": "60"})
|
|
assert sent == ["send-keys -t test-sess y"], (res.stdout, res.stderr, sent)
|
|
|
|
|
|
def test_s5b_the_cooldown_expiring_allows_one_more_try(mam_sandbox):
|
|
"""S5b: the cooldown is a rate limit, not a one-shot - a dialog that really
|
|
did not take the keys is retried once the window passes."""
|
|
res, sent = _run_dialogs(
|
|
mam_sandbox, "grok", "Do you trust the contents of this directory?",
|
|
timeout=4, env={"MAM_DIALOG_RESEND_S": "1"})
|
|
assert len(sent) >= 2, (res.stdout, res.stderr, sent)
|
|
assert set(sent) == {"send-keys -t test-sess y"}, sent
|
|
|
|
|
|
def test_s5c_readiness_is_judged_by_the_ready_tokens_not_by_the_dialog_leaving(
|
|
mam_sandbox):
|
|
"""S5c: a ready screen returns 0 without sending anything."""
|
|
res, sent = _run_dialogs(mam_sandbox, "grok", "Grok ready\n>>> ", timeout=3)
|
|
assert sent == [], sent
|
|
assert "rc=0" in res.stdout, res.stdout + res.stderr
|
|
|
|
|
|
def test_s6_no_auto_trust_leaves_trust_dialogs_alone_but_answers_notices(
|
|
mam_sandbox):
|
|
"""S6 (stubbed): --no-auto-trust. codex trust is left to a human with a
|
|
warning; the update notice still gets answered, since skipping it changes
|
|
nothing and it blocks the session just as hard."""
|
|
res, sent = _run_dialogs(
|
|
mam_sandbox, "codex", "Do you trust the contents of this directory?\n > 1. Yes, continue",
|
|
timeout=1, env={"MAM_AUTO_TRUST": "0"})
|
|
assert sent == [], sent
|
|
assert "no-auto-trust" in res.stderr, res.stderr
|
|
|
|
res, sent = _run_dialogs(
|
|
mam_sandbox, "codex", "Update available\n 1. Update now\n 3. Skip until next version",
|
|
timeout=1, env={"MAM_AUTO_TRUST": "0"})
|
|
assert sent == ["send-keys -t test-sess 2"], (res.stderr, sent)
|
|
|
|
|
|
def test_s6b_create_session_no_longer_gates_dialogs_on_claude(mam_sandbox):
|
|
"""S6b (source read): the #24 root cause was the `AGENT = claude` gate at
|
|
the create_session call site, so the gate's absence is the regression."""
|
|
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
src = open(os.path.join(repo, ".agents", "skills", "multi-agent-mux-create",
|
|
"scripts", "create_session.sh")).read()
|
|
call = [l for l in src.splitlines() if "handle_startup_dialogs" in l]
|
|
assert call, "create_session no longer answers start-up dialogs at all"
|
|
assert all('"$AGENT"' in l for l in call), call
|
|
assert 'if [ "$AGENT" = "claude" ] && [ -z "$LAST_VISIBLE_STATUS_OVERRIDE" ]' not in src
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# item 3 — onboarding delivery is judged by the agent's own `started` (#23)
|
|
# --------------------------------------------------------------------------
|
|
|
|
def _create_sh(tmp_path):
|
|
return tmp_path / ".agents" / "skills" / "multi-agent-mux-create" / "scripts" / "create_session.sh"
|
|
|
|
|
|
def _extract(path, start_marker, end_marker, close=None):
|
|
"""Lift a block out of create_session.sh so the test runs the shipped code
|
|
rather than a paraphrase of it. A moved marker fails loudly here.
|
|
|
|
`close` closes the construct the end marker opens: the block runs to the
|
|
first line equal to it after the end marker (e.g. the `fi` of an `if`)."""
|
|
lines = open(path).read().splitlines(True)
|
|
start = next((i for i, l in enumerate(lines) if start_marker in l), None)
|
|
assert start is not None, "start marker %r is gone from %s" % (start_marker, path)
|
|
end = next((i for i, l in enumerate(lines[start:], start) if end_marker in l), None)
|
|
assert end is not None, "end marker %r is gone from %s" % (end_marker, path)
|
|
if close is not None:
|
|
end = next((i for i, l in enumerate(lines[end:], end) if l.strip() == close), None)
|
|
assert end is not None, "block opened at %r is never closed with %r" % (end_marker, close)
|
|
return "".join(lines[start:end + 1])
|
|
|
|
|
|
def _seed_job(tmp_path, job_id, status="pending", started_token=None):
|
|
jobs = tmp_path / ".mam" / "jobs"
|
|
jobs.mkdir(parents=True, exist_ok=True)
|
|
(jobs / ("%s.json" % job_id)).write_text(json.dumps(
|
|
{"job_id": job_id, "status": status, "dispatch_token": "tok-1",
|
|
"attempt_id": 1, "ack_timeout_sec": 2}))
|
|
ev = jobs / ("%s.events.log" % job_id)
|
|
if started_token is None:
|
|
if ev.exists():
|
|
ev.unlink()
|
|
else:
|
|
ev.write_text(json.dumps(
|
|
{"job": job_id, "event": "started", "dispatch_token": started_token}) + "\n")
|
|
return jobs
|
|
|
|
|
|
def _run_delivery(tmp_path, inject_rc, job_id="job1", **seed):
|
|
"""Run create_session.sh's own delivery block with a stubbed injection."""
|
|
lib = tmp_path / ".agents" / "skills" / "lib.sh"
|
|
block = _extract(_create_sh(tmp_path), "# Inject instructions into the herdr pane",
|
|
"delegate_wait_started", close="fi")
|
|
_seed_job(tmp_path, job_id, **seed)
|
|
pub_log = tmp_path / "published.txt"
|
|
script = '''
|
|
set -euo pipefail
|
|
source "%s"
|
|
SESSION_NAME=test-sess
|
|
DELEGATE_JOB_ID=%s
|
|
DELEGATE_JOB_TOKEN=tok-1
|
|
DELEGATE_JOB_ATTEMPT=1
|
|
DELEGATE_JOB_ACK_S=2
|
|
instructions="hello"
|
|
inject_instructions() { return %d; }
|
|
delegate_publish_event() { printf '%%s\\n' "$2" >> "%s"; }
|
|
%s
|
|
echo DELIVERED
|
|
''' % (lib, job_id, inject_rc, pub_log, block)
|
|
env = dict(os.environ, MAM_JOBS_DIR=str(tmp_path / ".mam" / "jobs"))
|
|
res = subprocess.run(["bash", "-c", script], capture_output=True, text=True,
|
|
cwd=str(tmp_path), env=env)
|
|
published = pub_log.read_text().split() if pub_log.exists() else []
|
|
return res, published
|
|
|
|
|
|
def test_s7_create_session_no_longer_publishes_started_for_the_agent(mam_sandbox):
|
|
"""S7 (source): create_session used to publish `started` itself, so the
|
|
registry showed a started job even when the agent never saw the prompt.
|
|
With that line present no started-based verdict can mean anything."""
|
|
src = open(_create_sh(mam_sandbox)).read()
|
|
assert 'delegate_publish_event "$DELEGATE_JOB_ID" started' not in src
|
|
assert "delegate_wait_started" in src
|
|
|
|
|
|
def test_s7b_delivery_succeeds_on_the_agents_own_started(mam_sandbox):
|
|
"""S7b (real registry files, stubbed injection): a started line carrying
|
|
this dispatch token is the evidence."""
|
|
res, published = _run_delivery(mam_sandbox, 0, started_token="tok-1")
|
|
assert "DELIVERED" in res.stdout, res.stdout + res.stderr
|
|
assert published == [], published
|
|
|
|
|
|
def test_s7c_delivery_succeeds_when_the_registry_left_pending(mam_sandbox):
|
|
"""S7c: publish_event syncs the record on this host, so a status past
|
|
`pending` is the same evidence."""
|
|
res, _ = _run_delivery(mam_sandbox, 0, status="running")
|
|
assert "DELIVERED" in res.stdout, res.stdout + res.stderr
|
|
|
|
|
|
def test_s7d_no_started_marks_the_job_but_keeps_the_session(mam_sandbox):
|
|
"""S7d: the job failed, the session did not. The handshake is marked
|
|
`error` and warned about loudly, but a live agent is not killed over a
|
|
slow onboarding - the same rule this script already applies one step
|
|
earlier to a TUI that never signalled ready."""
|
|
res, published = _run_delivery(mam_sandbox, 0)
|
|
assert res.returncode == 0, res.stdout + res.stderr
|
|
assert published == ["error"], published
|
|
assert "did not acknowledge the onboarding prompt" in res.stderr
|
|
assert "preserved" in res.stderr
|
|
|
|
|
|
def test_s7e_a_started_from_another_dispatch_is_not_evidence(mam_sandbox):
|
|
"""S7e: a loop job keeps one events.log across attempts, so only a started
|
|
carrying THIS dispatch token counts."""
|
|
res, published = _run_delivery(mam_sandbox, 0, started_token="tok-OTHER")
|
|
assert published == ["error"], published
|
|
|
|
|
|
def test_s8_rc4_is_a_warning_and_the_started_wait_decides(mam_sandbox):
|
|
"""S8 (stubbed injection): rc 4 is "Enter not seen accepted", which is
|
|
exactly where a still-starting TUI lands - #23's false failure. It must no
|
|
longer abort; the started wait decides both ways."""
|
|
res, published = _run_delivery(mam_sandbox, 4, started_token="tok-1")
|
|
assert "DELIVERED" in res.stdout, res.stdout + res.stderr
|
|
assert "rc=4" in res.stderr, res.stderr
|
|
assert published == [], published
|
|
|
|
res, published = _run_delivery(mam_sandbox, 4)
|
|
assert published == ["error"], published
|
|
assert "did not acknowledge the onboarding prompt" in res.stderr
|
|
|
|
|
|
@pytest.mark.parametrize("rc", [1, 2, 3])
|
|
def test_s8b_the_other_injection_failures_still_abort_at_once(mam_sandbox, rc):
|
|
"""S8b: rc 1/2/3 say the pane never quiesced, a dialog was blocking, or the
|
|
text never appeared. No later `started` repairs those, so they stay hard
|
|
failures - only rc 4 was relaxed."""
|
|
res, published = _run_delivery(mam_sandbox, rc, started_token="tok-1")
|
|
assert res.returncode == 1, res.stdout + res.stderr
|
|
assert "DELIVERED" not in res.stdout
|
|
assert published == ["error"], published
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# item 4 — a rollback verifies the kill (#23)
|
|
# --------------------------------------------------------------------------
|
|
|
|
def _run_rollback(tmp_path, kill_rc, still_alive, row=True, job_id="job1"):
|
|
lib = tmp_path / ".agents" / "skills" / "lib.sh"
|
|
yaml_path = tmp_path / ".mam" / "agent-sessions.yaml"
|
|
if row:
|
|
yaml_path.write_text(
|
|
"herdr_sessions:\n"
|
|
"- name: test-sess\n"
|
|
" status: running\n"
|
|
" role: creator\n"
|
|
" delegate_job_id: %s\n"
|
|
" pane:\n"
|
|
" index: 0\n"
|
|
" pid: 111\n"
|
|
" cmd: claude\n"
|
|
" cwd: %s\n" % (job_id, tmp_path))
|
|
else:
|
|
yaml_path.write_text("herdr_sessions: []\n")
|
|
_seed_job(tmp_path, job_id)
|
|
block = _extract(_create_sh(tmp_path), "# _mark_row_rolled_back <status> <note>",
|
|
"trap cleanup_herdr_on_error EXIT")
|
|
pub_log = tmp_path / "published.txt"
|
|
# The trap is installed and fired the way create_session fires it: an
|
|
# errexit shell exiting non-zero. rc 9 through means "no exit 43".
|
|
script = '''
|
|
set -euo pipefail
|
|
source "%s"
|
|
SESSION_NAME=test-sess
|
|
DELEGATE_JOB_ID=%s
|
|
DELEGATE_JOB_TOKEN=tok-1
|
|
DELEGATE_JOB_ATTEMPT=1
|
|
ROW_APPENDED=%d
|
|
AGENT_SESSIONS_YAML="%s"
|
|
delegate_publish_event() { printf '%%s\\n' "$2" >> "%s"; }
|
|
_herdr() {
|
|
case "${1:-}" in
|
|
kill-session) return %d ;;
|
|
has-session) return %d ;;
|
|
esac
|
|
return 0
|
|
}
|
|
%s
|
|
exit 9
|
|
''' % (lib, job_id, 1 if row else 0, yaml_path, pub_log,
|
|
kill_rc, 0 if still_alive else 1, block)
|
|
env = dict(os.environ, MAM_JOBS_DIR=str(tmp_path / ".mam" / "jobs"),
|
|
AGENT_SESSIONS_YAML=str(yaml_path))
|
|
res = subprocess.run(["bash", "-c", script], capture_output=True, text=True,
|
|
cwd=str(tmp_path), env=env)
|
|
published = pub_log.read_text().split() if pub_log.exists() else []
|
|
return res, published, yaml_path.read_text()
|
|
|
|
|
|
def test_s9_a_kill_that_left_the_session_alive_is_reported(mam_sandbox):
|
|
"""S9 (stubbed herdr, real YAML + registry): the kill failed and the
|
|
session is still up. #23's rollback swallowed exactly this and printed
|
|
"rolled back" anyway, leaving a `running` row against a failed job."""
|
|
res, published, yaml_text = _run_rollback(mam_sandbox, kill_rc=1, still_alive=True)
|
|
assert res.returncode == 43, res.stdout + res.stderr
|
|
assert "test-sess" in res.stderr and "kill-session" in res.stderr
|
|
assert "rollback-failed" in yaml_text, yaml_text
|
|
assert "status: running" in yaml_text, "a session that is still up must not be called gone"
|
|
assert "delegate_job_id: null" in yaml_text, yaml_text
|
|
assert published == ["cancelled"], published
|
|
|
|
|
|
def test_s10_a_successful_kill_keeps_the_previous_behaviour(mam_sandbox):
|
|
"""S10: nothing is still running, so no exit 43 and no warning - but the
|
|
row and the job are still cleaned up, which is the other half of #23."""
|
|
res, published, yaml_text = _run_rollback(mam_sandbox, kill_rc=0, still_alive=False)
|
|
assert res.returncode == 9, res.stdout + res.stderr
|
|
assert "rolled-back" in yaml_text, yaml_text + "\nSTDERR:" + res.stderr
|
|
assert "status: terminated" in yaml_text, yaml_text
|
|
assert "delegate_job_id: null" in yaml_text, yaml_text
|
|
assert published == ["cancelled"], published
|
|
|
|
|
|
def test_s10b_a_rollback_before_the_row_existed_touches_no_yaml(mam_sandbox):
|
|
"""S10b: failing before the append must not invent a row."""
|
|
res, published, yaml_text = _run_rollback(mam_sandbox, kill_rc=0, still_alive=False,
|
|
row=False)
|
|
assert res.returncode == 9, res.stdout + res.stderr
|
|
assert "herdr_sessions: []" in yaml_text, yaml_text
|