fix(issue-9): translate only a structured server_not_running error in the shim's ls

The ls translation searched stderr and the whole stdout for the substring
server_not_running, so a successful `agent list` that merely carried the text
(an agent named server_not_running-codex, or metadata) made the shim report
"no server running". reconcile.sh reads that as a confirmed-empty server, and
drift A would then mark every live row on it terminated (codex review of
0918917).

- keep the RPC exit code and translate only a failed call whose error
  envelope (the whole stream or any line of it) has
  error.code == "server_not_running"; every other failure keeps the unknown
  path (nonzero exit, herdr_confirmed False) as before
- tests: the real herdr 0.9 envelope, a successful list carrying the
  substring in a name and in metadata, other failures staying unknown, and
  reconcile keeping both live rows with no drift A; the new cases fail on
  0918917
- delegate wrapper: log a non-pending status seen during the inject ack
  grace as "registry advanced" instead of attributing it to the agent
- VERSIONS.md v4.4.2: describe the structured check and count 16 new tests

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGpJXhSZasJLU38ksDghe

.agents/skills/lib.sh

@@ -1245,15 +1245,42 @@ except Exception:
     # running", tmux's phrasing) instead of an opaque parse failure, which
     # would flip herdr_confirmed=False and freeze drift detection for every
     # server on the next heartbeat.
+    # Only a FAILED call whose structured error envelope carries
+    # error.code == "server_not_running" means "down". A successful list can
+    # legitimately contain that text (an agent named server_not_running-codex),
+    # and reading it as "down" made reconcile terminate every live row on that
+    # server. Any other failure falls through to the unknown path below.
     _ls_err=$(mktemp 2>/dev/null || echo "/tmp/mam_ls_err_$$")
-    _ls_raw="$(_real_herdr agent list 2>"$_ls_err")" || true
+    _ls_rc=0
-    if grep -q server_not_running "$_ls_err" 2>/dev/null \
+    _ls_raw="$(_real_herdr agent list 2>"$_ls_err")" || _ls_rc=$?
-       || printf '%s' "$_ls_raw" | grep -q server_not_running; then
+    _ls_down=0
-      rm -f "$_ls_err" 2>/dev/null || true
+    if [ "$_ls_rc" -ne 0 ]; then
+      _ls_down=$(printf '%s' "$_ls_raw" | MAM_LS_ERR_FILE="$_ls_err" python3 -c '
+import json, os, sys
+out = sys.stdin.read()
+try:
+    with open(os.environ["MAM_LS_ERR_FILE"]) as f:
+        err_text = f.read()
+except Exception:
+    err_text = ""
+down = 0
+for text in [out, err_text] + out.splitlines() + err_text.splitlines():
+    try:
+        doc = json.loads(text)
+    except Exception:
+        continue
+    err = doc.get("error") if isinstance(doc, dict) else None
+    if isinstance(err, dict) and err.get("code") == "server_not_running":
+        down = 1
+        break
+print(down)
+' 2>/dev/null || echo 0)
+    fi
+    rm -f "$_ls_err" 2>/dev/null || true
+    if [ "$_ls_down" = "1" ]; then
       echo "no server running (herdr session ${_sess:-default})" >&2
       exit 1
     fi
-    rm -f "$_ls_err" 2>/dev/null || true
     printf '%s\n' "$_ls_raw" | \
       MAM_LS_FORMAT="$format" MAM_REAL_HERDR="$REAL_HERDR" MAM_LS_SESSION="$_sess" \
       TZ=UTC LC_ALL=C python3 -c '

.agents/skills/multi-agent-mux-delegate-job/multi-agent-mux-delegate-job

@@ -569,15 +569,18 @@ _disk_status() {
 # reports `working`, yet publishes `started` within seconds. Closing such a job
 # as agent_blocked strands a live agent whose later `completed` the CAS table
 # then refuses. Before finalising, give the registry MAM_INJECT_ACK_GRACE_S
-# (default 10 s) to show the agent's own acknowledgement: any status other than
+# (default 10 s) to advance past pending. Any other status means the job moved
-# pending means the prompt was delivered.
+# on - normally the agent's own `started` - so the prompt is treated as
+# delivered. The status is not pinned to this dispatch and may come from an
+# external writer; the terminal handling after this still keeps such an
+# error/cancelled as it is rather than turning it into success.
 _inject_acknowledged() {
   local job_id="$1" deadline st
   deadline=$(( $(date +%s) + ${MAM_INJECT_ACK_GRACE_S:-10} ))
   while :; do
     st="$(_disk_status "$job_id")"
     if [ -n "$st" ] && [ "$st" != "pending" ]; then
-      echo "inject: herdr could not confirm delivery, but job $job_id is already '$st' — the agent acknowledged it; continuing." >&2
+      echo "inject: herdr could not confirm delivery, but job $job_id already advanced to '$st' in the registry; treating the prompt as delivered and continuing." >&2
       return 0
     fi
     [ "$(date +%s)" -ge "$deadline" ] && return 1

VERSIONS.md

@@ -44,14 +44,14 @@
 #### 🌟 주요 개선 사항
 1. **herdr 서버 기동 env 마커 denylist (Issue #9, `lib.sh` 쉼)**: 쉼이 `herdr --session <name> server` 를 부트스트랩할 때 세션 식별 마커만 제거한 환경을 넘긴다 — `CLAUDECODE`, `CLAUDE_CODE_CHILD_SESSION`, `CLAUDE_CODE_SESSION_ID`, `CLAUDE_CODE_SESSION_ATTENDED`, `CLAUDE_CODE_ENTRYPOINT`, `CLAUDE_PID`, `CLAUDE_CODE_MESSAGING_TOKEN`, `CLAUDE_CODE_MESSAGING_SOCKET`, `CLAUDE_CODE_BRIDGE_SESSION_ID`, `ANTIGRAVITY_CONVERSATION_ID`, `HERMES_SESSION_ID`, `GROK_SESSION_ID`, `OPENCODE_SESSION_ID`. `CLAUDE_PROJECT_DIR`·`CLAUDE_CONFIG_DIR` 등 사용자 설정은 그대로 상속된다. 상속된 `CLAUDE_CODE_CHILD_SESSION` 때문에 서브에이전트 claude 의 transcript 저장이 꺼져 resume 이 불가능하던 증상을 해소.
 2. **조회 경로 자동 기동 금지 `MAM_NO_AUTOSTART=1` (Issue #9)**: 이 값이 설정되면 쉼은 서버를 띄우지 않고 호출만 전달한다. `status.sh`·`reconcile.sh` 가 이를 export 하므로 read-only `multi-agent-mux-status` 조회가 더 이상 서버 데몬을 기동하지 않는다.
-3. **쉼 `ls` 의 "no server running" 번역 (Issue #9)**: 서버가 내려가 있으면 쉼의 `ls` 가 `server_not_running` 을 `no server running (herdr session <name>)` 으로 보고한다. `reconcile.sh` 는 이를 빈 서버로 판정해 `herdr_confirmed` 를 유지한다(파싱 실패로 모든 서버의 drift 감지가 멈추는 회귀 방지).
+3. **쉼 `ls` 의 "no server running" 번역 (Issue #9)**: 서버가 내려가 있으면 쉼의 `ls` 가 `no server running (herdr session <name>)` 으로 보고하고, `reconcile.sh` 는 이를 빈 서버로 판정해 `herdr_confirmed` 를 유지한다(파싱 실패로 모든 서버의 drift 감지가 멈추는 회귀 방지). 번역은 `agent list` 가 **실패**하고 그 오류 봉투의 `error.code` 가 정확히 `server_not_running` 일 때만 적용한다. 성공한 목록에 같은 문자열이 들어 있거나(예: 에이전트 이름 `server_not_running-codex`) 다른 실패인 경우는 번역하지 않으므로, 살아 있는 서버가 빈 서버로 오판되어 drift A 로 전 세션이 종료 처리되지 않고 그 밖의 실패는 종전대로 unknown(`herdr_confirmed=False`)으로 남는다.
 4. **accepted 제출의 전달 확인 `SKS_DELIVERY_GRACE_S` (Issue #14, `send_keys_safe`)**: `agent prompt --wait` 결과가 `timeout`/`agent_prompt_stalled` 이면 herdr 가 입력을 이미 수락한 것으로 보고, `SKS_DELIVERY_GRACE_S`(기본 15초) 동안 `agent_status` working, 제출 전 화면에 없던 busy 토큰(`esc to interrupt`), 제출 전 스냅샷 대비 pane 변화 중 하나를 전달 확인 신호로 인정한다. 신호가 없으면 종전대로 rc 4 로 fail-closed 하고, 그 밖의 오류(`transport_error` 등)는 accepted 로 취급하지 않는다. 단일 주입 계약(재전송 없음)은 유지된다.
 5. **래퍼 레지스트리 ack 유예 `MAM_INJECT_ACK_GRACE_S` (Issue #14, `multi-agent-mux-delegate-job`)**: `send_keys_safe` 가 전달을 확인하지 못해도 잡을 `agent_blocked` 로 닫기 전에 `MAM_INJECT_ACK_GRACE_S`(기본 10초) 동안 레지스트리를 관측하고, `pending` 이 아닌 상태(에이전트의 `started` 등)가 보이면 전달된 것으로 보고 진행한다(`_inject_acknowledged`).
 6. **R1 유예의 pane busy 토큰 인정 (Issue #14, `_agent_still_working`)**: 구독자 idle 타임아웃 후 R1 유예가 herdr `working` 외에 pane 최근 8줄의 `esc to interrupt` 토큰도 작업 중 근거로 받아들인다. idle 타임아웃보다 긴 codex 리뷰가 턴 도중 `deadline_timeout` 으로 닫히고 이후 `completed` 가 거부되던 문제를 방지.
 
 #### ✅ 품질 검증
 - **T1-7 트리거 변경** (`tests/test_issue6_fail_closed.py`): 주입 실패 유발값을 `prompt_wait_result=timeout` → `transport_error` 로 변경(Issue #14 이후 `timeout` 은 accepted 제출이므로 더 이상 확정 실패가 아님)하고 `MAM_INJECT_ACK_GRACE_S=1` 로 대기를 단축.
-- **신규 테스트 13개**: `tests/test_issue9_server_env.py` 5개(마커 제거, `MAM_NO_AUTOSTART` 부트스트랩 생략·호출 전달, status 조회 무기동, 정적 opt-out, `ls` 번역), `tests/test_issue14_delivery_ack.py` 8개(busy 토큰·pane 변화 확인, 무신호 fail-closed, 선행 busy 토큰 배제, `transport_error` 비수용, mock herdr timeout 확인, 래퍼 레지스트리 ack, idle 타임아웃 pane busy 유예).
+- **신규 테스트 16개**: `tests/test_issue9_server_env.py` 8개(마커 제거, `MAM_NO_AUTOSTART` 부트스트랩 생략·호출 전달, status 조회 무기동, 정적 opt-out, `ls` 번역(최소·herdr 0.9 실제 오류 봉투), 성공 목록 속 `server_not_running` 문자열 비오판, 그 밖의 실패 unknown 유지, reconcile drift A 비발생), `tests/test_issue14_delivery_ack.py` 8개(busy 토큰·pane 변화 확인, 무신호 fail-closed, 선행 busy 토큰 배제, `transport_error` 비수용, mock herdr timeout 확인, 래퍼 레지스트리 ack, idle 타임아웃 pane busy 유예).
 - **테스트 기대값 정정**: `tests/test_a4_adapter_contract.py` 가 `4d444ce` 의 agy `exit_key=/exit` 변경을 반영하도록 수정(그동안 전체 스위트에서 1건 실패하던 문제 해소).
 - **실측 (커밋 `497633f` 기록)**: 새로 부트스트랩한 서버에 마커가 없고 claude pane 의 transcript 저장이 복구됨, codex 잡 `d04ebcb5` 의 `started`/`completed` 가 `agent_blocked` 없이 수락됨.
 

tests/test_issue14_delivery_ack.py

@@ -229,7 +229,7 @@ def test_wrapper_honours_registry_ack_over_unconfirmed_inject(tmp_path, mock_her
     assert ack.returncode == 0, ack.stderr
 
     stdout, stderr = proc.communicate(timeout=90)
-    assert "acknowledged it; continuing" in stderr, f"ack guard did not fire:\n{stderr}\n{stdout}"
+    assert "treating the prompt as delivered and continuing" in stderr, f"ack guard did not fire:\n{stderr}\n{stdout}"
     assert "프롬프트 주입 실패" not in stderr
     assert proc.returncode == 3, f"expected the broker_unavailable path (rc=3), got {proc.returncode}\n{stderr}"
 

tests/test_issue9_server_env.py

@@ -15,6 +15,8 @@ import subprocess
 import sys
 from pathlib import Path
 
+import pytest
+
 REPO_ROOT = Path(__file__).resolve().parent.parent
 RECONCILE_SH = REPO_ROOT / ".agents" / "skills" / "multi-agent-mux-monitor" / "scripts" / "reconcile.sh"
 STATUS_SH = REPO_ROOT / ".agents" / "skills" / "multi-agent-mux-status" / "scripts" / "status.sh"
@@ -54,6 +56,24 @@ KEPT_CONFIG = {
     "MAM_CUSTOM_SETTING": "kept",
 }
 
+# What herdr 0.9 writes to stderr (exit 1) for `agent list` against a stopped
+# session server.
+REAL_DOWN_ENVELOPE = {
+    "id": "cli:agent:list",
+    "error": {
+        "code": "server_not_running",
+        "message": "no herdr server is running at /tmp/sessions/down-srv/herdr.sock; "
+                   "run `herdr session attach down-srv` to start or attach it",
+    },
+}
+
+# A successful list in which the down-server code appears only as data: in an
+# agent name (the sanitizer allows underscores) and in unrelated metadata.
+LISTING_WITH_SUBSTRING = {"result": {"agents": [
+    {"name": "server_not_running-codex", "agent": "codex"},
+    {"name": "creator-claude-01", "agent": "claude", "title": "server_not_running"},
+]}}
+
 
 def _install_fake_herdr(sandbox: Path) -> dict:
     fakebin = sandbox / "fakebin"
@@ -72,6 +92,12 @@ def _install_fake_herdr(sandbox: Path) -> dict:
     return env
 
 
+def _fake_agent_list(sandbox: Path, body: str) -> None:
+    """Replace the fake's fallback answer (what `agent list` gets) with `body`."""
+    fake = sandbox / "fakebin" / "herdr"
+    fake.write_text(FAKE_HERDR.replace('print(json.dumps({"result": {"agents": []}}))', body))
+
+
 def _calls(env: dict) -> list:
     log = Path(env["FAKE_HERDR_LOG"])
     if not log.exists():
@@ -90,6 +116,15 @@ def _run_through_shim(sandbox: Path, env: dict, session: str) -> subprocess.Comp
     )
 
 
+def _shim_ls(sandbox: Path, env: dict, server: str) -> subprocess.CompletedProcess:
+    """The exact `ls` query reconcile.sh sends for each registered server."""
+    lib = sandbox / ".agents" / "skills" / "lib.sh"
+    return subprocess.run(
+        ["bash", "-c", f'source "{lib}" >/dev/null 2>&1; herdr -L {server} ls -F "#{{session_name}}|#{{session_created}}"'],
+        capture_output=True, text=True, cwd=str(sandbox), env=env, timeout=60,
+    )
+
+
 def test_bootstrap_strips_agent_runtime_markers(mam_sandbox):
     env = _install_fake_herdr(mam_sandbox)
     env.update(RUNTIME_MARKERS)
@@ -160,22 +195,85 @@ def test_reconcile_and_status_opt_out_of_autostart_statically():
             f"{script.name} must opt out of the shim's server bootstrap"
 
 
-def test_ls_reports_down_server_as_no_server_running(mam_sandbox):
+@pytest.mark.parametrize("envelope", [
+    {"error": {"code": "server_not_running"}},
+    REAL_DOWN_ENVELOPE,
+], ids=["minimal", "herdr-0.9"])
+def test_ls_reports_down_server_as_no_server_running(mam_sandbox, envelope):
     """reconcile.sh treats a nonzero `ls` as 'empty' only when stderr says
     'no server running'; anything else flips herdr_confirmed=False for every
     server. The shim must therefore translate server_not_running explicitly."""
     env = _install_fake_herdr(mam_sandbox)
     env["MAM_NO_AUTOSTART"] = "1"
-    fake = mam_sandbox / "fakebin" / "herdr"
+    _fake_agent_list(mam_sandbox,
-    fake.write_text(FAKE_HERDR.replace(
+                     f'sys.stderr.write({json.dumps(json.dumps(envelope))} + "\\n"); sys.exit(1)')
-        'print(json.dumps({"result": {"agents": []}}))',
+
-        'sys.stderr.write(json.dumps({"error": {"code": "server_not_running"}}) + "\\n"); sys.exit(1)',
+    res = _shim_ls(mam_sandbox, env, "down-srv")
-    ))
-    lib = mam_sandbox / ".agents" / "skills" / "lib.sh"
-    res = subprocess.run(
-        ["bash", "-c", f'source "{lib}" >/dev/null 2>&1; herdr -L down-srv ls -F "#{{session_name}}|#{{session_created}}"'],
-        capture_output=True, text=True, cwd=str(mam_sandbox), env=env, timeout=60,
-    )
     assert res.returncode != 0
     assert "no server running" in res.stderr, res.stderr
     assert res.stdout.strip() == ""
+
+
+def test_ls_lists_agents_whose_names_contain_server_not_running(mam_sandbox):
+    """A successful list is never a down server, whatever text it carries.
+    Matching the bare substring let one agent named server_not_running-codex
+    hide every agent its server returned."""
+    env = _install_fake_herdr(mam_sandbox)
+    env["MAM_NO_AUTOSTART"] = "1"
+    _fake_agent_list(mam_sandbox, f"print({json.dumps(json.dumps(LISTING_WITH_SUBSTRING))})")
+
+    res = _shim_ls(mam_sandbox, env, "live-srv")
+    assert res.returncode == 0, res.stderr
+    assert "no server running" not in res.stderr, res.stderr
+    names = [line.split("|", 1)[0] for line in res.stdout.splitlines() if line.strip()]
+    assert names == ["server_not_running-codex", "creator-claude-01"], res.stdout
+
+
+@pytest.mark.parametrize("stderr_text", [
+    json.dumps({"error": {"code": "transport_error",
+                          "message": "socket reset while probing server_not_running"}}),
+    "warning: server_not_running cache entry is stale",
+    "",
+], ids=["other-code", "plain-text", "empty"])
+def test_ls_leaves_other_failures_unknown(mam_sandbox, stderr_text):
+    """Only the structured code means "down". Any other failure keeps the
+    unknown path (nonzero exit without 'no server running'), so reconcile
+    drops herdr_confirmed instead of reading a confirmed-empty server."""
+    env = _install_fake_herdr(mam_sandbox)
+    env["MAM_NO_AUTOSTART"] = "1"
+    _fake_agent_list(mam_sandbox,
+                     f'sys.stderr.write({json.dumps(stderr_text)} + "\\n"); sys.exit(1)')
+
+    res = _shim_ls(mam_sandbox, env, "flaky-srv")
+    assert res.returncode != 0
+    assert "no server running" not in res.stderr, res.stderr
+    assert res.stdout.strip() == ""
+
+
+def test_reconcile_keeps_live_rows_when_a_name_contains_server_not_running(mam_sandbox):
+    """End to end through reconcile.sh: the server answers a successful list in
+    which the down-server code appears as data. Both registered rows must stay
+    alive with herdr_confirmed intact, so drift A cannot mark them terminated."""
+    env = _install_fake_herdr(mam_sandbox)
+    _fake_agent_list(mam_sandbox, f"print({json.dumps(json.dumps(LISTING_WITH_SUBSTRING))})")
+    rows = "".join(
+        f"  - name: {name}\n"
+        "    status: running\n"
+        "    herdr_session: live-srv\n"
+        "    pane:\n"
+        "      cwd: /tmp/nonexistent-workspace\n"
+        for name in ("server_not_running-codex", "creator-claude-01")
+    )
+    (mam_sandbox / ".mam" / "agent-sessions.yaml").write_text("herdr_sessions:\n" + rows)
+    reconcile = mam_sandbox / "skills" / "multi-agent-mux-monitor" / "scripts" / "reconcile.sh"
+
+    res = subprocess.run(["bash", str(reconcile), "--once", "--emit-diff", "--dry-run"],
+                         capture_output=True, text=True, cwd=str(mam_sandbox), env=env, timeout=120)
+    assert res.returncode == 0, res.stderr
+
+    report = json.loads(res.stdout)
+    assert report["herdr_confirmed"] is True, report
+    alive = set(report["herdr_sessions_alive"])
+    assert {"server_not_running-codex|live-srv", "creator-claude-01|live-srv"} <= alive, alive
+    drift_a = [d for d in report["drifts"] if d.get("class") == "A"]
+    assert not drift_a, drift_a