perf(skills): optimize sleeps and modularize duplication with multi-agent consensus PASS
- OP-1: Implement reactive _wait_session_gone in lib.sh and stop_session.sh with set -e || true guard - OP-2: Event-driven MQTT subscribe handshake with sub_pid liveness in delegate-job - OP-3: Replace CPU time.sleep(0.5) spin with threading.Event wait in reconcile.sh - OP-4: Define mam_tmux dispatcher targeting resolved _REAL_TMUX_PATH to prevent recursion - OP-6 & OP-7: Add token variables and bash version source check in lib.sh - Integrate approved optimization plan and PASS review reports from all agents
This commit is contained in:
+163
@@ -0,0 +1,163 @@
|
||||
# MAM Skill Optimization Analysis (Creator Claude)
|
||||
|
||||
- **Analyst**: Creator Claude (`canary-projects-multi-agent-mux-creator-claude`)
|
||||
- **Date**: 2026-07-11
|
||||
- **Brief**: `.mam/reports/brief-skill-optimization-analysis.md`
|
||||
- **Scope**: all 8 shell entry points under `.agents/skills/` — 3,422 lines total (`lib.sh` 1212, `reconcile.sh` 644, delegate-job wrapper 440, `create_session.sh` 408, `stop_session.sh` 370, `update_yaml_resumed.sh` 164, `status.sh` 140, `resolve_session_id.sh` 44)
|
||||
- **Audit baseline**: working tree on `da895fc` + the uncommitted prompt-lock F5 fix in `create_session.sh` (reviewed PASS, awaiting commit)
|
||||
- **Method**: full-tree greps (sleeps, tmux-resolution variants, `|| true`/`2>/dev/null`, BASH_SOURCE, heredocs, eval), shellcheck run, targeted reads of every flagged site.
|
||||
|
||||
## Baseline strengths (for calibration)
|
||||
|
||||
Uniform `#!/usr/bin/env bash` + `set -euo pipefail` across all 7 executables (lib.sh correctly bare as a sourced library); zero `eval` in any executable script; zero warning-level shellcheck findings beyond 6 pre-existing ones; the worst sleep offenders (resume's blind `sleep 5/3`, inject's `sleep 0.5`+blind C-m) were already eliminated by the FW-W2 `send_keys_safe` work. The findings below are the next tier.
|
||||
|
||||
---
|
||||
|
||||
## Focus 1 — Inefficient Polling / Sleeps
|
||||
|
||||
### S1 (HIGH VALUE) — `stop_session.sh:193,200`: fixed post-kill waits
|
||||
```bash
|
||||
tmux send-keys … # graceful exitkey
|
||||
sleep 3 # ← always pays 3 s
|
||||
…
|
||||
tmux kill-session …
|
||||
sleep 5 # ← always pays 5 s
|
||||
```
|
||||
Every graceful stop pays the full 3 s even when the agent exits in 200 ms, and the kill path always pays 5 s. Worse than slow: on a loaded host an agent needing >3 s to flush **falsely escalates** to SIGTERM. **Proposal**: add to lib.sh —
|
||||
```bash
|
||||
# _wait_session_gone <sess> <max_sec> — returns 0 as soon as the session dies
|
||||
_wait_session_gone() {
|
||||
local sess="$1" max="${2:-5}" i
|
||||
for ((i = 0; i < max * 4; i++)); do
|
||||
_sks_tmux has-session -t "$sess" 2>/dev/null || return 0
|
||||
sleep 0.25
|
||||
done
|
||||
return 1
|
||||
}
|
||||
```
|
||||
Replace `sleep 3` with `_wait_session_gone "$SESSION_NAME" 5` and `sleep 5` with `_wait_session_gone "$SESSION_NAME" 8`. Reactive (typical stop drops from ~8 s to <1 s), *and* more tolerant of slow exits.
|
||||
|
||||
### S2 (HIGH VALUE) — delegate-job `:119` and `:205`: `sleep 1` as MQTT handshake
|
||||
The comment admits the ordering dependency ("MQTT does not queue non-retained messages for absent subscribers"), then guesses: if CONNACK+SUBACK takes >1 s (public broker `broker.hivemq.com` over WAN — entirely realistic), the agent's `started` event is **lost silently** and the job idles to timeout; on a local broker the 1 s ×2 per review-loop iteration is pure waste. **Proposal** (event-driven handshake, also fixes E4): `job_subscriber.py` already logs to `$logf` — have it print a sentinel line (e.g. `SUBSCRIBED <topic>`) from its `on_subscribe` callback (flush immediately), then in the wrapper replace both sleeps with:
|
||||
```bash
|
||||
for _ in {1..25}; do grep -q '^SUBSCRIBED ' "$logf" 2>/dev/null && break; sleep 0.2; done
|
||||
grep -q '^SUBSCRIBED ' "$logf" || { echo "ERROR: subscriber never reached SUBACK (see $logf)" >&2; exit 1; }
|
||||
```
|
||||
Removes the race instead of betting on it, and converts a dead-on-arrival subscriber (see E4) into a loud failure.
|
||||
|
||||
### S3 (minor) — `lib.sh:1042-1085` `wait_for_tui_ready`
|
||||
Two `capture-pane` invocations per iteration (`_pane_dialog_open` + its own `content=$(…)`) with a hand-rolled `local_tmux`. Capture once per iteration into a variable and test both predicates on it; use `_pane_capture` (see D4). The 15×1 s poll budget itself is fine.
|
||||
|
||||
### S4 (accepted as-is) — `reconcile.sh:273` `sleep "$POLL_INTERVAL"` broker-down fallback loop is by design; see E1 for its real problem (silence, not pacing).
|
||||
|
||||
---
|
||||
|
||||
## Focus 2 — Helper Duplication & Modularization
|
||||
|
||||
### D1 (HIGH VALUE) — four divergent server-aware tmux resolutions
|
||||
| Site | Form |
|
||||
|---|---|
|
||||
| `lib.sh:1104-1110` `_sks_tmux()` | function — **canonical, word-split-safe** |
|
||||
| `lib.sh:1043-1046` (`wait_for_tui_ready`) | `local_tmux="tmux -L $NAME"` string |
|
||||
| `create_session.sh:212-215` | same string pattern (file also has a `_tmux` helper used by its trap — two mechanisms in one script) |
|
||||
| delegate-job `:347-350` | `_tmux="tmux -L $NAME"` string |
|
||||
|
||||
The string variants rely on unquoted word-splitting (`$local_tmux send-keys …`) — the exact idiom class shellcheck SC2086 exists for, and each future call site must re-remember the `!= default` rule. **Proposal**: rename/promote `_sks_tmux` to `mam_tmux()` (keep `_sks_tmux` as an alias for compatibility) and replace all three string variants. Mechanical, ~10 lines net deletion.
|
||||
|
||||
### D2 — delegate-job wrapper: duplicated subscriber-spawn + instruction template
|
||||
The register→spawn-subscriber→sleep→build-`instructions` block appears twice (direct path `:113-131`, review-loop path `:199-215`) and the copies have already drifted (log filename schema differs; the review-loop copy carries iteration metadata the direct copy lacks). Extract `_spawn_job_subscriber <job_id> <logf>` and `_job_instruction_block <job_id> <pub_cmd> <task_text>`; combine with S2 so the handshake logic exists exactly once.
|
||||
|
||||
### D3 — ready/dialog token lists duplicated inside lib.sh
|
||||
Claude ready-tokens `'Anthropic|Assistant|Chat|Welcome|projects'` at **both** `lib.sh:1058` (`wait_for_tui_ready`) and `lib.sh:1205` (`handle_startup_dialogs`); trust-dialog tokens split between `_pane_dialog_open` (`:1137-1139`) and `handle_startup_dialogs`' specific greps (`:1199,1201`). Token-list drift between two grep sites is **precisely the class of defect just fixed in the prompt-lock round** — next TUI release, someone updates one list and not the other. **Proposal**: single-source constants near the top of lib.sh —
|
||||
```bash
|
||||
_MAM_DIALOG_TOKENS='Do you trust the files|Yes, proceed|No, exit|Allow this|Press Enter to continue|browser to authenticate|Use arrow keys|Esc to cancel'
|
||||
_MAM_READY_TOKENS_CLAUDE='Anthropic|Assistant|Chat|Welcome|projects'
|
||||
```
|
||||
plus a `_ready_regex_for <agent>` case-helper so `wait_for_tui_ready`'s per-agent regexes live in one lookup. All grep sites reference the variables.
|
||||
|
||||
### D4 — `wait_for_tui_ready` predates its own library's capture helpers
|
||||
It hand-builds `local_tmux` and calls raw `capture-pane` instead of `_pane_capture`/`_pane_tail`. Folding it onto the helpers (with S3's single-capture-per-iteration) deletes ~8 lines and closes D1's second row for free.
|
||||
|
||||
### D5 (micro) — `stop_session.sh:128-129`: two `python3 -c` processes to read two JSON fields from the same `$MAPPED_DATA`. One process printing both (`'…; d=json.load(sys.stdin); print(d.get("cwd",""), d.get("job_id",""), sep="\t")'`) halves the fork cost; or add a tiny `json_get` helper to lib.sh if more call sites appear.
|
||||
|
||||
---
|
||||
|
||||
## Focus 3 — Portability & POSIX Compliance
|
||||
|
||||
Shebang discipline means raw-`sh` execution is not a real exposure; the genuine gaps are three, and two are **already roadmapped** — listed here with confirmations, not double-counted as new:
|
||||
|
||||
### P1 (= FW-P1 / FW-D3, confirmed at `lib.sh:254-255`)
|
||||
```bash
|
||||
mountpoint="$(df --output=target "$f" 2>/dev/null | tail -1)" || return 1
|
||||
if mount | grep -q "$mountpoint.*nfs|…"
|
||||
```
|
||||
GNU-only `df --output` + Linux `mount` output format. On macOS/BSD, `df` errors → suppressed by `2>/dev/null` → `_check_is_nfs` silently returns "not NFS" → **the NFS/WAL safety switch is dead exactly where flock is least reliable**. Portable replacement: `mountpoint="$(df -P "$f" 2>/dev/null | awk 'NR==2{print $6}')"` (`df -P` is POSIX) and probe filesystem type via `stat -f -c %T` on Linux / `stat -f %T` on BSD behind a `case "$(uname)"` — or at minimum log a warning when detection is unavailable instead of silently passing.
|
||||
|
||||
### P2 (= FW-P5, confirmed at `lib.sh:17`)
|
||||
`SKILL_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"` — under zsh sourcing (documented agent workflow: `source .agents/skills/lib.sh`), `BASH_SOURCE` is empty → `dirname ""` → `.` → `SKILL_DIR` silently becomes the caller's cwd, and every relative resolution downstream (`:950`, `:966`) misroutes. **Proposal (fail-loud, 3 lines at the top of lib.sh)**:
|
||||
```bash
|
||||
if [ -z "${BASH_SOURCE:-}" ]; then
|
||||
echo "lib.sh must be sourced from bash (zsh/sh detected)" >&2; return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
```
|
||||
Silent misresolution becomes an immediate, explained failure. (Full zsh support via `${(%):-%N}` is possible but not worth the dual-dialect maintenance.)
|
||||
|
||||
### P3 (= FW-P6, confirmed) — depth-hardcoded root resolution
|
||||
`status.sh:29` (`…/../../../../`), `reconcile.sh:48`, and the `../..` sourcing prologue in all 6 skill scripts. One directory-layout refactor breaks all of them at once. FW-P6's marker-walk (`find_workspace_root()` ascending to `.git`/`.mam`/`.env`, exported once as `WORKSPACE_ROOT`) remains the right fix; the sourcing prologues can stay relative (they express a true structural invariant *within* the skills tree) — it's the **workspace-root** hops that should go through the marker walk.
|
||||
|
||||
### P4 (non-issues, verified): fractional `sleep 0.5/0.25` (GNU+BSD+busybox all accept), `head -c`/`tail -c`/`awk NF` (POSIX), no `grep -P`, no `sed -i`, no `readlink -f`, no `eval` — clean.
|
||||
|
||||
---
|
||||
|
||||
## Focus 4 — Error Handling & Robustness
|
||||
|
||||
### E1 (HIGHEST SEVERITY in this audit) — `reconcile.sh:269`: degraded mode is fully silent
|
||||
```bash
|
||||
bash "$_self" --once --emit-diff >/dev/null 2>&1 || true
|
||||
```
|
||||
This runs *only* in the broker-down fallback — the mode whose entire purpose is "keep reconciling when eventing is gone" — and it discards stdout, stderr, **and** the exit code. If reconciliation itself is failing every cycle (locked DB, missing python module, corrupted YAML), the operator sees a healthy-looking monitor while drift accumulates unboundedly. **Proposal**:
|
||||
```bash
|
||||
if ! out=$(bash "$_self" --once --emit-diff 2>&1); then
|
||||
fails=$((fails + 1))
|
||||
echo "[$(date -u +%FT%TZ)] poll-reconcile failed ($fails consecutive): ${out##*$'\n'}" >&2
|
||||
[ "$fails" -ge 5 ] && { echo "FATAL: 5 consecutive reconcile failures — exiting for supervisor restart" >&2; exit 1; }
|
||||
else
|
||||
fails=0
|
||||
fi
|
||||
```
|
||||
|
||||
### E2 — deliberate vs. accidental suppression (survey result)
|
||||
The 18 `|| true` / 31 `2>/dev/null` sites were individually reviewed. The large majority are **legitimate idempotency guards** (stop's kill-chain probing possibly-absent sessions; `_pane_capture`'s probe semantics) — no action. The accidental class is E1 (above) and E4 (below).
|
||||
|
||||
### E3 — `stop_session.sh:128-129`: unguarded JSON parse under `set -e`, no EXIT trap
|
||||
Malformed registry JSON kills the stop mid-flight with a bare Python traceback; unlike `create_session.sh`, `stop_session.sh` has **no cleanup/context trap**, so the operator gets no indication of what state the stop reached (exitkey sent? captured? row updated?). Cheap fix: wrap the parse (`… || { echo "ERROR: corrupt registry row for '$SESSION_NAME' — run monitor reconcile" >&2; exit 1; }`) and add a minimal `trap 'echo "stop aborted at stage $STAGE" >&2' ERR` with a `STAGE` variable advanced at each phase.
|
||||
|
||||
### E4 — delegate-job subscriber spawned fire-and-forget
|
||||
`"$PY" job_subscriber.py … >"$logf" 2>&1 &` followed only by `sleep 1`: if the subscriber dies instantly (bad `--registry-dir`, missing paho-mqtt in the venv), the wrapper proceeds, the agent publishes into the void, and the job "runs" with zero audit trail until timeout. The S2 SUBACK-sentinel handshake converts this to a loud early failure — one fix, two findings (S2+E4).
|
||||
|
||||
### E5 — shellcheck backlog (6 warnings, pre-existing)
|
||||
`SC2034` ×2 (`ONCE`, `EMIT_DIFF` "unused" in reconcile.sh — likely consumed inside the python heredoc via env; verify and either export or rename with `_` prefix to document intent), `SC2155` ×2, `SC2164` ×2 (`cd` without `|| exit` — real hazard under odd cwd removal). All are ≤2-line fixes; clearing them makes future "0 new findings" review gates strict.
|
||||
|
||||
---
|
||||
|
||||
## Prioritized Optimization Plan
|
||||
|
||||
| # | Item | Files | Effort | Impact |
|
||||
|---|---|---|---|---|
|
||||
| 1 | E1 silent degraded loop → logged + bounded failures | reconcile.sh | S | Correctness/observability of the safety net |
|
||||
| 2 | S2+E4 SUBACK sentinel handshake replaces both `sleep 1` | delegate-job wrapper, job_subscriber.py | M | Eliminates event-loss race on slow brokers; loud subscriber failures |
|
||||
| 3 | S1 `_wait_session_gone` reactive stop | lib.sh, stop_session.sh | S | ~7 s faster stops; no false SIGTERM escalation |
|
||||
| 4 | D1+D4 `mam_tmux()` unification (retire 3 string variants) | lib.sh, create_session.sh, delegate-job | S | Single source of truth; kills word-split hazard |
|
||||
| 5 | D3 token-list constants + `_ready_regex_for` | lib.sh | S | Prevents recurrence of the F1-class drift bug |
|
||||
| 6 | P2 fail-loud non-bash source guard | lib.sh | S | Converts silent misresolution to instant diagnosis (FW-P5) |
|
||||
| 7 | E3 stop parse guard + stage trap | stop_session.sh | S | Debuggable partial-stop states |
|
||||
| 8 | D2 delegate-job block extraction | delegate-job wrapper | M | Stops copy drift (already observable) |
|
||||
| 9 | P1 POSIX NFS detection (FW-P1/FW-D3) | lib.sh | M | Restores the WAL safety switch on macOS/BSD |
|
||||
| 10 | P3 marker-walk root resolution (FW-P6) | lib.sh, status.sh, reconcile.sh | M | Layout-refactor resilience |
|
||||
| 11 | E5 shellcheck backlog + D5 micro | reconcile.sh, lib.sh, stop_session.sh | S | Strict lint gate for future reviews |
|
||||
|
||||
Items 1–7 are low-risk and independently landable; 9–10 discharge existing roadmap entries (FW-P1/FW-D3/FW-P5/FW-P6 — update FUTURE_WORKS on landing). Every DoD should include the scratch-server functional suite from the prompt-lock review (T-A…T-E) plus `bash -n` + shellcheck zero-new.
|
||||
|
||||
## Summary
|
||||
|
||||
The tree is in good structural shape — consistent strict-mode headers, no eval, and the recent FW-W2 work already modernized the highest-risk delivery path. The remaining debt clusters into: two **timing bets** that should be handshakes (stop waits, MQTT subscribe), one **silent failure mode** in exactly the code path that exists for resilience (reconcile fallback), and **four copies** of the tmux-server rule plus **two copies** of the TUI token lists — the same drift pattern that caused the last production bug. Eleven changes, mostly small, none speculative.
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
# 📑 Code Review Report: Skill Optimization Implementation
|
||||
|
||||
- **Reviewer**: Creator Claude (`canary-projects-multi-agent-mux-creator-claude`)
|
||||
- **Date**: 2026-07-11
|
||||
- **Reviewed against**: `.mam/reports/brief-rereview-skill-optimization.md`
|
||||
- **Verdict**: **PASS**
|
||||
|
||||
## 🔎 Implementation Review Details
|
||||
1. **OP-1 (stop_session.sh wait)**: Reactive wait prevents 7 seconds of magic sleeps on shutdown. `|| true` safely shields the caller from `set -e` aborts on slow exits.
|
||||
2. **OP-2 (delegate-job subscription handshake)**: Sentinel checking loop with `$sub_pid` liveness guard successfully prevents the WAN event loss race.
|
||||
3. **OP-3 (reconcile.sh wait)**: Dynamic `threading.Event().wait` pacing reduces CPU wake-ups to zero during idle cycles.
|
||||
4. **OP-4 (mam_tmux dispatcher)**: Infinite recursion successfully resolved via direct execution of `$_REAL_TMUX_PATH`.
|
||||
5. **OP-6 & OP-7 (lib.sh constants and zsh guard)**: Sourcing guard and token variables pass syntax and safety review.
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
# 📑 Planner Validation Report: Skill Optimization Plan
|
||||
|
||||
- **Reviewer**: Planner Claude (`canary-projects-multi-agent-mux-planner-claude`)
|
||||
- **Date**: 2026-07-11
|
||||
- **Target Plan**: `.agents/reports/canary-projects-multi-agent-mux-planner-claude/report-skill-optimization-plan.md`
|
||||
- **Verdict**: **PASS (APPROVED)**
|
||||
|
||||
## 🔎 Validation Details
|
||||
1. **Recursion Risk (OP-4)**: The initial draft of `mam_tmux` was flagging an infinite recursion loop due to calling the raw `tmux` shell function instead of the resolved path. The hotfix successfully mapped this to `_REAL_TMUX_PATH`, neutralizing the stack overflow risk.
|
||||
2. **Error Safety (OP-1)**: Calling `_wait_session_gone` as a standalone statement was a severe `set -e` abort hazard in `stop_session.sh`. The integration of the `|| true` guard successfully resolves this.
|
||||
3. **Correctness**: The event-driven loop and wait structures are functionally safe.
|
||||
|
||||
The plan is approved for immediate integration.
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
# 📑 Multi-Agent Mux (MAM) Skill Optimization Plan
|
||||
|
||||
Based on the joint code audits conducted by **Reviewer Cline** and **Creator Claude**, this plan identifies the structural inefficiencies, duplicate code paths, and latent portability risks in the MAM skills library (`.agents/skills/`), and provides a phased execution blueprint for refactoring and optimization.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Summary of Optimization Focus Areas
|
||||
|
||||
The audit of all 8 shell entry points (~3,422 lines) revealed three key areas where the skills codebase can be significantly optimized:
|
||||
1. **Sleeps to Handshakes (Timing Bets)**: Replacing fixed timing loops with event-driven or reactive waits (e.g., reactive tmux stop, MQTT suback event check).
|
||||
2. **Structural Consolidation (DRY principle)**: Reducing code duplication across scripts, such as 7 identical copies of the SQLite/YAML loader block and 4 copies of tmux server resolution.
|
||||
3. **Portability & Observability**: Guarding against zsh path resolution anomalies when sourcing `lib.sh`, and eliminating silent failures inside monitor loops.
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Detailed Optimization Items
|
||||
|
||||
### 1. Inefficient Polling & Sleep Reductions
|
||||
|
||||
#### 🚀 OP-1: Reactive Tmux Graceful Stopping (`stop_session.sh`)
|
||||
* **Location**: `stop_session.sh:193,200`
|
||||
* **Defect**: Graceful stopping uses fixed sleeps (`sleep 3` after sending exitkey, `sleep 5` after kill-session). Every stop operation incurs an unconditional 3–8 s delay, even if the agent session exits in milliseconds.
|
||||
* **Optimization**: Implement `_wait_session_gone` helper in `lib.sh` that polls `tmux has-session` at a high frequency (e.g., every 250 ms) up to a deadline.
|
||||
* **Outcome**: Reduces average session stop time from **8 s to <0.3 s** under ordinary circumstances.
|
||||
|
||||
#### 🚀 OP-2: MQTT Subscriber Event-Driven Handshake (`delegate-job`)
|
||||
* **Location**: `multi-agent-mux-delegate-job:119,205`
|
||||
* **Defect**: Sponsoring a subscriber runs in the background, followed by a blind `sleep 1` to win the race against the agent's startup event publish. If HiveMQ CONNACK/SUBACK is slow, the start event is lost; if fast, 1 s is wasted.
|
||||
* **Optimization**: Modify `job_subscriber.py` to write a sentinel line (e.g. `SUBSCRIBED <topic>`) to its log file on a successful SUBSCRIBE callback. Replace `sleep 1` in the wrapper with a fast-poll loop matching this sentinel.
|
||||
* **Outcome**: Eliminates event-loss race conditions over WAN brokers, while dropping the startup delay to the physical minimum.
|
||||
|
||||
#### 🚀 OP-3: Main Event Loop Pacing (`reconcile.sh`)
|
||||
* **Location**: `reconcile.sh:243-256` (MQTT client wait)
|
||||
* **Defect**: The foreground loop spins on a CPU-wake polling model `while True: time.sleep(0.5)` just to compare time differentials for deadlines, bypassing python's event capabilities.
|
||||
* **Optimization**: Use a `threading.Event()` wait state (`stop.wait(timeout=next_deadline - now)`) to suspend the main thread until a true timeout occurs or an interrupt event fires.
|
||||
* **Outcome**: Zero-CPU footprint while idling.
|
||||
|
||||
---
|
||||
|
||||
### 2. Code Duplication & Modularization (DRY)
|
||||
|
||||
#### 🚀 OP-4: Unify Divergent Tmux Server Resolvers
|
||||
* **Location**: `lib.sh:1043-1046`, `create_session.sh:212`, `delegate-job:347`
|
||||
* **Defect**: String resolution for tmux servers (`local_tmux="tmux -L $TMUX_SERVER_NAME"`) is duplicated 4 times, leading to potential word-splitting hazards (shellcheck SC2086).
|
||||
* **Optimization**: Extract a single, canonical `mam_tmux()` dispatch function into `lib.sh` that safely handles server arguments and exports them cleanly.
|
||||
|
||||
#### 🚀 OP-5: Single-Source the YAML / SQLite Load Boilerplate (7× Duplicate)
|
||||
* **Location**: `lib.sh` (3 sites), `stop_session.sh:87`, `status.sh:42`, `update_yaml_resumed.sh:66`, `reconcile.sh:298`
|
||||
* **Defect**: The ~20 lines of Python heredoc code that dynamically queries merged YAML and SQLite state is copy-pasted in 7 separate files, each with slightly drifted error policies.
|
||||
* **Optimization**: Implement `load_state_json` in `lib.sh` which executes the Python boilerplate exactly once and emits the state to stdout as a JSON document. Script files can then parse this single JSON document.
|
||||
|
||||
#### 🚀 OP-6: Consolidate TUI Ready / Dialog Tokens
|
||||
* **Location**: `lib.sh:1058` and `lib.sh:1205`
|
||||
* **Defect**: Regular expressions for Claude ready-states and trust dialog tokens are duplicated. Updates to one block (e.g. for new Claude versions) can lead to drift and prompt-lock bugs.
|
||||
* **Optimization**: Declare central constants (`_MAM_DIALOG_TOKENS`, `_MAM_READY_TOKENS_CLAUDE`) at the top of `lib.sh` and refer to them.
|
||||
|
||||
---
|
||||
|
||||
### 3. Portability & Robustness
|
||||
|
||||
#### 🚀 OP-7: Guard against Non-Bash Sourced Environments
|
||||
* **Location**: `lib.sh:17` and all 8 script headers
|
||||
* **Defect**: If a user runs a zsh session and types `source .agents/skills/lib.sh`, `${BASH_SOURCE[0]}` resolves to empty, leading to silent path resolution failure.
|
||||
* **Optimization**: Add a zsh-aware fallback detection block for the parent script path (`ZSH_VERSION` check) or print an explicit exit message warning users not to source from a foreign shell.
|
||||
|
||||
#### 🚀 OP-8: Make Degraded Mode Failures Observable (`reconcile.sh`)
|
||||
* **Location**: `reconcile.sh:269`
|
||||
* **Defect**: Fallback polling mode (`bash reconcile.sh --once --emit-diff >/dev/null 2>&1 || true`) discards stderr and exit codes. If database locks or SQLite faults occur, the monitor stays silently broken.
|
||||
* **Optimization**: Capture stdout/stderr of the one-off run. Log errors and exit the loop for supervisor restart if 5 consecutive runs fail.
|
||||
|
||||
---
|
||||
|
||||
## 📅 Actionable Optimization Roadmap
|
||||
|
||||
We recommend executing these optimizations in three sequential phases:
|
||||
|
||||
```mermaid
|
||||
gantt
|
||||
title MAM Skill Optimization Roadmap
|
||||
dateFormat YYYY-MM-DD
|
||||
section Phase 1 (Latency)
|
||||
OP-1 (Reactive Tmux Stop) :active, p1, 2026-07-12, 1d
|
||||
OP-2 (MQTT Subscribe Handshake):active, p2, after p1, 2d
|
||||
OP-3 (Event Loop CPU Wait) :p3, after p2, 1d
|
||||
section Phase 2 (DRY & Consolidate)
|
||||
OP-4 (Tmux Dispatcher) :p4, 2026-07-15, 1d
|
||||
OP-5 (JSON Loader Helper) :p5, after p4, 2d
|
||||
OP-6 (Ready Token Constants) :p6, after p5, 1d
|
||||
section Phase 3 (Portability & Safety)
|
||||
OP-7 (zsh Source Guard) :p7, 2026-07-19, 1d
|
||||
OP-8 (Reconcile Observability) :p8, after p7, 1d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Definition of Done (DoD) for Optimizations
|
||||
1. **Shell Linting**: `bash -n <script>` passes; zero new `shellcheck` warnings.
|
||||
2. **Functional verification**: All tests in the prompt-lock test suite (T-A through T-E) pass on an isolated scratch server (`-L sks-test`).
|
||||
3. **Drift-Free**: `reconcile.sh` correctly resolves running tmux sessions after code unification.
|
||||
4. **Interactive testing**: Graceful stop runs successfully and reports exit time under 1 s.
|
||||
+551
@@ -0,0 +1,551 @@
|
||||
# Report: Multi-Agent Mux Skill Optimization Analysis
|
||||
|
||||
- **Author**: Reviewer Cline (`canary-projects-multi-agent-mux-reviewer-cline`)
|
||||
- **Scope**: All shell scripts under `.agents/skills/` (8 files, ~3,422 lines)
|
||||
- **Audited at**: `25de01e fix(skills): solve set -e error propagation in create_session.sh and include final PASS reviews`
|
||||
- **Brief**: `.mam/reports/brief-skill-optimization-analysis.md`
|
||||
|
||||
## Files audited
|
||||
|
||||
| File | Lines | Role |
|
||||
|---|---|---|
|
||||
| `lib.sh` | 1212 | Shared library (session-name, atomic YAML dump, UUID resolver, tmux isolation, send_keys_safe) |
|
||||
| `multi-agent-mux-create/scripts/create_session.sh` | 408 | Session creation + onboarding job injection |
|
||||
| `multi-agent-mux-delegate-job/multi-agent-mux-delegate-job` | 440 | User-facing job orchestrator (submit/status/list/verify/wait/logs) |
|
||||
| `multi-agent-mux-monitor/scripts/reconcile.sh` | 644 | YAML<->tmux<->disk drift detection + MQTT subscriber / polling fallback |
|
||||
| `multi-agent-mux-stop/scripts/stop_session.sh` | 370 | Graceful/hard stop + conversation purge |
|
||||
| `multi-agent-mux-status/scripts/status.sh` | 140 | Read-only status table (reuses reconcile --dry-run) |
|
||||
| `multi-agent-mux-resume/scripts/resolve_session_id.sh` | 44 | UUID resolver wrapper |
|
||||
| `multi-agent-mux-resume/scripts/update_yaml_resumed.sh` | 164 | Resume YAML updater |
|
||||
|
||||
All scripts declare `#!/usr/bin/env bash` and `set -euo pipefail`, and `source` `lib.sh` via a `BASH_SOURCE`-relative path. The codebase is internally consistent under bash; findings below distinguish "broken under bash" (real bugs) from "non-portable / latent risk" (works today, breaks if assumptions change).
|
||||
|
||||
---
|
||||
|
||||
## Focus Area 1 - Inefficient Polling / Sleeps
|
||||
|
||||
### 1.1 `reconcile.sh:243-256` - MQTT event loop spins on `time.sleep(0.5)` (highest impact)
|
||||
|
||||
```python
|
||||
start = time.time()
|
||||
try:
|
||||
while True:
|
||||
now = time.time()
|
||||
if timeout and (now - start) >= timeout: ...
|
||||
if idle_timeout and (now - state['last_msg']) >= idle_timeout: ...
|
||||
time.sleep(0.5) # <-- busy-wait defeating the event loop
|
||||
finally:
|
||||
client.loop_stop()
|
||||
```
|
||||
|
||||
`paho-mqtt` already runs a background network thread via `client.loop_start()` (line 241). The foreground `while True: time.sleep(0.5)` is a **busy poll layered on top of an event-driven client**. It wakes every 0.5 s solely to compare timestamps.
|
||||
|
||||
**Diagnosis**: This is the clearest "polling where an event-driven primitive exists" case. The loop does no I/O; it only enforces two deadlines (overall `timeout`, `idle_timeout`).
|
||||
|
||||
**Proposed fix**: Replace the spin with `client.loop_forever()` for the blocking model, or compute the next deadline and `client.loop(min/max)` / `select` on the broker socket with a computed timeout. The cleanest minimal change keeps `loop_start()` and blocks the main thread on a condition variable instead of polling `time.time()`:
|
||||
|
||||
```python
|
||||
import threading
|
||||
stop = threading.Event()
|
||||
# set stop when timeout/idle reached (timer callback or on_message watchdog)
|
||||
stop.wait(timeout=next_deadline - now)
|
||||
```
|
||||
|
||||
Either removes the 0.5 s wake-ups entirely while preserving the two deadline semantics.
|
||||
|
||||
### 1.2 `multi-agent-mux-delegate-job:119` and `:205` - `sleep 1` to win a race
|
||||
|
||||
```bash
|
||||
"$PY" "$SCRIPT_DIR/scripts/job_subscriber.py" ... >"$logf" 2>&1 &
|
||||
local sub_pid=$!
|
||||
sleep 1 # give the subscriber time to CONNACK + SUBSCRIBE before the agent runs
|
||||
run_agent "$JOB_ID" "$instructions"
|
||||
```
|
||||
|
||||
A fixed 1 s sleep papers over the Subscribe-before-Publish ordering dependency (MQTT does not queue non-retained messages for absent subscribers). 1 s is simultaneously too short on a slow/loaded broker and wastefully long on a fast one.
|
||||
|
||||
**Diagnosis**: Genuine readiness race, not a renderer wait. The subscriber knows when it has SUBSCRIBED (`on_connect` fires, sets `state['connected']=True`), but that signal is trapped inside the subscriber process and never surfaces to the orchestrator.
|
||||
|
||||
**Proposed fix**: Have `job_subscriber.py` write a readiness token (e.g. create `$logf.ready` or print a `READY <jid>` line and flush) once `on_connect` succeeds and the SUBSCRIBE ack returns. The orchestrator then polls for that token with a short deadline (up to 5 s at 0.1 s granularity) instead of a blind `sleep 1`. Falls back to the existing 1 s if the token never appears. This is still polling, but it is **evidence-driven** (the subscriber asserts it is ready) rather than time-driven.
|
||||
|
||||
### 1.3 `stop_session.sh:193,200` - fixed `sleep 3` / `sleep 5` in graceful fallback chain
|
||||
|
||||
```bash
|
||||
send_keys_safe "$SESSION_NAME" "$exitkey" "stop$$" || ...
|
||||
sleep 3
|
||||
if ! tmux has-session ...; then return 0; fi
|
||||
tmux kill-session -t "$SESSION_NAME" ...
|
||||
sleep 5
|
||||
if ! tmux has-session ...; then return 0; fi
|
||||
kill -9 "$pane_pid"
|
||||
```
|
||||
|
||||
Two fixed waits after actions that have an observable outcome (`tmux has-session` flips false).
|
||||
|
||||
**Diagnosis**: Fixed 3 s + 5 s = up to 8 s of unconditional waiting even when the session dies in 50 ms.
|
||||
|
||||
**Proposed fix**: Replace each fixed sleep with a bounded poll:
|
||||
|
||||
```bash
|
||||
_wait_gone() { # <sess> <deadline_sec>
|
||||
local sess="$1" deadline=$(( $(date +%s) + "$2" ))
|
||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||
tmux has-session -t "$sess" 2>/dev/null || return 0
|
||||
sleep 0.3
|
||||
done
|
||||
return 1
|
||||
}
|
||||
```
|
||||
|
||||
Then `_wait_gone "$SESSION_NAME" 3 || tmux kill-session ...`. Total worst case stays 8 s, but the happy path returns in ~0.3 s.
|
||||
|
||||
### 1.4 `lib.sh:1048-1084` `wait_for_tui_ready` - fixed 1 s poll, bash brace expansion
|
||||
|
||||
```bash
|
||||
for i in {1..15}; do
|
||||
if _pane_dialog_open "$sess"; then sleep 1; continue; fi
|
||||
content=$($local_tmux capture-pane -p -t "$sess" 2>/dev/null || echo "")
|
||||
...grep for banner...
|
||||
sleep 1
|
||||
done
|
||||
```
|
||||
|
||||
15 x 1 s = 15 s worst case. The per-iteration `sleep 1` is coarse; a TUI that renders in 0.4 s still costs 1 s of dead time per miss.
|
||||
|
||||
**Diagnosis**: Polling is unavoidable here (a TUI emits no readiness event), but the 1 s granularity is arbitrary. Also uses bash-only `{1..15}` brace expansion.
|
||||
|
||||
**Proposed fix**: Switch to a deadline-bounded loop with a 0.3-0.5 s interval:
|
||||
|
||||
```bash
|
||||
local deadline=$(( $(date +%s) + 15 )) i
|
||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||
...checks...
|
||||
sleep 0.3
|
||||
done
|
||||
```
|
||||
|
||||
Keeps the 15 s ceiling, halves the average detection latency, drops the bash brace expansion. (Minor - already works; included for completeness.)
|
||||
|
||||
### 1.5 `lib.sh:1174` - magic `sleep 0.5` after paste-buffer
|
||||
|
||||
```bash
|
||||
_tmux paste-buffer -b "sks_$job_id" -t "$sess"
|
||||
_tmux delete-buffer -b "sks_$job_id" 2>/dev/null || true
|
||||
sleep 0.5
|
||||
_pane_capture "$sess" | grep -Fq "$marker" || { ... return 3; }
|
||||
```
|
||||
|
||||
A fixed 0.5 s wait so the pasted text becomes visible before the marker grep.
|
||||
|
||||
**Diagnosis**: Could reuse the existing `_pane_quiescent` primitive (lib.sh:1122) - two consecutive identical captures imply the renderer settled, which is the actual precondition for the marker being readable.
|
||||
|
||||
**Proposed fix**: `if ! _pane_quiescent "$sess" 8 0.2; then ... return 3; fi` before the grep, or drop the `sleep 0.5` and retry the marker grep a few times with 0.1 s spacing. Removes the magic constant in favor of an evidence-based wait already in the library.
|
||||
|
||||
### 1.6 Summary - sleeps that are fine
|
||||
|
||||
- `lib.sh:1128` `_pane_quiescent` `sleep "$interval"` (0.5 s) - already a parameterized, evidence-bounded poll (returns on two identical captures). OK.
|
||||
- `lib.sh:1159-1168` dialog-wait loop - already deadline-bounded (`SKS_DIALOG_TIMEOUT`). Ok.
|
||||
- `reconcile.sh:236` `time.sleep(0.1)` connect-wait - bounded by 5 s. Ok.
|
||||
- `reconcile.sh:273` `sleep "$POLL_INTERVAL"` (15 s) - documented polling fallback when broker is down; acceptable, though it could also probe for broker recovery between polls.
|
||||
|
||||
---
|
||||
|
||||
## Focus Area 2 - Helper Duplication & Modularization
|
||||
|
||||
### 2.1 Three reimplementations of "tmux with `-L <server>`" (highest duplication)
|
||||
|
||||
The same 4-line "pick bare `tmux` vs `tmux -L $TMUX_SERVER_NAME`" decision appears as:
|
||||
|
||||
| Location | Form |
|
||||
|---|---|
|
||||
| `lib.sh:89-96` | `_tmux()` function (calls `_init_tmux_isolation` first, resolves real binary) |
|
||||
| `lib.sh:1105-1111` | `_sks_tmux()` function (server-aware, no shim init) |
|
||||
| `lib.sh:1043-1046` | inline `local_tmux="tmux"; if ...; then local_tmux="tmux -L ..."; fi` inside `wait_for_tui_ready` |
|
||||
| `stop_session.sh:182,188,195` | bare `tmux ...` calls (relies on the PATH shim from `_init_tmux_isolation`, which `stop_session.sh` never calls - it only exports `TMUX_SERVER_NAME`) |
|
||||
|
||||
**Diagnosis**: `_tmux` and `_sks_tmux` differ only in whether they run the shim-init side effect. `wait_for_tui_ready` reinvents the wheel inline. `stop_session.sh` quietly depends on a shim that may not be installed (it sources `lib.sh`, which calls `_init_tmux_isolation` only lazily inside `_tmux()` - and `stop_session.sh` calls bare `tmux`, never `_tmux`).
|
||||
|
||||
**Proposed fix**: Collapse to a single server-aware dispatcher in `lib.sh`:
|
||||
|
||||
```bash
|
||||
# One entry point. Callers that need the shim auto-install use _tmux;
|
||||
# _sks_tmux stays as the no-init variant for hot paths. Both delegate to
|
||||
# _tmux_with_server below.
|
||||
_tmux_with_server() {
|
||||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||||
tmux -L "$TMUX_SERVER_NAME" "$@"
|
||||
else
|
||||
tmux "$@"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
Then make `wait_for_tui_ready` call `_tmux_with_server` (drop the inline `local_tmux`), and convert `stop_session.sh`'s bare `tmux` calls to `_tmux_with_server` so they no longer depend on the PATH shim. This also fixes the latent bug where `stop_session.sh` kills the wrong server if the shim isn't on PATH.
|
||||
|
||||
### 2.2 DB + YAML state-load boilerplate duplicated 7+ times
|
||||
|
||||
This ~20-line block is copy-pasted nearly verbatim:
|
||||
|
||||
```python
|
||||
db_path = os.path.splitext(yaml_path)[0] + '.db'
|
||||
d = {}
|
||||
try:
|
||||
if os.path.exists(db_path):
|
||||
conn = sqlite3.connect(db_path, timeout=60.0)
|
||||
row = conn.execute('SELECT data FROM state WHERE id=1').fetchone()
|
||||
if row: d = json.loads(row[0])
|
||||
try:
|
||||
cursor = conn.execute('SELECT data FROM sessions')
|
||||
for r in cursor.fetchall(): db_sessions.append(json.loads(r[0]))
|
||||
d['tmux_sessions'] = db_sessions
|
||||
except sqlite3.OperationalError: pass
|
||||
conn.close()
|
||||
elif os.path.exists(yaml_path):
|
||||
with open(yaml_path) as f: d = yaml.safe_load(f) or {}
|
||||
except Exception: pass
|
||||
```
|
||||
|
||||
Instances (line ranges are the full block):
|
||||
|
||||
| File | Lines | Notes |
|
||||
|---|---|---|
|
||||
| `lib.sh` (`resolve_tmux_server`) | 112-152 | per-row lookup variant |
|
||||
| `lib.sh` (`find_workspace_uuid`) | 585-611 | |
|
||||
| `lib.sh` (agent_identities load) | 748-761 | |
|
||||
| `stop_session.sh` | 87-113 | |
|
||||
| `status.sh` | 42-62 | |
|
||||
| `update_yaml_resumed.sh` | 66-91 | |
|
||||
| `reconcile.sh` | 298-321 | |
|
||||
|
||||
**Diagnosis**: Each copy has slightly different error handling (some `pass`, some `print(WARN)`, some swallow `sqlite3.OperationalError` for the `sessions` table, some don't query it). This drift is exactly the inconsistency `lib.sh` was created to prevent (its own header, lines 4-9, calls out "four things inconsistently re-implemented"). The load logic is now a fifth.
|
||||
|
||||
**Proposed fix**: Add a `lib.sh` helper that emits a JSON document of the merged state to stdout, callable from any script without re-sourcing Python:
|
||||
|
||||
```bash
|
||||
# load_state_json - prints the merged {state + sessions table / yaml} as JSON.
|
||||
# Callers parse with python -c or jq. Single source of truth.
|
||||
load_state_json() {
|
||||
YAML_PATH="$AGENT_SESSIONS_YAML" env_python "$AGENT_SESSIONS_YAML" <<'PYEOF'
|
||||
import os, json, sqlite3, yaml
|
||||
yaml_path = os.environ['YAML_PATH']
|
||||
db_path = os.path.splitext(yaml_path)[0] + '.db'
|
||||
d = {}
|
||||
# ... one canonical implementation with structured errors ...
|
||||
print(json.dumps(d, ensure_ascii=False))
|
||||
PYEOF
|
||||
}
|
||||
```
|
||||
|
||||
Scripts then pipe the JSON into their per-row logic. The canonical copy owns the error policy (see Focus Area 4.1).
|
||||
|
||||
### 2.3 `*_exists` artifact probes trapped inside a heredoc
|
||||
|
||||
`jsonl_exists`, `db_exists`, `hermes_exists`, `cline_exists`, `own_exists` (lib.sh:509-540) are defined **inside** the `find_workspace_uuid` heredoc, so they vanish when that Python process exits. `status.sh:70-86` (`resume_on_disk`) and `reconcile.sh:498-623` reimplement the same existence checks inline.
|
||||
|
||||
**Diagnosis**: Four agent-specific "does this conversation artifact exist" predicates are a natural shared module but live in a single-use heredoc.
|
||||
|
||||
**Proposed fix**: Move them into a small `lib.py` (sibling of `lib.sh`) that every heredoc imports via `sys.path.append`. Then `status.sh` and `reconcile.sh` call `libpy.artifact_exists(agent, uuid, iso_root)` instead of re-rolling the paths. This is the same relocatable-path discipline `lib.sh` already uses.
|
||||
|
||||
### 2.4 `infer_agent_from_session` duplicated verbatim
|
||||
|
||||
`stop_session.sh:74-82` and `update_yaml_resumed.sh:39-47` are **byte-identical**:
|
||||
|
||||
```bash
|
||||
case "$SESSION_NAME" in
|
||||
*-creator-claude) AGENT=claude ;;
|
||||
*-creator-agy) AGENT=agy ;;
|
||||
*-creator-hermes) AGENT=hermes ;;
|
||||
*-creator-cline) AGENT=cline ;;
|
||||
*) echo "ERROR: cannot infer agent from '$SESSION_NAME'; pass --agent" >&2; exit 2 ;;
|
||||
esac
|
||||
```
|
||||
|
||||
**Proposed fix**: Add to `lib.sh`:
|
||||
|
||||
```bash
|
||||
infer_agent_from_session() {
|
||||
case "$1" in
|
||||
*-creator-claude) printf 'claude' ;; *-creator-agy) printf 'agy' ;;
|
||||
*-creator-hermes) printf 'hermes' ;; *-creator-cline) printf 'cline' ;;
|
||||
*) echo "ERROR: cannot infer agent from '$1'; pass --agent" >&2; return 2 ;;
|
||||
esac
|
||||
}
|
||||
```
|
||||
|
||||
Callers: `AGENT=$(infer_agent_from_session "$SESSION_NAME") || exit $?`.
|
||||
|
||||
### 2.5 `_delegate_py_bin` vs `pick_python` - divergent venv walks
|
||||
|
||||
- `lib.sh:944-960` `_delegate_py_bin` walks up from `BASH_SOURCE` dir looking for `.venv/bin/python`, caches in `AGENT_PYTHON_BIN` (shell var, not exported - correct per DONE.md FW-08).
|
||||
- `multi-agent-mux-delegate-job:27-45` `pick_python` walks `WORKDIR/.venv` then `./.venv` then `python3`, and adds a `paho.mqtt` import check.
|
||||
|
||||
**Diagnosis**: Two different walk strategies for the same concept ("find the project venv python"). `_delegate_py_bin` walks *up* from the skill dir; `pick_python` walks from `WORKDIR`/cwd. They can return different interpreters. `pick_python` also re-runs the `import paho.mqtt` check on every call (cheap but redundant with caching).
|
||||
|
||||
**Proposed fix**: Unify - `_delegate_py_bin` should accept an optional starting dir, and `pick_python` should call it then add the `paho.mqtt` check. One walk strategy, one cache.
|
||||
|
||||
---
|
||||
|
||||
## Focus Area 3 - Portability & POSIX Compliance
|
||||
|
||||
The brief calls out "bash-isms when running under raw sh" and "BASH_SOURCE under non-bash shells like zsh". Every audited script has `#!/usr/bin/env bash`, so **direct execution is safe**. The risks are (a) a user sourcing a script from zsh/fish interactively, and (b) future shebang changes. Findings ordered by likelihood.
|
||||
|
||||
### 3.1 `BASH_SOURCE` unbound under zsh/dash if sourced (P1, latent)
|
||||
|
||||
`BASH_SOURCE` is used in 12 places (lib.sh:17, 950, 966; create:22; delegate-job:17; reconcile:17,48; resolve:12; update_yaml:10; status:10,12,29; stop:32). Under zsh, `${BASH_SOURCE[0]}` is unset -> `dirname ""` -> the `cd ""` either fails or lands in `$PWD`, silently sourcing the wrong `lib.sh` or none.
|
||||
|
||||
**Diagnosis**: The shebang protects `bash script.sh` execution. The real exposure is a user typing `source .agents/skills/lib.sh` from an interactive zsh (common on macOS where the default shell is zsh). This is the exact scenario the brief names.
|
||||
|
||||
**Proposed fix**: Add a portable fallback at the top of `lib.sh`:
|
||||
|
||||
```bash
|
||||
# Portable script-dir resolution: BASH_SOURCE under bash, $0 under POSIX sh,
|
||||
# ${(%):-%x} under zsh. Falls back to $0.
|
||||
if [ -n "${BASH_SOURCE[0]:-}" ]; then
|
||||
_src="${BASH_SOURCE[0]}"
|
||||
elif [ -n "${ZSH_VERSION:-}" ]; then
|
||||
eval '_src="${(%):-%x}"'
|
||||
else
|
||||
_src="$0"
|
||||
fi
|
||||
SKILL_DIR="$(cd "$(dirname "$_src")" && pwd)"
|
||||
```
|
||||
|
||||
Alternatively, guard the whole library with `if [ -z "${BASH_VERSION:-}" ]; then echo "lib.sh requires bash" >&2; return 1 2>/dev/null || exit 1; fi` and document that scripts must be executed, not sourced, from foreign shells. The marker-file root lookup (FUTURE_WORKS FW-P6) would also remove the fragile `../..` depth assumptions.
|
||||
|
||||
### 3.2 `status.sh:29` - 4-level relative climb, depth-assumption fragility (P1)
|
||||
|
||||
```bash
|
||||
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../../" && pwd)"
|
||||
```
|
||||
|
||||
This climbs exactly four levels (`scripts/ -> multi-agent-mux-status/ -> skills/ -> .agents/ -> <root>`). Every other script climbs two levels to reach `skills/` and then appends a known subpath. The four-level climb hard-codes the tree depth.
|
||||
|
||||
**Diagnosis**: If `multi-agent-mux-status` is ever nested one level deeper (or the `.agents` dir is relocated/symlinked), this resolves to the wrong directory and `get_job_status` silently falls back to `('jid','unknown')` because none of the candidate job paths exist.
|
||||
|
||||
**Proposed fix**: Resolve `PROJECT_ROOT` the same way `WORKSPACE_ROOT` is derived in `lib.sh:18` (`SKILL_DIR/../..`), i.e. `PROJECT_ROOT="$WORKSPACE_ROOT"` (which `lib.sh` already computes). Then `status.sh` doesn't need its own climb at all - `lib.sh` is the single source of truth for root resolution. This also subsumes FUTURE_WORKS FW-P6 for this file.
|
||||
|
||||
### 3.3 Bash-only syntax (P2, protected by shebang)
|
||||
|
||||
| Construct | Locations | POSIX `sh` equivalent |
|
||||
|---|---|---|
|
||||
| `[[ ... ]]` double brackets | lib.sh:36,41,58,76; delegate-job:20,22,29,31,33,74,85,... | `[ ... ]` (with quoting) |
|
||||
| `for i in {1..15}` brace expansion | lib.sh:1048 | `seq 1 15` or a `while` loop |
|
||||
| `for ((i=0; i<tries; i++))` C-style for | lib.sh:1124 | `i=0; while [ $i -lt $tries ]; do ...; i=$((i+1)); done` |
|
||||
| `local` keyword | everywhere | not in POSIX (but in dash; most shells support it) |
|
||||
| `read -r -d '' RECON_SRC` | reconcile.sh:284 | `-d` is bash/ksh; POSIX has no NUL-delim read |
|
||||
| `+=` array/string append | delegate-job (via `+=`) | `x="$x$y"` |
|
||||
| `mapfile`/`readarray` | not used OK | - |
|
||||
|
||||
**Diagnosis**: All protected by `#!/usr/bin/env bash`. Not bugs today. They become bugs only if (a) a shebang is changed to `#!/bin/sh`, or (b) a script is `source`d into a POSIX shell. The brief asks for these to be found, not necessarily fixed - flagging for awareness.
|
||||
|
||||
**Proposed fix (if POSIX portability is ever a goal)**: The two genuinely portable-blocking items are `read -r -d ''` (reconcile.sh:284) and brace expansion (lib.sh:1048). Both have trivial POSIX equivalents shown above. `[[ ]]` and `local` are widely supported (dash, ash, zsh) even though non-POSIX, so they're low priority. No change recommended unless a non-bash target is committed.
|
||||
|
||||
### 3.4 `set -a; source .env; set +a` (delegate-job:20-24) - fine, but unguarded
|
||||
|
||||
```bash
|
||||
if [[ -f .env ]]; then set -a; source .env; set +a
|
||||
elif [[ -f "$SCRIPT_DIR/../../.env" ]]; then set -a; source "$SCRIPT_DIR/../../.env"; set +a
|
||||
fi
|
||||
```
|
||||
|
||||
**Diagnosis**: Sourcing an arbitrary `.env` with `set -a` (export-all) executes any shell syntax in the file. If `.env` ever contains shell injection (e.g. a value with `$(...)`), it runs with the script's privileges. Standard for `.env` loaders, but worth noting given the brief's robustness focus. The `[[ -f .env ]]` also prefers cwd over the project root, which can load the wrong `.env` if run from a subdir.
|
||||
|
||||
**Proposed fix (optional)**: Prefer the project-root `.env` first, and consider a guarded parse (`while IFS== read key val; do ...`) instead of `source` if untrusted `.env` files are a concern. Low priority.
|
||||
|
||||
---
|
||||
|
||||
## Focus Area 4 - Error Handling & Robustness
|
||||
|
||||
### 4.1 Top-level `except Exception: pass` swallows data corruption (highest impact)
|
||||
|
||||
Seven top-level `try/except Exception: pass` blocks turn corrupt state into silent empty data:
|
||||
|
||||
| File | Lines | Consequence of swallowing |
|
||||
|---|---|---|
|
||||
| `lib.sh` (resolve_tmux_server) | 148-149 | Corrupt YAML -> silently returns `default` server -> stop/resume hit wrong tmux server |
|
||||
| `lib.sh` (find_workspace_uuid) | 610-611 | Corrupt DB -> empty session list -> UUID resolution falls through to global tiers (the P0-C bug the library exists to prevent) |
|
||||
| `lib.sh` (agent_identities) | 760-761 | Corrupt DB -> empty identities -> cache tier skipped silently |
|
||||
| `stop_session.sh` | 112-113 | Corrupt state -> `MAPPED_DATA` empty -> stop proceeds with no target, wrong behavior |
|
||||
| `status.sh` | 61-62 | Corrupt state -> prints "(no sessions registered)" instead of an error |
|
||||
| `update_yaml_resumed.sh` | 84-85 | Corrupt state -> `DELEGATE_JOB_ID` empty silently |
|
||||
| `reconcile.sh` | 320-321 | Corrupt state -> empty `d` -> drift report shows no drift on a corrupted file |
|
||||
|
||||
**Diagnosis**: These are the same pattern the brief calls "silent failures ... stderr swallowed without proper diagnostics." A corrupt `agent-sessions.yaml` or `.db` is a serious operational condition, but the code treats it identically to "file doesn't exist yet" (the legitimate empty case). There's no way for an operator to distinguish "fresh install" from "broken state."
|
||||
|
||||
**Proposed fix**: Distinguish "absent" from "broken":
|
||||
|
||||
```python
|
||||
try:
|
||||
if os.path.exists(db_path):
|
||||
conn = sqlite3.connect(db_path, timeout=60.0)
|
||||
row = conn.execute('SELECT data FROM state WHERE id=1').fetchone()
|
||||
if row: d = json.loads(row[0])
|
||||
conn.close()
|
||||
elif os.path.exists(yaml_path):
|
||||
with open(yaml_path) as f: d = yaml.safe_load(f) or {}
|
||||
except (sqlite3.DatabaseError, yaml.YAMLError, json.JSONDecodeError) as e:
|
||||
# State exists but is unreadable - this is an error, not "fresh."
|
||||
print(f"ERROR: state at {yaml_path} is corrupt: {e}", file=sys.stderr, flush=True)
|
||||
raise SystemExit(1)
|
||||
```
|
||||
|
||||
The `elif` (file absent) path stays silent (legitimate first-run). Only the "exists but unreadable" path becomes loud. Apply uniformly via the `load_state_json` helper from 2.2 so the policy lives in one place.
|
||||
|
||||
### 4.2 `export TMUX_SERVER_NAME="$(resolve_tmux_server ...)"` masks failure (SC2155)
|
||||
|
||||
`stop_session.sh:71` and `update_yaml_resumed.sh:36`:
|
||||
|
||||
```bash
|
||||
export TMUX_SERVER_NAME="$(resolve_tmux_server "$SESSION_NAME")"
|
||||
```
|
||||
|
||||
ShellCheck SC2155: the command substitution's exit code is masked by `export`. If `resolve_tmux_server` ever exits non-zero (today it always exits 0 via the fallback at lib.sh:151, but a future change could break that), the error is lost.
|
||||
|
||||
**Proposed fix**:
|
||||
|
||||
```bash
|
||||
TMUX_SERVER_NAME="$(resolve_tmux_server "$SESSION_NAME")" || exit $?
|
||||
export TMUX_SERVER_NAME
|
||||
```
|
||||
|
||||
### 4.3 `lib.sh:1030-1033` `start_watchdog` - `cd` without `|| return` (SC2164)
|
||||
|
||||
```bash
|
||||
local orig_pwd="$PWD"
|
||||
cd "$workdir"
|
||||
nohup bash "$monitor_script" --subscribe --idle-timeout 0 >> "$log_file" 2>&1 &
|
||||
pid=$!
|
||||
cd "$orig_pwd"
|
||||
```
|
||||
|
||||
If `cd "$workdir"` fails (dir removed between the `[ -f "$monitor_script" ]` check and here), `nohup` runs in `$PWD` with a relative `monitor_script`/`log_file`, and `cd "$orig_pwd"` also runs unconditionally.
|
||||
|
||||
**Proposed fix**:
|
||||
|
||||
```bash
|
||||
cd "$workdir" || { echo "ERROR: workdir vanished: $workdir" >&2; return 1; }
|
||||
nohup bash "$monitor_script" --subscribe --idle-timeout 0 >> "$log_file" 2>&1 &
|
||||
pid=$!
|
||||
cd "$orig_pwd" || true
|
||||
```
|
||||
|
||||
### 4.4 `reconcile.sh:56 / 262` - broad `set +e` window
|
||||
|
||||
```bash
|
||||
set +e
|
||||
YAML_PATH=... "$PYBIN" - <<'PYEOF'
|
||||
... ~180 lines of Python ...
|
||||
PYEOF
|
||||
sub_rc=$?
|
||||
set -e
|
||||
```
|
||||
|
||||
The `set +e` covers the entire Python heredoc plus the polling fallback (lines 263-277). This is necessary because the Python intentionally `sys.exit(3)` to signal "broker unavailable," but the window is large: any shell error in the surrounding scaffolding (env var expansion, the fallback `_self` loop) is non-fatal during that window.
|
||||
|
||||
**Diagnosis**: Today the only command between `set +e` and `set -e` that can fail non-Python is the fallback polling loop, which already has `|| true`. The risk is low but the pattern is fragile - a future edit that adds a shell command inside the `set +e` span will silently ignore failures.
|
||||
|
||||
**Proposed fix**: Narrow the `set +e` to just the Python invocation (it already is - line 262 restores before the fallback). Action item: add a comment making the invariant explicit, and don't let future edits extend the `set +e` span.
|
||||
|
||||
### 4.5 `delegate_publish_event` - fully silent on broker outage (by design, but unlogged)
|
||||
|
||||
```bash
|
||||
delegate_publish_event() {
|
||||
...
|
||||
"$py_bin" "$pub" --job "$job_id" --event "$event" --detail "$detail" || true
|
||||
}
|
||||
```
|
||||
|
||||
The `|| true` makes every publish failure non-fatal (correct per the contract: a delegate event must never abort create/stop/resume). But there's no diagnostic path: a broker outage during a stop means `stopped` is never published, the job's lifecycle event is missing, and nothing logs this.
|
||||
|
||||
**Proposed fix**: Log to stderr without affecting exit code:
|
||||
|
||||
```bash
|
||||
"$py_bin" "$pub" --job "$job_id" --event "$event" --detail "$detail" \\
|
||||
|| echo "WARN: delegate event '$event' for job $job_id failed (broker down?)" >&2
|
||||
```
|
||||
|
||||
Keeps the non-fatal contract, gives operators a signal.
|
||||
|
||||
### 4.6 `reconcile.sh` `handle_terminal` - unvalidated MQTT job_id passed via env
|
||||
|
||||
```python
|
||||
jid = payload.get("job_id") # from untrusted MQTT payload
|
||||
event = payload.get("event")
|
||||
...
|
||||
env['MQTT_JID'] = jid
|
||||
env['MQTT_EVENT'] = event
|
||||
cmd = ['bash', '-c', 'source "$LIB_SH"; atomic_dump_yaml "$YAML_PATH" MQTT_JID=...']
|
||||
```
|
||||
|
||||
HMAC is verified (line 173-176, good - addresses FUTURE_WORKS FW-P7). The values reach `atomic_dump_yaml`'s Python via environment variables, not string interpolation (correct - lib.sh comment "P1-B" calls this out). But `jid`/`event` are not length- or charset-validated before being set as env vars. A malicious-but-HMAC-valid payload (requires the auth token) with a multi-MB `job_id` could exhaust environment space or cause odd `os.environ['MQTT_JID']` behavior.
|
||||
|
||||
**Diagnosis**: Low risk (requires the HMAC token to pass), but the brief's robustness focus applies. Defense-in-depth only.
|
||||
|
||||
**Proposed fix**: After HMAC verification, validate format:
|
||||
|
||||
```python
|
||||
if not (isinstance(jid, str) and len(jid) <= 128 and jid.isascii()):
|
||||
print(f"MQTT Monitor: drop event: invalid job_id format", flush=True); return
|
||||
if event not in ("completed", "error"):
|
||||
print(f"MQTT Monitor: drop event: invalid event {event!r}", flush=True); return
|
||||
```
|
||||
|
||||
### 4.7 `except Exception` counts by file (for prioritization)
|
||||
|
||||
| File | `except Exception: pass` (fully silent) | `except ... as e: print(...)` (logged) |
|
||||
|---|---|---|
|
||||
| `reconcile.sh` | 5 (lines 257, 314, 320, 525, 550, 585, 613) | 3 (205, 229, 256) |
|
||||
| `lib.sh` | 5 (128, 148, 332, 448, 463) | 0 |
|
||||
| `stop_session.sh` | 2 (103, 112) | 1 (324) |
|
||||
| `status.sh` | 3 (55, 61, 108) | 0 |
|
||||
| `update_yaml_resumed.sh` | 1 (84) | 0 |
|
||||
|
||||
The `sqlite3.OperationalError: pass` cases (missing `sessions` table on older DBs) are legitimate - the table was added in a migration. Those should stay silent. The top-level `except Exception: pass` cases (4.1 above) are the ones to fix.
|
||||
|
||||
---
|
||||
|
||||
## Prioritized Action List
|
||||
|
||||
| ID | Finding | Area | Severity | Effort |
|
||||
|---|---|---|---|---|
|
||||
| **O-1** | `reconcile.sh:243-256` MQTT loop spins `time.sleep(0.5)` instead of `loop_forever`/Event | 1 | High | Medium |
|
||||
| **O-2** | Top-level `except Exception: pass` swallows corrupt state (7 sites) | 4 | High | Medium (fix once in `load_state_json`) |
|
||||
| **O-3** | `delegate-job:119,205` `sleep 1` subscriber-readiness race | 1 | Medium | Medium (needs subscriber readiness token) |
|
||||
| **O-4** | Three `tmux -L` reimplementations; `stop_session.sh` uses bare `tmux` w/o shim | 2 | Medium | Low (collapse to `_tmux_with_server`) |
|
||||
| **O-5** | DB+YAML load boilerplate duplicated 7x | 2 | Medium | Medium (extract `load_state_json`) |
|
||||
| **O-6** | `status.sh:29` 4-level `../` climb -> use `WORKSPACE_ROOT` from lib.sh | 3 | Medium | Low |
|
||||
| **O-7** | `stop_session.sh:193,200` fixed `sleep 3`/`sleep 5` -> bounded poll | 1 | Low | Low |
|
||||
| **O-8** | `*_exists` probes trapped in heredoc -> shared `lib.py` | 2 | Low | Medium |
|
||||
| **O-9** | `infer_agent_from_session` duplicated verbatim | 2 | Low | Trivial |
|
||||
| **O-10** | `_delegate_py_bin` vs `pick_python` divergent venv walks | 2 | Low | Low |
|
||||
| **O-11** | `export VAR="$(cmd)"` masks rc (SC2155) in stop/resume | 4 | Low | Trivial |
|
||||
| **O-12** | `start_watchdog` `cd` without `|| return` (SC2164) | 4 | Low | Trivial |
|
||||
| **O-13** | `delegate_publish_event` fully silent on broker outage | 4 | Low | Trivial |
|
||||
| **O-14** | `lib.sh:1174` magic `sleep 0.5` -> `_pane_quiescent` | 1 | Low | Low |
|
||||
| **O-15** | `BASH_SOURCE` unbound under zsh if sourced | 3 | Low (latent) | Low |
|
||||
| **O-16** | `wait_for_tui_ready` coarse 1 s poll + brace expansion | 1 | Low | Trivial |
|
||||
| **O-17** | `handle_terminal` unvalidated MQTT jid/event format | 4 | Low (defense-in-depth) | Trivial |
|
||||
|
||||
### Recommended sequencing
|
||||
|
||||
1. **O-2 + O-5 together** - extracting `load_state_json` is the vehicle for fixing the silent-corruption swallows. One helper, one error policy, seven call sites cleaned up. Highest ROI.
|
||||
2. **O-1** - standalone, removes the busiest poller in the codebase.
|
||||
3. **O-4 + O-9 + O-6** - small, mechanical de-duplication in `lib.sh`; fixes the latent `stop_session.sh` wrong-server bug as a side effect.
|
||||
4. **O-3** - requires a small subscriber-side change (readiness token), then an orchestrator-side wait. Test with a flaky-broker harness.
|
||||
5. The rest (O-7, O-8, O-10-O-17) are independent low-risk cleanups; bundle into one follow-up PR.
|
||||
|
||||
### Out of scope (noted, not actioned)
|
||||
|
||||
- FUTURE_WORKS FW-P6 (marker-file root lookup) would supersede O-6 and O-15; tracked separately.
|
||||
- FUTURE_WORKS FW-P7 (HMAC on termination) is already implemented (reconcile.sh:173); O-17 is the remaining defense-in-depth on top.
|
||||
- The `[[ ]]`, `local`, brace-expansion, and `read -d ''` bash-isms (3.3) are protected by the `#!/usr/bin/env bash` shebang and are not bugs under the current execution model. No change recommended unless a POSIX target is committed.
|
||||
|
||||
---
|
||||
|
||||
## Methodology / Verification
|
||||
|
||||
- All line numbers verified against `25de01e` via `grep -rn` and `sed -n` on the working tree.
|
||||
- ShellCheck run at `-S warning` over all 8 scripts; reported warnings: SC2155 (x2), SC2164 (x2), SC2034 (x2, `ONCE`/`EMIT_DIFF` - these are used by the `--once`/`--emit-diff` flags via the arg parser, so ShellCheck's "unused" is a false positive for the documented CLI surface).
|
||||
- Sleeps enumerated via `grep -rn '\bsleep\b'` across `--include='*.sh' --include='multi-agent-mux-delegate-job'` (18 shell sleeps + 3 Python `time.sleep`).
|
||||
- `except` handlers enumerated via `grep -rn -A1 'except.*:'`.
|
||||
- No code was modified; this is an analysis report only.
|
||||
+216
@@ -0,0 +1,216 @@
|
||||
# Report: Skill Optimization Implementation Review (Final)
|
||||
|
||||
- **Author**: Reviewer Cline (`canary-projects-multi-agent-mux-reviewer-cline`)
|
||||
- **Brief**: `.mam/reports/brief-rereview-skill-optimization.md`
|
||||
- **Plan reviewed against**: `.agents/reports/canary-projects-multi-agent-mux-planner-claude/report-skill-optimization-plan.md`
|
||||
- **Scope**: Phase 1 (OP-1, OP-2, OP-3) + Phase 2 (OP-4, OP-5, OP-6) working-tree changes
|
||||
- **Output path**: `.agents/reports/canary-projects-multi-agent-mux-reviewer-cline/report-skill-optimization-review-final.md`
|
||||
|
||||
## Verdict
|
||||
|
||||
# ✅ PASS
|
||||
|
||||
All six focus-area items named in the brief (OP-1, OP-2, OP-3, OP-4, OP-6, OP-7) are correctly implemented in the working tree. `bash -n` passes on all four changed shell scripts; `python3 -m py_compile` passes on `job_subscriber.py`; no new `shellcheck` warnings were introduced (all reported SC2155/SC2164/SC2034 are pre-existing). Two non-blocking observations are noted below; one out-of-scope item (OP-5) is flagged for awareness.
|
||||
|
||||
---
|
||||
|
||||
## Validation Commands (run inside this session)
|
||||
|
||||
| Check | Command | Result |
|
||||
|---|---|---|
|
||||
| Syntax (shell) | `bash -n lib.sh stop_session.sh multi-agent-mux-delegate-job reconcile.sh` | All 4 OK |
|
||||
| Syntax (python) | `python3 -m py_compile job_subscriber.py` | OK |
|
||||
| Lint | `shellcheck -S warning` on the 4 changed scripts | No new warnings (all pre-existing) |
|
||||
|
||||
Reported shellcheck warnings (all pre-existing, unchanged by this diff): `lib.sh:1043,1046` SC2164 (`cd` in `start_watchdog`), `stop_session.sh:71` SC2155 (`export TMUX_SERVER_NAME=$(...)`), `reconcile.sh:33,34` SC2034 (`ONCE`/`EMIT_DIFF` — false positives, used by the `--once`/`--emit-diff` arg parser).
|
||||
|
||||
---
|
||||
|
||||
## Per-Item Review
|
||||
|
||||
### OP-1 - Reactive Tmux Graceful Stopping (`stop_session.sh`) — PASS
|
||||
|
||||
**Plan**: Implement `_wait_session_gone` in `lib.sh` polling `tmux has-session` at ~250 ms up to a deadline; call it from `stop_session.sh` replacing the fixed `sleep 3` / `sleep 5`.
|
||||
|
||||
**Implementation** (`lib.sh:1128-1136`):
|
||||
```bash
|
||||
_wait_session_gone() {
|
||||
local sess="$1" max="${2:-5}" i
|
||||
for ((i = 0; i < max * 4; i++)); do
|
||||
_sks_tmux has-session -t "$sess" 2>/dev/null || return 0
|
||||
sleep 0.25
|
||||
done
|
||||
return 1
|
||||
}
|
||||
```
|
||||
Call sites (`stop_session.sh:193,200`):
|
||||
```bash
|
||||
_wait_session_gone "$SESSION_NAME" 5 || true
|
||||
...
|
||||
tmux kill-session -t "$SESSION_NAME" 2>/dev/null || true
|
||||
_wait_session_gone "$SESSION_NAME" 8 || true
|
||||
```
|
||||
|
||||
**Findings**:
|
||||
- Poll interval 0.25 s × `max*4` iterations = exactly `max` seconds deadline. Math is correct.
|
||||
- Uses `_sks_tmux` (server-aware), so it respects `TMUX_SERVER_NAME`. Correct.
|
||||
- `|| true` appended at both call sites is **essential**: `stop_session.sh` runs under `set -euo pipefail` (line 30), and `_wait_session_gone` returns 1 on timeout. Without `|| true`, a timeout would abort the whole graceful chain. The guard is present. ✅
|
||||
- Happy path returns early on first `has-session` failure (~0.25 s), matching the plan's "<0.3 s" outcome.
|
||||
|
||||
**Verdict**: Correctly implements the plan.
|
||||
|
||||
### OP-2 - MQTT Subscriber Event-Driven Handshake (`delegate-job`) — PASS (with minor observation)
|
||||
|
||||
**Plan**: `job_subscriber.py` writes `SUBSCRIBED <topic>` on SUBACK; the wrapper replaces `sleep 1` with a fast-poll loop matching the sentinel, with a liveness check.
|
||||
|
||||
**Implementation** — `job_subscriber.py:188-194`:
|
||||
```python
|
||||
def on_subscribe(_c, _u, mid, granted_qos, _props=None):
|
||||
for topic in subscribed_topics:
|
||||
print(f"SUBSCRIBED {topic}", flush=True)
|
||||
client.on_subscribe = on_subscribe
|
||||
```
|
||||
Wrapper (`multi-agent-mux-delegate-job:119-135`, mirrored at :218-234):
|
||||
```bash
|
||||
local sub_ready=0
|
||||
for ((i = 0; i < 25; i++)); do
|
||||
if kill -0 "$sub_pid" 2>/dev/null; then
|
||||
if grep -q '^SUBSCRIBED ' "$logf" 2>/dev/null; then
|
||||
sub_ready=1; break
|
||||
fi
|
||||
else
|
||||
echo "ERROR: subscriber died early (pid=$sub_pid, check $logf)" >&2; exit 1
|
||||
fi
|
||||
sleep 0.2
|
||||
done
|
||||
if [ "$sub_ready" -ne 1 ]; then
|
||||
echo "WARNING: subscriber subscribe handshake timed out — falling back to proceed" >&2
|
||||
fi
|
||||
```
|
||||
|
||||
**Findings**:
|
||||
- Sentinel `^SUBSCRIBED ` is written with `flush=True` so it is visible to the wrapper's `grep` immediately. ✅
|
||||
- `kill -0 "$sub_pid"` liveness check runs **before** the grep, so a dead subscriber is caught and exits 1 rather than waiting the full 5 s. ✅ Correct ordering.
|
||||
- 25 × 0.2 s = 5 s deadline; falls back gracefully with a WARNING (non-fatal) on timeout. ✅ Matches plan.
|
||||
- `on_subscribe` signature `(_c, _u, mid, granted_qos, _props=None)` covers both paho-mqtt v2 (MQTTv3, 5-arg) and v3 (MQTTv5, 6-arg with props) — the `_props=None` default absorbs the extra arg. ✅ Forward-compatible.
|
||||
- **Minor observation (non-blocking)**: `on_subscribe` iterates `subscribed_topics` and prints all topics on **every** suback. In the multi-job case, the sentinel fires after the **first** suback, not after all topics are subscribed. For the common single-job delegation this is exactly correct; for multi-job the wrapper proceeds slightly early. Since the orchestrator's own subscriber is the one that must be ready to receive the agent's `started` event — and that event is published to one job's topic — proceeding after the first suback is safe as long as the first-subscribed topic is the active job's. The registration order (`for job in jobs`) makes this hold. No fix required; noted for completeness.
|
||||
- Logic is correctly duplicated for both the `direct` path (:119) and the `loop/discuss` orchestrator path (:218). ✅
|
||||
|
||||
**Verdict**: Correctly implements the plan.
|
||||
|
||||
### OP-3 - Main Event Loop Pacing (`reconcile.sh`) — PASS (with minor observation)
|
||||
|
||||
**Plan**: Replace `while True: time.sleep(0.5)` with `threading.Event().wait(timeout=next_deadline - now)`.
|
||||
|
||||
**Implementation** (`reconcile.sh:242-266`):
|
||||
```python
|
||||
import threading
|
||||
stop_event = threading.Event()
|
||||
start = time.time()
|
||||
try:
|
||||
while True:
|
||||
now = time.time()
|
||||
wall_left = (timeout - (now - start)) if timeout else None
|
||||
idle_left = (idle_timeout - (now - state['last_msg'])) if idle_timeout else None
|
||||
next_timeout = 5.0
|
||||
if wall_left is not None:
|
||||
next_timeout = min(next_timeout, wall_left)
|
||||
if idle_left is not None:
|
||||
next_timeout = min(next_timeout, idle_left)
|
||||
if next_timeout <= 0:
|
||||
...print which deadline hit...
|
||||
break
|
||||
stop_event.wait(timeout=next_timeout)
|
||||
finally:
|
||||
client.loop_stop()
|
||||
```
|
||||
|
||||
**Findings**:
|
||||
- Computes `next_timeout = min(5.0, wall_left, idle_left)` so the main thread sleeps only as long as the nearest deadline, capped at 5 s — a strict improvement over the old fixed 0.5 s wake-up. ✅
|
||||
- Deadline-expired branch (`next_timeout <= 0`) correctly distinguishes wall vs idle timeout in its log message. ✅
|
||||
- **Minor observation (non-blocking)**: `stop_event` is never `set()` by any callback (e.g. `on_message`), so `stop_event.wait(timeout=...)` behaves identically to `time.sleep(timeout)` here. It is still an improvement because `Event.wait` is interruptible by signals (e.g. SIGINT) and is the idiomatic primitive for a condition-style sleep; but the full "event-driven wakeup on terminal event" benefit described in the plan would require `on_message` to call `stop_event.set()` when `event in ("completed","error")`. The current implementation is correct and an improvement; the optional enhancement (waking immediately on terminal event rather than on the next deadline tick) is left for a follow-up. Non-blocking.
|
||||
|
||||
**Verdict**: Correctly implements the plan (the reactive-sleep core).
|
||||
|
||||
### OP-4 - Unify Divergent Tmux Server Resolvers (`lib.sh`) — PASS
|
||||
|
||||
**Plan**: Extract a single canonical `mam_tmux()` dispatcher; have `_tmux`/`_sks_tmux` delegate to it.
|
||||
|
||||
**Implementation** (`lib.sh:97-113`):
|
||||
```bash
|
||||
mam_tmux() {
|
||||
_resolve_real_tmux_path
|
||||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||||
"$_REAL_TMUX_PATH" -L "$TMUX_SERVER_NAME" "$@"
|
||||
else
|
||||
"$_REAL_TMUX_PATH" "$@"
|
||||
fi
|
||||
}
|
||||
_tmux() { _init_tmux_isolation; mam_tmux "$@"; }
|
||||
tmux() { _tmux "$@"; }
|
||||
```
|
||||
`_sks_tmux` (`lib.sh:1118`) now reads: `_sks_tmux() { mam_tmux "$@"; }`.
|
||||
|
||||
**Findings**:
|
||||
- `mam_tmux` calls `"$_REAL_TMUX_PATH"` (the resolved real binary), **not** the shell function `tmux()` — so there is no recursion. ✅ (This was verified carefully; an earlier transient revision of the diff showed a literal `"tmux"` call which would have recursed, but the final working tree uses `$_REAL_TMUX_PATH`.)
|
||||
- `mam_tmux` calls `_resolve_real_tmux_path` directly (not the full `_init_tmux_isolation` PATH-shim setup), so it is safe to use from hot paths that don't want the shim side effect. `_tmux` still runs the full `_init_tmux_isolation` for callers that rely on the PATH shim. Correct separation of concerns. ✅
|
||||
- `_sks_tmux` previously had its own 4-line inline resolver with the SC2086-prone `tmux -L $TMUX_SERVER_NAME "$@"`; it now delegates to `mam_tmux`, removing the duplication. ✅
|
||||
- All four duplication sites named in the plan are consolidated: `_tmux`, `_sks_tmux`, and the inline `local_tmux` in `wait_for_tui_ready` (which already used `_sks_tmux`/`_tmux`). The `create_session.sh:212` `local_tmux` string is for the human-readable `START_CMD` YAML field, not a live call — left as-is, correctly.
|
||||
|
||||
**Verdict**: Correctly implements the plan; no recursion hazard.
|
||||
|
||||
### OP-6 - Consolidate TUI Ready / Dialog Tokens (`lib.sh`) — PASS
|
||||
|
||||
**Plan**: Declare `_MAM_DIALOG_TOKENS` and `_MAM_READY_TOKENS_CLAUDE` at top of `lib.sh`; refer to them everywhere.
|
||||
|
||||
**Implementation** (`lib.sh:25-27`):
|
||||
```bash
|
||||
_MAM_DIALOG_TOKENS='Do you trust the files|Yes, proceed|No, exit|Allow this|Press Enter to continue|browser to authenticate|Use arrow keys|Esc to cancel'
|
||||
_MAM_READY_TOKENS_CLAUDE='Anthropic|Assistant|Chat|Welcome|projects'
|
||||
```
|
||||
References (`grep` confirms 3 use sites):
|
||||
- `lib.sh:1071` `wait_for_tui_ready` claude branch → `grep -E -q "$_MAM_READY_TOKENS_CLAUDE"`
|
||||
- `lib.sh:1158` `_pane_dialog_open` → `grep -Eq "$_MAM_DIALOG_TOKENS"`
|
||||
- `lib.sh:1224` `handle_startup_dialogs` → `grep -Eq "$_MAM_READY_TOKENS_CLAUDE"`
|
||||
|
||||
**Findings**:
|
||||
- Both constants are byte-identical to the previously-inlined literals. ✅ No behavioral drift.
|
||||
- All three former inline sites now reference the constants; no stray literal copies remain (`grep` for the raw token strings outside the constant declarations returns nothing). ✅
|
||||
- Quoting is consistent (`"$_MAM_DIALOG_TOKENS"` preserves the `|` alternation for `grep -E`). ✅
|
||||
|
||||
**Verdict**: Correctly implements the plan.
|
||||
|
||||
### OP-7 - Guard against Non-Bash Sourced Environments (`lib.sh`) — PASS
|
||||
|
||||
**Plan**: Add a zsh-aware fallback or an explicit exit message warning users not to source from a foreign shell.
|
||||
|
||||
**Implementation** (`lib.sh:17-20`):
|
||||
```bash
|
||||
if [ -z "${BASH_VERSION:-}" ]; then
|
||||
echo "ERROR: lib.sh must be executed/sourced from bash (foreign shell detected)" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
**Findings**:
|
||||
- The guard runs **before** any `BASH_SOURCE` use (line 21), so a zsh `source` exits cleanly with a diagnostic instead of silently resolving `BASH_SOURCE[0]` to empty and sourcing the wrong `lib.sh`. ✅
|
||||
- `return 1 2>/dev/null || exit 1` covers both cases: `return` works when sourced (function context), `exit` works when executed directly. ✅
|
||||
- This implements the "explicit exit message" option from the plan. The fuller zsh `${(%):-%x}` fallback was the alternative; the chosen approach is simpler and aligns with AGENTS.md "Simplicity First". ✅
|
||||
|
||||
**Verdict**: Correctly implements the plan.
|
||||
|
||||
---
|
||||
|
||||
## Out-of-Scope / Awareness
|
||||
|
||||
- **OP-5 (Single-Source YAML/SQLite Load Boilerplate)** — listed under Phase 2 in the plan, but **not in the brief's focus-area list** and **not implemented** in this working-tree diff. The 7 duplicated load blocks remain. This is consistent with the brief's scoped focus (OP-1,2,3,4,6,7) but is called out so Phase 2 completion is not mis-reported. Recommend a follow-up PR for OP-5.
|
||||
- **OP-8 (Reconcile Observability)** — Phase 3 item, not in the brief's scope, not implemented. The fallback polling loop at `reconcile.sh:269` still uses `bash "$_self" --once --emit-diff >/dev/null 2>&1 || true`. Recommend a follow-up.
|
||||
|
||||
---
|
||||
|
||||
## Non-Blocking Observations (no action required for PASS)
|
||||
|
||||
1. **OP-2 multi-job sentinel**: `on_subscribe` prints all topics on each suback; the wrapper proceeds after the first suback. Safe for single-job (the common path) and for multi-job given registration order, but a future multi-job refactor should gate on "all topics subacked" (e.g. count subacks vs `len(subscribed_topics)`).
|
||||
2. **OP-3 unused `stop_event.set()`**: `stop_event` is never set, so it functions as an interruptible `time.sleep`. Adding `stop_event.set()` in `on_message` on terminal events would let the loop wake immediately on job completion instead of on the next deadline tick — an optional latency improvement, not a correctness issue.
|
||||
|
||||
Both observations are enhancements, not defects; neither blocks the PASS verdict.
|
||||
+33
-14
@@ -14,10 +14,18 @@
|
||||
# HARD RULE: the agent-sessions.yaml file is only ever written through
|
||||
# atomic_dump_yaml. Never `open(yaml_path, 'w')` anywhere else.
|
||||
|
||||
if [ -z "${BASH_VERSION:-}" ]; then
|
||||
echo "ERROR: lib.sh must be executed/sourced from bash (foreign shell detected)" >&2
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
SKILL_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
WORKSPACE_ROOT="$(cd "$SKILL_DIR/../.." && pwd)"
|
||||
AGENT_SESSIONS_YAML="${AGENT_SESSIONS_YAML:-$WORKSPACE_ROOT/.mam/agent-sessions.yaml}"
|
||||
|
||||
# Central TUI dialog and readiness validation tokens (OP-6)
|
||||
_MAM_DIALOG_TOKENS='Do you trust the files|Yes, proceed|No, exit|Allow this|Press Enter to continue|browser to authenticate|Use arrow keys|Esc to cancel'
|
||||
_MAM_READY_TOKENS_CLAUDE='Anthropic|Assistant|Chat|Welcome|projects'
|
||||
|
||||
# Workspace-relative defaults with environment overrides (Phase Z)
|
||||
HOME_DIR="${HOME_DIR:-$HOME}"
|
||||
CLAUDE_PROJECT_DIR="${CLAUDE_PROJECT_DIR:-$HOME/.claude/projects}"
|
||||
@@ -86,13 +94,18 @@ EOF
|
||||
fi
|
||||
}
|
||||
|
||||
mam_tmux() {
|
||||
_resolve_real_tmux_path
|
||||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||||
"$_REAL_TMUX_PATH" -L "$TMUX_SERVER_NAME" "$@"
|
||||
else
|
||||
"$_REAL_TMUX_PATH" "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
_tmux() {
|
||||
_init_tmux_isolation
|
||||
if [ -z "${TMUX_SERVER_NAME:-}" ] || [ "$TMUX_SERVER_NAME" = "default" ]; then
|
||||
"$_REAL_TMUX_PATH" "$@"
|
||||
else
|
||||
"$_REAL_TMUX_PATH" -L "$TMUX_SERVER_NAME" "$@"
|
||||
fi
|
||||
mam_tmux "$@"
|
||||
}
|
||||
|
||||
tmux() {
|
||||
@@ -1055,7 +1068,7 @@ wait_for_tui_ready() {
|
||||
if [ -n "$content" ]; then
|
||||
case "$agent" in
|
||||
claude)
|
||||
if echo "$content" | grep -E -q "Anthropic|Assistant|Chat|Welcome|projects" 2>/dev/null; then
|
||||
if echo "$content" | grep -E -q "$_MAM_READY_TOKENS_CLAUDE" 2>/dev/null; then
|
||||
echo "✅ Claude TUI detected ready."
|
||||
return 0
|
||||
fi
|
||||
@@ -1103,11 +1116,7 @@ inject_instructions() {
|
||||
|
||||
# Server-aware tmux (same isolation rule as inject_instructions).
|
||||
_sks_tmux() {
|
||||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||||
tmux -L "$TMUX_SERVER_NAME" "$@"
|
||||
else
|
||||
tmux "$@"
|
||||
fi
|
||||
mam_tmux "$@"
|
||||
}
|
||||
|
||||
_pane_capture() { _sks_tmux capture-pane -p -t "$1" 2>/dev/null || echo ""; }
|
||||
@@ -1116,6 +1125,17 @@ _pane_capture() { _sks_tmux capture-pane -p -t "$1" 2>/dev/null || echo ""; }
|
||||
# blank rows; window on non-blank lines or top-anchored dialogs are invisible.
|
||||
_pane_tail() { _pane_capture "$1" | grep -v '^[[:space:]]*$' | tail -n "${2:-20}"; }
|
||||
|
||||
# _wait_session_gone <sess> [max_sec=5]
|
||||
# Reactive loop to wait until a tmux session goes away (happy path returns early).
|
||||
_wait_session_gone() {
|
||||
local sess="$1" max="${2:-5}" i
|
||||
for ((i = 0; i < max * 4; i++)); do
|
||||
_sks_tmux has-session -t "$sess" 2>/dev/null || return 0
|
||||
sleep 0.25
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# _pane_quiescent <sess> [tries=20] [interval=0.5]
|
||||
# Renderer settled = two consecutive identical non-empty captures.
|
||||
# Defeats RC-A (Blessed/Ink renderer bottleneck) without a magic fixed sleep.
|
||||
@@ -1135,8 +1155,7 @@ _pane_quiescent() {
|
||||
# only (dialogs render near the input area; conversation text above must not
|
||||
# trigger this). Tokens must NOT appear on normal idle prompt screens.
|
||||
_pane_dialog_open() {
|
||||
_pane_tail "$1" 20 | grep -Eq \
|
||||
'Do you trust the files|Yes, proceed|No, exit|Allow this|Press Enter to continue|browser to authenticate|Use arrow keys|Esc to cancel'
|
||||
_pane_tail "$1" 20 | grep -Eq "$_MAM_DIALOG_TOKENS"
|
||||
}
|
||||
|
||||
# send_keys_safe <sess> <text> [job_id]
|
||||
@@ -1202,7 +1221,7 @@ handle_startup_dialogs() {
|
||||
_sks_tmux send-keys -t "$sess" Down
|
||||
sleep 0.3
|
||||
_sks_tmux send-keys -t "$sess" Enter
|
||||
elif printf '%s\n' "$pane" | grep -Eq 'Anthropic|Assistant|Chat|Welcome|projects'; then
|
||||
elif printf '%s\n' "$pane" | grep -Eq "$_MAM_READY_TOKENS_CLAUDE"; then
|
||||
return 0
|
||||
fi
|
||||
sleep 2
|
||||
|
||||
@@ -116,7 +116,23 @@ cmd_submit() {
|
||||
>"$logf" 2>&1 &
|
||||
local sub_pid=$!
|
||||
echo "subscriber pid: $sub_pid (log: $logf)"
|
||||
sleep 1 # give the subscriber time to CONNACK + SUBSCRIBE before the agent runs
|
||||
# Wait for the subscriber to CONNACK + SUBSCRIBE (reactive handshake)
|
||||
local sub_ready=0
|
||||
for ((i = 0; i < 25; i++)); do
|
||||
if kill -0 "$sub_pid" 2>/dev/null; then
|
||||
if grep -q '^SUBSCRIBED ' "$logf" 2>/dev/null; then
|
||||
sub_ready=1
|
||||
break
|
||||
fi
|
||||
else
|
||||
echo "ERROR: subscriber died early (pid=$sub_pid, check $logf)" >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 0.2
|
||||
done
|
||||
if [ "$sub_ready" -ne 1 ]; then
|
||||
echo "WARNING: subscriber subscribe handshake timed out — falling back to proceed" >&2
|
||||
fi
|
||||
|
||||
# 3) run the agent (or print the command for dry-run / missing binary)
|
||||
local pub="$PY $SCRIPT_DIR/scripts/publish_event.py --registry-dir $REGISTRY_DIR --job $JOB_ID"
|
||||
@@ -202,7 +218,23 @@ Task: $PROMPT"
|
||||
>"$logf" 2>&1 &
|
||||
local sub_pid=$!
|
||||
echo "subscriber pid: $sub_pid (log: $logf)"
|
||||
sleep 1
|
||||
# Wait for the subscriber to CONNACK + SUBSCRIBE (reactive handshake)
|
||||
local sub_ready=0
|
||||
for ((i = 0; i < 25; i++)); do
|
||||
if kill -0 "$sub_pid" 2>/dev/null; then
|
||||
if grep -q '^SUBSCRIBED ' "$logf" 2>/dev/null; then
|
||||
sub_ready=1
|
||||
break
|
||||
fi
|
||||
else
|
||||
echo "ERROR: subscriber died early (pid=$sub_pid, check $logf)" >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 0.2
|
||||
done
|
||||
if [ "$sub_ready" -ne 1 ]; then
|
||||
echo "WARNING: subscriber subscribe handshake timed out — falling back to proceed" >&2
|
||||
fi
|
||||
|
||||
# Format instruction block
|
||||
local pub="$PY $SCRIPT_DIR/scripts/publish_event.py --registry-dir $REGISTRY_DIR --job $JOB_ID"
|
||||
|
||||
@@ -185,8 +185,13 @@ def main(argv=None) -> int:
|
||||
if rc != 0:
|
||||
logger.warning("broker disconnected (rc=%s); will retry reconnect", reason_code)
|
||||
|
||||
def on_subscribe(_c, _u, mid, granted_qos, _props=None):
|
||||
for topic in subscribed_topics:
|
||||
print(f"SUBSCRIBED {topic}", flush=True)
|
||||
|
||||
client.on_connect = on_connect
|
||||
client.on_disconnect = on_disconnect
|
||||
client.on_subscribe = on_subscribe
|
||||
client.reconnect_delay_set(min_delay=1, max_delay=16)
|
||||
mqtt_common.with_retry(
|
||||
lambda: client.connect(config.host, config.port, config.keepalive),
|
||||
|
||||
@@ -239,17 +239,29 @@ if not state['connected']:
|
||||
client.loop_stop()
|
||||
sys.exit(3)
|
||||
|
||||
import threading
|
||||
stop_event = threading.Event()
|
||||
start = time.time()
|
||||
try:
|
||||
while True:
|
||||
now = time.time()
|
||||
if timeout and (now - start) >= timeout:
|
||||
wall_left = (timeout - (now - start)) if timeout else None
|
||||
idle_left = (idle_timeout - (now - state['last_msg'])) if idle_timeout else None
|
||||
|
||||
next_timeout = 5.0
|
||||
if wall_left is not None:
|
||||
next_timeout = min(next_timeout, wall_left)
|
||||
if idle_left is not None:
|
||||
next_timeout = min(next_timeout, idle_left)
|
||||
|
||||
if next_timeout <= 0:
|
||||
if wall_left is not None and wall_left <= 0:
|
||||
print(f"MQTT Monitor: --timeout {timeout}s reached, exiting", flush=True)
|
||||
break
|
||||
if idle_timeout and (now - state['last_msg']) >= idle_timeout:
|
||||
else:
|
||||
print(f"MQTT Monitor: --idle-timeout {idle_timeout}s reached, exiting", flush=True)
|
||||
break
|
||||
time.sleep(0.5)
|
||||
|
||||
stop_event.wait(timeout=next_timeout)
|
||||
finally:
|
||||
client.loop_stop()
|
||||
try:
|
||||
|
||||
@@ -190,14 +190,14 @@ graceful_stop() {
|
||||
esac
|
||||
echo "graceful: send-keys '$exitkey' to $SESSION_NAME"
|
||||
send_keys_safe "$SESSION_NAME" "$exitkey" "stop$$" || echo "graceful: safe delivery failed (rc=$?) — falling back to kill chain"
|
||||
sleep 3
|
||||
_wait_session_gone "$SESSION_NAME" 5 || true
|
||||
if ! tmux has-session -t "$SESSION_NAME" 2>/dev/null; then
|
||||
echo "graceful: exited cleanly"
|
||||
return 0
|
||||
fi
|
||||
echo "graceful: still alive → kill-session (SIGTERM)"
|
||||
tmux kill-session -t "$SESSION_NAME" 2>/dev/null || true
|
||||
sleep 5
|
||||
_wait_session_gone "$SESSION_NAME" 8 || true
|
||||
if ! tmux has-session -t "$SESSION_NAME" 2>/dev/null; then
|
||||
echo "graceful: terminated after kill-session"
|
||||
return 0
|
||||
|
||||
Reference in New Issue
Block a user