- Wrap inject_instructions with safe || rc=0 under set -e to prevent premature exit - Ensure terminal error events are published on instruction injection failure - Update 2-reviewers final review reports with Round 3 PASS verdicts
214 lines
14 KiB
Markdown
214 lines
14 KiB
Markdown
# Prompt-Lock & Input Delivery Failure — Code-Level Analysis Report
|
||
|
||
- **Reviewer**: Reviewer Cline (`canary-projects-multi-agent-mux-reviewer-cline`, role: reviewer)
|
||
- **Date**: 2026-07-11
|
||
- **Brief**: `.mam/reports/brief-prompt-lock-fix.md`
|
||
- **Scope**: Code-level analysis of the "prompt typing lock / input delivery failure" in the TMUX-based multi-agent environment; concrete prevention helper + migration plan.
|
||
|
||
---
|
||
|
||
## 1. The Problem (recap)
|
||
|
||
When the orchestrator issues commands via `tmux send-keys` to a target agent TUI:
|
||
1. Text gets printed inside the prompt input box but is **never submitted** (Enter ignored) or the cursor freezes.
|
||
2. Root causes: blessed UI renderer thread bottleneck during heavy output; dialog popups (Approve/Reject permission prompts) stealing input focus; OAuth/list-selection dialog blocks intercepting keystrokes.
|
||
|
||
The core failure mode is: **`send-keys` delivers keystrokes to whatever currently has focus.** If a permission dialog, an OAuth browser-prompt, or a list-selection popup is open, the keystrokes go to the dialog (or are swallowed), not the main input box — so the prompt text appears but Enter does nothing, or the cursor appears frozen.
|
||
|
||
---
|
||
|
||
## 2. Exact Code Locations — Every `send-keys` / Input-Delivery Site
|
||
|
||
I grepped the entire `.agents/` tree. There are **6 input-delivery sites**; only 2 use a helper, the rest are raw `tmux send-keys`.
|
||
|
||
### Site A — `lib.sh:1086-1100` (`inject_instructions`) — HELPER, central
|
||
```bash
|
||
inject_instructions() {
|
||
local sess="$1" instructions="$2" job_id="${3:-onboard}"
|
||
local local_tmux="tmux"
|
||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||
local_tmux="tmux -L $TMUX_SERVER_NAME"
|
||
fi
|
||
$local_tmux set-buffer -b "job_buf_$job_id" "$instructions"
|
||
$local_tmux paste-buffer -b "job_buf_$job_id" -t "$sess"
|
||
sleep 0.5
|
||
$local_tmux send-keys -t "$sess" C-m # ← Enter, no focus guard
|
||
$local_tmux delete-buffer -b "job_buf_$job_id"
|
||
}
|
||
```
|
||
**Vulnerability**: No focus recovery. If a permission/dialog popup is open when `C-m` fires, Enter goes to the dialog. The fixed `sleep 0.5` is too short under heavy renderer load (the brief's "renderer thread bottleneck"). No delivery verification.
|
||
|
||
### Site B — `multi-agent-mux-delegate-job:364-369` — DUPLICATE of Site A, raw
|
||
```bash
|
||
$_tmux set-buffer -b "job_buf_$job_id" "$instructions"
|
||
$_tmux paste-buffer -b "job_buf_$job_id" -t "$sess"
|
||
sleep 0.5
|
||
$_tmux send-keys -t "$sess" C-m
|
||
$_tmux delete-buffer -b "job_buf_$job_id"
|
||
```
|
||
**Vulnerability**: Identical logic to Site A, copy-pasted (violates lib.sh's "single source of truth" mandate, lib.sh header §4.1). Same no-focus-guard + too-short-sleep defects. This is the delegate-job path — the *primary* way the orchestrator hands work to agents, so it's the highest-traffic vulnerable site.
|
||
|
||
### Site C — `resume/SKILL.md:150-156` — RAW, dialog auto-handle (MOST VULNERABLE)
|
||
```bash
|
||
# auto-handle trust / bypass dialogs
|
||
sleep 5
|
||
tmux send-keys -t "$SESSION_NAME" Enter 2>/dev/null || true
|
||
sleep 3
|
||
tmux send-keys -t "$SESSION_NAME" Down 2>/dev/null || true
|
||
sleep 0.3
|
||
tmux send-keys -t "$SESSION_NAME" Enter 2>/dev/null || true
|
||
```
|
||
**Vulnerability**: This is the *exact* lock symptom from the brief. It fires blind `Enter`/`Down`/`Enter` on fixed sleeps to auto-dismiss a trust dialog. Problems: (a) no check that a dialog actually exists — if the TUI rendered late and focus is still the main input, these keystrokes type garbage into the prompt; (b) if a *different* dialog (OAuth, list-select) appeared instead of the expected trust prompt, `Down`+`Enter` selects the wrong option; (c) `2>/dev/null || true` swallows all errors silently — the operator never learns delivery failed; (d) no `wait_for_tui_ready` gate before sending.
|
||
|
||
### Site D — `stop_session.sh:192` (`graceful_stop`) — RAW
|
||
```bash
|
||
tmux send-keys -t "$SESSION_NAME" "$exitkey" Enter 2>/dev/null || true
|
||
```
|
||
**Vulnerability**: Sends `/exit` + Enter without focus recovery. If a permission dialog is open, `/exit` is typed into the dialog (harmless there) but Enter may dismiss the dialog with an unintended choice, and the agent never receives the exit command. The graceful chain *does* have a proper fallback (kill-session → SIGTERM → SIGKILL with `has-session` checks, lines 194-205), so this site is low-severity — but it still benefits from focus recovery.
|
||
|
||
### Site E — `create/SKILL.md:215` — RAW, probe (documentation example)
|
||
```bash
|
||
tmux send-keys -t "$SESSION_NAME" "" Enter
|
||
```
|
||
**Vulnerability**: This is in a verification snippet (sends empty + Enter). Low impact — it's an optional manual probe, not an automated path. But it sets a bad example for users.
|
||
|
||
### Site F — `create_session.sh:384` — USES HELPER (Site A)
|
||
```bash
|
||
inject_instructions "$SESSION_NAME" "$instructions" "$DELEGATE_JOB_ID"
|
||
```
|
||
**Status**: This is the correct pattern — it routes through `inject_instructions`. It's only as safe as Site A. Also note: `create_session.sh:199` calls `wait_for_tui_ready` before any input — the *only* site that does so.
|
||
|
||
### Existing mitigation already present (good, underused)
|
||
---
|
||
|
||
## 3. Proposed Prevention Helper: `send_keys_safe()`
|
||
|
||
Add to `lib.sh` immediately after `wait_for_tui_ready` (around line 1085). Design goals: (1) recover focus before every send, (2) verify delivery via capture-pane, (3) single source of truth replacing Sites A–E, (4) no new dependencies, (5) respect the existing `local_tmux` / `TMUX_SERVER_NAME` isolation pattern.
|
||
|
||
```bash
|
||
# send_keys_safe <session> <text> [--enter] [--no-focus-recovery] [--verify]
|
||
#
|
||
# Focus-safe, delivery-verified tmux send-keys. Restores input focus to the
|
||
# main prompt before sending, then (optionally) verifies the text reached the
|
||
# pane. Replaces raw `tmux send-keys` and the duplicated paste-buffer blocks
|
||
# across create/resume/stop/delegate-job to fix the "prompt lock" issue:
|
||
# keystrokes landing in a dialog popup instead of the main input box.
|
||
#
|
||
# Args:
|
||
# <session> target tmux session/pane
|
||
# <text> text to send (use "" for a bare Enter)
|
||
# --enter append C-m (Enter) after the text
|
||
# --no-focus-recovery skip the Escape/Ctrl-C focus-reset preamble (rare; only
|
||
# for sending into a known-open dialog on purpose)
|
||
# --verify capture-pane after send and confirm <text> is present
|
||
# (substring match, first line only); returns 1 on miss
|
||
# Environment:
|
||
# TMUX_SERVER_NAME honored (same isolation as inject_instructions)
|
||
# Returns: 0 on success, 1 on verify-fail or tmux error.
|
||
send_keys_safe() {
|
||
local sess="$1" text="$2"
|
||
shift 2
|
||
local do_enter=0 do_recover=1 do_verify=0
|
||
while [ $# -gt 0 ]; do
|
||
case "$1" in
|
||
--enter) do_enter=1 ;;
|
||
--no-focus-recovery) do_recover=0 ;;
|
||
--verify) do_verify=1 ;;
|
||
*) echo "send_keys_safe: unknown arg: $1" >&2; return 2 ;;
|
||
esac
|
||
shift
|
||
done
|
||
|
||
local local_tmux="tmux"
|
||
if [ -n "${TMUX_SERVER_NAME:-}" ] && [ "$TMUX_SERVER_NAME" != "default" ]; then
|
||
local_tmux="tmux -L $TMUX_SERVER_NAME"
|
||
fi
|
||
|
||
# --- 1. Focus recovery: dismiss any dialog / popup / list-select -------------
|
||
# Escape dismisses most permission & list-selection popups back to the prompt.
|
||
# C-c cancels a half-typed line / pending OAuth prompt that may hold focus.
|
||
# A short settle lets the blessed renderer re-render the main input box.
|
||
if [ "$do_recover" -eq 1 ]; then
|
||
$local_tmux send-keys -t "$sess" Escape 2>/dev/null || true
|
||
$local_tmux send-keys -t "$sess" C-c 2>/dev/null || true
|
||
sleep 0.3
|
||
fi
|
||
|
||
# --- 2. Deliver text via paste-buffer (atomic, no char-loss on long input) ----
|
||
local buf="sk_safe_$$"
|
||
$local_tmux set-buffer -b "$buf" -- "$text"
|
||
$local_tmux paste-buffer -b "$buf" -t "$sess"
|
||
$local_tmux delete-buffer -b "$buf"
|
||
sleep 0.4 # let the TUI ingest the paste before Enter / verify
|
||
|
||
# --- 3. Optional Enter -------------------------------------------------------
|
||
if [ "$do_enter" -eq 1 ]; then
|
||
$local_tmux send-keys -t "$sess" C-m
|
||
sleep 0.3
|
||
fi
|
||
|
||
# --- 4. Optional delivery verification ---------------------------------------
|
||
if [ "$do_verify" -eq 1 ] && [ -n "$text" ]; then
|
||
local got
|
||
got=$($local_tmux capture-pane -p -t "$sess" 2>/dev/null || echo "")
|
||
# match the first line of <text> against the pane (avoids wrapping noise)
|
||
local first_line
|
||
first_line="$(printf '%s\n' "$text" | head -n1 | sed 's/[][\\.^$*+?(){}|]/\\&/g')"
|
||
if [ -n "$first_line" ] && ! printf '%s' "$got" | grep -Fq -- "$first_line"; then
|
||
echo "send_keys_safe: delivery verify FAILED for session '$sess'" >&2
|
||
return 1
|
||
fi
|
||
fi
|
||
return 0
|
||
}
|
||
```
|
||
|
||
And refactor `inject_instructions` to delegate to it (keeps the existing call sites working):
|
||
|
||
```bash
|
||
inject_instructions() {
|
||
local sess="$1" instructions="$2" job_id="${3:-onboard}"
|
||
# Reuse the focus-safe helper; paste + Enter + verify.
|
||
send_keys_safe "$sess" "$instructions" --enter --verify
|
||
}
|
||
```
|
||
|
||
### Why this design fixes each root cause
|
||
| Brief root cause | How `send_keys_safe` addresses it |
|
||
|------------------|-----------------------------------|
|
||
| Blessed renderer thread bottleneck | `sleep 0.3` after focus-reset + `sleep 0.4` after paste give the renderer time to re-render the main input box before Enter; `--verify` detects a stuck renderer (text absent → return 1 → caller can retry). |
|
||
| Dialog popups stealing focus (Approve/Reject) | The `Escape` + `C-c` preamble dismisses/cancels the popup first, returning focus to the main prompt. |
|
||
| OAuth / list-selection dialog blocks intercepting keystrokes | `Escape` exits list-selects; `C-c` cancels OAuth prompts; if a dialog still holds focus, `--verify` fails and the caller learns instead of silently swallowing. |
|
||
| Silent failure (`2>/dev/null \|\| true`) | `--verify` makes delivery failure observable; raw sites currently swallow all errors. |
|
||
---
|
||
|
||
## 4. Draft Migration Plan
|
||
|
||
Order matters: introduce the helper first (no behavior change), then migrate sites one at a time (each verifiable). Per AGENTS.md §3 (Surgical Changes), each step touches only its own site.
|
||
|
||
| Step | File | Change | Verify |
|
||
|------|------|--------|--------|
|
||
| M1 | `lib.sh` (~line 1085) | Add `send_keys_safe()`; refactor `inject_instructions()` to call it. | `bash -n lib.sh`; existing `inject_instructions` callers (create_session.sh:384) still work — run a create + delegate-job and confirm the prompt is delivered and Enter submits. |
|
||
| M2 | `multi-agent-mux-delegate-job` lines 366-369 | Replace the 5-line paste-buffer block with `send_keys_safe "$sess" "$instructions" --enter --verify`. Removes the duplicate (lib.sh "single source of truth" mandate). | Delegate a job to an existing live session; confirm `--verify` passes and the agent receives the full prompt. |
|
||
| M3 | `resume/SKILL.md` lines 150-156 | Replace the blind `sleep 5; send-keys Enter; sleep 3; send-keys Down; sleep 0.3; send-keys Enter` block with: `wait_for_tui_ready "$SESSION_NAME" claude` first, then a single `send_keys_safe "$SESSION_NAME" "" --enter` to dismiss the trust dialog if present. Drop the hardcoded `Down` (it picks an option blindly). Add a capture-pane check: only send the dismiss Enter if a dialog keyword (e.g. `trust`, `approve`, `bypass`) is visible. | Resume a stopped claude session; confirm the trust dialog is dismissed and the prompt is responsive, *without* a stray `Down` corrupting a non-trust dialog. |
|
||
| M4 | `stop_session.sh` line 192 | Replace `tmux send-keys -t "$SESSION_NAME" "$exitkey" Enter` with `send_keys_safe "$SESSION_NAME" "$exitkey" --enter` (no `--verify` needed — the existing kill-session fallback chain already verifies exit). | Run `stop_session.sh --graceful`; confirm graceful exit still falls back to kill-session correctly. |
|
||
| M5 | `create/SKILL.md` line 215 | Update the documentation probe example to `send_keys_safe "$SESSION_NAME" "" --enter --verify` so the docs teach the safe pattern. | `bash -n` on any snippet; doc review. |
|
||
|
||
### Non-Goals (out of scope, per AGENTS.md §2)
|
||
- Not adding a generic dialog-state machine — the `Escape`/`C-c` preamble + `--verify` covers the 3 root causes without over-engineering.
|
||
- Not removing `2>/dev/null || true` from the focus-reset preamble (those keystrokes are best-effort by design; the *delivery* path uses `--verify`, which is the observable contract).
|
||
- Not touching `wait_for_tui_ready` itself — it's correct; M1/M3 just extend its usage to resume.
|
||
|
||
### Risk
|
||
| Risk | Severity | Mitigation |
|
||
|------|----------|-----------|
|
||
| `Escape`/`C-c` preamble cancels a legitimate in-flight user input | Medium | `--no-focus-recovery` escape hatch for intentional dialog sends; default path is automated orchestration where the pane is owned by the script, not a human. |
|
||
| `--verify` false-negative on wrapped/colored prompts | Low | `first_line` substring + `grep -F` is tolerant; worst case returns 1 and caller retries — safer than silent swallow. |
|
||
| `sleep 0.4` too short on slow renderers | Low | `--verify` is the real gate, not the sleep duration; sleep is a best-effort settle. |
|
||
|
||
---
|
||
|
||
## 5. Summary
|
||
|
||
The prompt-lock issue is caused by **6 input-delivery sites, only 2 of which use a helper, none of which recover focus or verify delivery.** The highest-risk site is `resume/SKILL.md:150-156` (blind dialog auto-dismiss). The fix is a single `send_keys_safe()` helper in `lib.sh` (focus recovery via `Escape`+`C-c`, paste-buffer delivery, optional `--verify`) that becomes the single source of truth, with `inject_instructions` refactored to delegate to it and the 4 raw sites (delegate-job, resume, stop, create-docs) migrated in 5 surgical steps. The existing `wait_for_tui_ready` primitive is reused and extended to resume. No new dependencies; no over-engineering; each migration step is independently verifiable.
|
||
`lib.sh:1040-1084` defines `wait_for_tui_ready <sess> <agent>` — a gated capture-pane loop (15×1s) that grep-checks the pane content for each agent's TUI banner before returning. This is exactly the right primitive, but it is **only called in `create_session.sh:199`**. Resume, stop, and delegate-job never gate on TUI readiness. |