# Cross-Code Review Report - Job e1c4e9c3

**Review Target**: Remote Docker deployment plan for `nats-server` (Track 1R)
**Scope**: `PRIVATE_SERVER.md`, `implementation_plan.md`, `.agents/skills/lib.sh`, `.mam.env.example`, `tests/test_deploy_freshness.py`
**Reviewer**: cline (herdr:canary-projects-multi-agent-mux-creator-cline)
**Date**: 2026-08-22
**Commit base**: c6b6c77 (HEAD)

---

## 1. Executive Summary

The changeset establishes a comprehensive remote `nats-server` Docker production deployment plan (Track 1R / M2b) across 5 files (+413 / -66 lines). It delivers all four task deliverables: production Docker Compose & nats.conf, networking/security guide, client config & verification playbooks, and a phased rollout roadmap. Seven new regression guards (D-15~D-21) lock the documentation invariants.

**Test results**: 297 tests collected (290 -> 297); deploy_freshness 20/20 pass; tier1+o2 67/67 pass; sanity 2/2 pass. No regressions detected in the fast subset.

**Verdict**: PASS. One Medium documentation inconsistency (section 9.2 production compose omits NATS 4222 while section 9.3 UFW and R-3 reference it) and several Low/Very Low findings - none require re-planning.

---

## 2. Changed Files Overview

| File | Delta | Purpose |
|---|---|---|
| `.agents/skills/lib.sh` | +8 / -5 | Claude startup dialog handling robustness in `wait_for_tui_ready` / `handle_startup_dialogs` |
| `.mam.env.example` | +4 | Document `MQTT_KEEPALIVE` env var |
| `PRIVATE_SERVER.md` | +239 / -36 | D-1~D-5 corrections, N-1 boundary, section 9 remote production guide, Appendix X |
| `implementation_plan.md` | +44 / -20 | Split M2 -> M2a/M2b, add Track 1R roadmap section 5, renumber sections |
| `tests/test_deploy_freshness.py` | +112 / -11 | D-11 cleanup (remove dead `recognized` set, add `MQTT_BIND`); add D-15~D-21 guards |

---

## 3. Detailed Review by File

### 3.1 `.agents/skills/lib.sh`

**Changes**:
1. `wait_for_tui_ready()` (line 1519): adds `handle_startup_dialogs "$sess" 1 || true` inside the 30-iteration loop for `$agent = "claude"` only.
2. `handle_startup_dialogs()` (line 1725): broadens trust-dialog regex to `'Do you trust the files|Yes, I trust this folder|Quick safety check'`.
3. (lines 1733-1735): adds a new `'Press Enter to continue'` branch; adds `${_MAM_READY_TOKENS_CLAUDE:-Anthropic|Assistant|Chat|Welcome}` fallback default.
4. (lines 1738-1739): reduces sleep from 2s->1s and `waited` increment from 2->1.

**Verification**:
- `bash -n` syntax check: PASS
- `_MAM_READY_TOKENS_CLAUDE` is defined at line 63 -> the `:-` fallback is defensive but harmless (consistent with prior N5 observation).
- The regex broadening correctly handles newer Claude dialog variants ("Quick safety check" appeared in recent Claude Code versions).

**Findings**:

**L-1 (Low) - Latency overhead in `wait_for_tui_ready`**: The new `handle_startup_dialogs "$sess" 1` call adds ~1s (one loop iteration with `sleep 1`) per `wait_for_tui_ready` iteration even when no dialog is present. Combined with the existing `sleep 1`, each of the 30 iterations now takes ~2s (max ~60s vs previous ~30s). Acceptable for TUI readiness but doubles worst-case latency. Not a blocker - the function returns early when ready tokens appear.

**L-2 (Low) - `handle_startup_dialogs` default timeout halved**: Changing `sleep 2; waited+=2` -> `sleep 1; waited+=1` halves the default timeout from ~40s to ~20s. When called with the default `timeout=20`, the function now runs at most ~20s instead of ~40s. This is reasonable for Claude dialogs (which appear within seconds) but reduces the safety margin for slow environments. The `wait_for_tui_ready` call uses `timeout=1` (1s), so it is unaffected by this change.

### 3.2 `.mam.env.example`

Adds `MQTT_KEEPALIVE=60` with a descriptive comment. Verified `mqtt_common.py:234` reads it via `_env_int("MQTT_KEEPALIVE", 60)` and the dataclass default is `keepalive: int = 60` (line 179). Consistent. PASS

### 3.3 `PRIVATE_SERVER.md`

**D-1 store_dir correction**: Changed from literal `"~/.local/share/nats/data"` (which does not expand in nats.conf) to `"/data"` (Docker) and `"$HOME/..."` (native, via unquoted `<<EOF` heredoc). Verified by D-15 guard.

**D-2 image pin**: `nats:latest` -> `nats:2.12-alpine`. The comment correctly notes `latest` is scratch-based (no `wget` for healthcheck). Alpine includes busybox `wget`. Verified by D-16 guard.

**D-3 port binding**: All ports now bind to `127.0.0.1` or `${*_BIND:-127.0.0.1}`. Port 8222 (unauthenticated monitoring) is hardcoded to `127.0.0.1`. Verified by D-17 guard.

**D-4 TLS examples**: TLS blocks use DNS domain names (`mam-broker.example.com`), not IP literals. Verified by D-18 guard.

**N-1 retained boundary**: Section 5.2 now explicitly documents that NATS/WebSocket subscribers joining after job termination will not receive retained MQTT terminal events, with two remediation paths (MQTT reconnect or JetStream opt-in).

**Section 9 Remote production guide**: Well-structured with:
- 9.1: Production nats.conf with multi-tenant accounts, `mam_observer` read-only user, JetStream, `ack_wait: 60s` for WAN, `max_ack_pending: 1024`.
- 9.2: Production compose with fail-closed env (`${VAR:?set in .env}`), healthcheck, log rotation.
- 9.3: Tailscale vs TLS comparison table, UFW rules, secret generation.
- 9.4: R-1~R-10 verification playbook + WAN latency probe.
- 9.5: 5-step cutover procedure.
- Appendix X: Account export/import for cross-trust-domain scenarios.

**Findings**:

**M-1 (Medium) - Section 9.2 production compose omits NATS 4222 port**: The section 9.2 `docker-compose.yml` (lines 405-408) publishes only ports 1883, 8222, 8080 - **missing `${NATS_BIND:-127.0.0.1}:4222:4222`**. This contradicts:
- The task brief which explicitly requires "NATS 4222" in the production compose.
- Section 9.3 UFW rule `sudo ufw allow in on tailscale0 to any port 4222 proto tcp` (line 447) - a dead rule since the container does not publish 4222 to the host.
- R-3 verification playbook (line 473) which nmap-tests 4222.

The section 4.1 *dev* compose (lines 101, 122) correctly includes 4222. The section 9.1 nats.conf enables NATS default port 4222 inside the container (nats-server listens on 4222 by default), but without the compose port mapping it is unreachable from the tailnet. For MAM-only deployments (MQTT 1883 only), 4222 is optional - but the UFW rule and R-3 test should then be updated to match, or the port should be added to section 9.2. **Fix**: Add `- "${NATS_BIND:-127.0.0.1}:4222:4222"` to section 9.2 ports, OR remove 4222 from section 9.3 UFW and R-3.

**V-1 (Very Low) - Misleading `store_dir` comment (line 73)**: `store_dir: "/data"` is annotated `# Docker ... (native execution $HOME expansion)` - but `/data` is a fixed absolute path that does NOT expand to `$HOME`. Native execution uses a separate config block (line 145, `"$HOME/.local/share/nats/data"`). The parenthetical comment is slightly misleading; a reader might expect `/data` to auto-expand. Cosmetic only.

### 3.4 `implementation_plan.md`

Splits M2 -> M2a (local spike) + M2b (remote production), adds Track 1R roadmap (new section 5), renumbers sections 5->6, 6->7, and removes the old section 7 dependency graph (content folded into the milestone flow diagram at line 33). The M2b gate condition correctly cites R-3/R-5/R-6/R-9 as the final gates.

**Findings**:

**V-2 (Very Low) - Unchecked guard implementation checkbox**: The M2b checklist item `- [ ] new guards G-D5 ~ G-D9, G-R1, G-R2 implementation and verification (290 -> 297)` is marked `[ ]` (incomplete), but the guards (D-15~D-21) are implemented in `test_deploy_freshness.py` and verified passing (297 collected, 7 new pass). This is a tracking discrepancy - the work is done but the checkbox is not toggled. Recommend `- [x]`.

### 3.5 `tests/test_deploy_freshness.py`

**D-11 cleanup**: Removed the unused `recognized` set (which contained `MAM_MQTT_HOST` for exclusion-checking that was never exercised) and added `MQTT_BIND` to `valid_mqtt_vars`. Verified `MAM_MQTT_HOST` appears nowhere in the codebase. The test only checks `MQTT_*`-prefixed vars (regex `\b(MQTT_[A-Z0-9_]+)\b`), so `NATS_BIND`/`WS_BIND` are correctly excluded from validation. PASS

**D-15~D-21 new guards**: All 7 guards pass. Verified:
- D-15: store_dir absolute path + unquoted heredoc PASS
- D-16: nats image alpine-pinned (no `latest`) PASS
- D-17: port 8222 bound to 127.0.0.1 PASS
- D-18: TLS blocks use DNS names, not IP literals PASS
- D-19: subject literals match `DEFAULT_TOPIC_ROOT` (`python.mqtt.jobs`) PASS
- D-20: `run_loop.sh` exports `MAM_ENV_FILE` (verified line 106) PASS
- D-21: `.mam.env.example` documents `MQTT_KEEPALIVE`; no uncommented `MQTT_RETRY_INTERVAL`/`MQTT_MAX_RETRIES` PASS

**Finding**:

**L-3 (Low) - D-19 regex is brittle**: `re.findall(r'["\'](python\.mqtt\.jobs\.[>*\w.]+)["\']', content)` scans the entire markdown (not just code blocks) and matches subject literals in quoted strings. If a future prose sentence contains a quoted subject like `"python.mqtt.jobs.test"` without a wildcard, it would be validated. Currently passes but the scope is broader than "config examples". Non-blocking.

---

## 4. Task Deliverable Coverage

| Requirement | Status | Location |
|---|---|---|
| Production Docker Compose (MQTT 1883, NATS 4222, WS 8080, HTTP 8222, JetStream volume, healthchecks) | Partial | section 9.2 compose has 1883/8222/8080 + healthcheck + nats-data volume; **missing 4222** (M-1) |
| Production nats.conf (ports, JetStream, healthcheck endpoint) | PASS | section 9.1 nats.conf |
| Remote networking & security (UFW, TLS/Certbot vs Tailscale, user auth) | PASS | section 9.3 comparison table + UFW rules + section 9.1 accounts/permissions |
| Client configuration (.mam.env) | PASS | section 6 `.mam.env` template + `.mam.env.example` MQTT_KEEPALIVE |
| Remote verification playbooks (ping, latency, pub/sub) | PASS | section 9.4 R-1~R-10 + WAN latency probe |
| Phased rollout roadmap (M2 local spike + remote switchover) | PASS | implementation_plan.md M2a/M2b + section 5 Track 1R roadmap |

---

## 5. Test Validation Summary

| Suite | Tests | Result |
|---|---|---|
| `tests/test_deploy_freshness.py` (full) | 20 | PASS 20 passed (13.55s) |
| D-15~D-21 (new guards) | 7 | PASS 7 passed (0.02s) |
| `tests/test_tier1_unit.py` + `test_o2_race_free_lock.py` | 67 | PASS 67 passed (19.08s) |
| `tests/test_sanity.py` | 2 | PASS 2 passed (9.73s) |
| `--collect-only` (full suite) | 297 | PASS 297 collected |
| `bash -n .agents/skills/lib.sh` | - | PASS syntax OK |

Full suite (297 tests) not executed end-to-end due to 30s tool timeout; tier2/3/4 tests require a live broker. The fast subset (89 tests across deploy, tier1, o2, sanity) passes cleanly with no regressions.

---

## 6. Findings Summary

| ID | Severity | File | Description | Fix |
|---|---|---|---|---|
| **M-1** | Medium | PRIVATE_SERVER.md section 9.2 | Production compose omits NATS 4222; contradicts section 9.3 UFW rule and R-3 test | Add `4222:4222` port mapping to section 9.2, or remove 4222 from section 9.3/R-3 |
| L-1 | Low | lib.sh:1519 | `handle_startup_dialogs` call adds ~1s/iteration to `wait_for_tui_ready` | Acceptable; consider gating on `_pane_dialog_open` first |
| L-2 | Low | lib.sh:1738 | Default timeout halved (40s->20s) via sleep 2->1 | Acceptable for Claude; verify slow-env tolerance |
| L-3 | Low | test_deploy_freshness.py D-19 | Regex scans full markdown, not just code blocks | Narrow to code_blocks scope if desired |
| V-1 | Very Low | PRIVATE_SERVER.md:73 | Misleading store_dir comment ("$HOME expansion") | Clarify comment |
| V-2 | Very Low | implementation_plan.md | Guard checkbox unchecked despite work done | Toggle to `[x]` |

---

## 7. Recommendation

The changeset is production-ready for the M2b documentation milestone. The only Medium finding (M-1: section 9.2 missing 4222) is a documentation inconsistency resolvable by a one-line compose edit or removing the corresponding UFW/R-3 reference - no re-planning required. All 7 new guards pass; no test regressions; bash syntax valid; codebase cross-references (`mqtt_common.py` DEFAULT_TOPIC_ROOT, MQTT_KEEPALIVE; `run_loop.sh` MAM_ENV_FILE) all verified.

[VERDICT: PASS]