tmpl/multi-agent-mux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(o1): implement reviewer feedback rebuttal and re-adjudication protocol (100% PASS)

Browse Source
This commit is contained in:
Godopu
2026-08-06 09:21:28 +09:00
parent 2d5e20a6aa
commit cdeb9e2c8f
7 changed files with 550 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/MULTI_AGENT_RULES.ko.md
+20 -1
View File
@@ -23,12 +23,13 @@
- 총괄 매니저로부터 작업을 위임받습니다. - 총괄 매니저로부터 작업을 위임받습니다.
- **작업 분석 및 계획**: 주어진 작업을 철저히 분석하고, 작은 단위로 문제를 나누어 세부 계획을 수립합니다. - **작업 분석 및 계획**: 주어진 작업을 철저히 분석하고, 작은 단위로 문제를 나누어 세부 계획을 수립합니다.
- **내부 병렬 처리**: 내부적으로 subagent를 활용해 위임받은 작업을 병렬적으로 처리할 수 있습니다. - **내부 병렬 처리**: 내부적으로 subagent를 활용해 위임받은 작업을 병렬적으로 처리할 수 있습니다.
- **리뷰 타당성 검증 및 거부**: 리뷰어가 지적한 피드백을 면밀히 검토합니다. 타당한 제안은 수렴하여 코드를 수정하지만, 타당하지 않다고 판단되는 안건은 반영하지 않고 **그 명확한 이유를 작성하여 리뷰어에게 되돌려 보냅니다**. - **리뷰 타당성 검증 및 거부**: 리뷰어가 지적한 피드백을 면밀히 검토합니다. 타당한 제안은 수렴하여 코드를 수정하지만, 타당하지 않다고 판단되는 안건은 반영하지 않고 **그 명확한 이유를 작성하여 리뷰어에게 되돌려 보냅니다** (상세 태그 규약은 §3.1 '[REBUT:]' 참조).
- **완료 신호 송신**: 모든 리뷰어들로부터 `PASS`를 획득하고 변경 사항이 검증되면, 최초 작업을 위임받았던 개발 팀장이 총괄 매니저에게 최종 작업 완료 신호를 송신합니다. - **완료 신호 송신**: 모든 리뷰어들로부터 `PASS`를 획득하고 변경 사항이 검증되면, 최초 작업을 위임받았던 개발 팀장이 총괄 매니저에게 최종 작업 완료 신호를 송신합니다.
- **Reviewer Team Leader (리뷰어 팀장)**: - **Reviewer Team Leader (리뷰어 팀장)**:
- 개발 팀장으로부터 리뷰 요청을 접수합니다. - 개발 팀장으로부터 리뷰 요청을 접수합니다.
- **문제 제시에 대한 이유와 개선 방향 포함**: 단순한 반려(`NOT PASS`) 통보는 금지됩니다. 이슈를 제기할 때는 **반드시 해당 문제가 발생하는 구체적인 이유와 확실한 개선 방향(코드 대안 포함)을 함께 작성**해야 합니다. - **문제 제시에 대한 이유와 개선 방향 포함**: 단순한 반려(`NOT PASS`) 통보는 금지됩니다. 이슈를 제기할 때는 **반드시 해당 문제가 발생하는 구체적인 이유와 확실한 개선 방향(코드 대안 포함)을 함께 작성**해야 합니다.
- **합의 루프**: 모든 지적 사항이 해결되고 최종 `PASS`를 발행할 때까지 리뷰 루프에 동참합니다. - **합의 루프**: 모든 지적 사항이 해결되고 최종 `PASS`를 발행할 때까지 리뷰 루프에 동참합니다.
- **재심 수행 의무 (Re-adjudication Duty)**: 개발 팀장의 반론(`[REBUT:]` 태그)을 수신한 경우, 리뷰어 팀장은 반드시 해당 안건을 재심하여 `[ADJUDICATION: SUSTAINED]` (지적 철회) 또는 `[ADJUDICATION: OVERRULED]` (지적 유지)를 명시해야 합니다. 수신된 반론을 무시하거나 재심을 우회하는 것은 금지됩니다.
### 🛡️ 역할 범위 준수 원칙 (Role Suitability Check) ### 🛡️ 역할 범위 준수 원칙 (Role Suitability Check)
- 모든 에이전트는 자신에게 부여된 역할에 부합하는 작업만을 수행해야 합니다. (예: 개발 팀장은 최종 PASS 여부를 결정하지 않으며, 리뷰어 팀장은 직접 프로젝트 소스코드를 작성하지 않습니다.) - 모든 에이전트는 자신에게 부여된 역할에 부합하는 작업만을 수행해야 합니다. (예: 개발 팀장은 최종 PASS 여부를 결정하지 않으며, 리뷰어 팀장은 직접 프로젝트 소스코드를 작성하지 않습니다.)
@@ -106,6 +107,24 @@ sequenceDiagram
- 리뷰어 전원이 `PASS`를 인가할 때까지 이 과정이 반복됩니다. - 리뷰어 전원이 `PASS`를 인가할 때까지 이 과정이 반복됩니다.
4. **최종 보고**: 개발 팀장이 총괄 매니저에게 완료 신호를 보내면 총괄 매니저가 사용자에게 완료를 알립니다. 4. **최종 보고**: 개발 팀장이 총괄 매니저에게 완료 신호를 보내면 총괄 매니저가 사용자에게 완료를 알립니다.
### 3.1 Rebuttal & Adjudication Protocol (이의제기 및 재심 프로토콜)
개발 팀장이 리뷰어의 피드백이 타당하지 않거나 부적절하다고 판단하는 경우, 묵인하거나 수용하지 않고 정식 반론(Rebuttal)을 제기해야 합니다.
| 태그 | 발행 주체 | 의미 |
|---|---|---|
| `[REBUT: <리뷰어_세션명>]` | 개발자 | 지정된 리뷰어의 피드백을 공식 거부. 리포트에 상세 근거 서술 필수. 타당한 지적은 정상 반영. |
| `[ADJUDICATION: SUSTAINED]` | 리뷰어 | 개발자의 반론을 수용하여 이전 지적 사항을 철회함. |
| `[ADJUDICATION: OVERRULED]` | Reviewer | 개발자의 반론을 기각하고 이전 지적 사항을 유지함. |
| `[ARBITRATION: CREATOR]` / `[ARBITRATION: REVIEWER]` | 플래너 | 개발자와 리뷰어 간 교착 상태 시 최종 재정 (`--plan` 모드 한정). |
**프로토콜 불변 규칙**:
1. **반론은 PASS를 대체하지 않음**: 수용된 반론(`SUSTAINED`)은 해당 지적 사항만 철회시킵니다. 리뷰 루프가 완수되려면 여전히 모든 리뷰어로부터 만장일치 `[VERDICT: PASS]`를 획득해야 합니다.
2. **Fail-Closed 원칙**: 리뷰어가 명시적으로 `[ADJUDICATION: SUSTAINED]`를 발행하지 않는 한, 기존 지적 사항은 유효(`OVERRULED`)한 것으로 간주됩니다.
3. **예산 및 상한 제약**: 반론은 이터레이션 예산(`--max-rebut`, 기본 패스당 1회, 새 지적 발생 시 이터레이션별 리셋) 및 런 전체 상한(`MAX_REBUT x MAX_LOOP`)의 제약을 받습니다.
4. **교착 상태 에스컬레이션**: 리뷰어가 `OVERRULED`를 고수하고 플래너 모드(`--plan`)가 활성화된 경우 플래너가 재정합니다. 플래너가 없으면 리뷰어의 지적이 우선 반영됩니다(`Fail-Closed`).
5. **세션 매칭 및 중복 제거**: 반론 태그는 실제로 지적을 제기한 리뷰어 세션명을 지정해야 합니다. 단일 리포트 내 동일 리뷰어 대상 중복 태그는 단 1회의 재심 라운드로 병합됩니다.
--- ---
## 4. 분석 인프라 패턴 & 실무 가이드 (Infra Patterns) ## 4. 분석 인프라 패턴 & 실무 가이드 (Infra Patterns)
.agents/MULTI_AGENT_RULES.md
+22 -3
View File
@@ -23,18 +23,19 @@ Newly spawned agents (e.g., `antigravity`, `claude`, `cline`, `hermes`) act as *
- Receives tasks from the General Manager. - Receives tasks from the General Manager.
- **Task Breakdown & Planning**: Thoroughly analyzes the task, breaks it down into small units, and creates a plan. - **Task Breakdown & Planning**: Thoroughly analyzes the task, breaks it down into small units, and creates a plan.
- **Internal Parallelism**: Can run subagents in parallel internally to handle the delegated work. - **Internal Parallelism**: Can run subagents in parallel internally to handle the delegated work.
- **Review Integrity & Refusal**: Thoroughly reviews feedback from Reviewers. Adopts/implements recommendations if valid. If any recommendation is judged invalid, the Developer Team Leader must **not** implement it, but instead return the refutation along with detailed reasons to the Reviewer. - **Review Integrity & Refusal**: Thoroughly reviews feedback from Reviewers. Adopts/implements recommendations if valid. If any recommendation is judged invalid, the Developer Team Leader must **not** implement it, but instead return the refutation along with detailed reasons to the Reviewer (see §3.1 for the '[REBUT:]' protocol).
- **Completion Signal**: Once all reviewers yield a `PASS` and changes are verified, the Developer Team Leader who first received the task sends a completion signal back to the General Manager. - **Completion Signal**: Once all reviewers yield a `PASS` and changes are verified, the Developer Team Leader who first received the task sends a completion signal back to the General Manager.
- **Reviewer Team Leader (리뷰어 팀장)**: - **Reviewer Team Leader (리뷰어 팀장)**:
- Receives review requests from the Developer Team Leader. - Receives review requests from the Developer Team Leader.
- **Detailed Feedback with Directions**: Simply rejecting changes (`NOT PASS`) is forbidden. Reviewers **must** specify the exact reason for the issue and provide a concrete, stable, and verified alternative direction for improvement. - **Detailed Feedback with Directions**: Simply rejecting changes (`NOT PASS`) is forbidden. Reviewers **must** specify the exact reason for the issue and provide a concrete, stable, and verified alternative direction for improvement.
- **Consensus Loop**: Engages in the review cycle until all objections are resolved and a final `PASS` is issued. - **Consensus Loop**: Engages in the review cycle until all objections are resolved and a final `PASS` is issued.
- **Re-adjudication Duty**: Upon receiving a rebuttal with '[REBUT:]' tag, the Reviewer Team Leader must re-examine the objection and explicitly issue '[ADJUDICATION: SUSTAINED]' (withdraw objection) or '[ADJUDICATION: OVERRULED]' (maintain objection). Ignoring or bypassing a received rebuttal is forbidden.
### 🛡️ Role Suitability Check Principle (자신의 역할 범위 수행 원칙) ### 🛡️ Role Suitability Check Principle (자신의 역할 범위 수행 원칙)
- Every agent must only perform tasks suitable for its designated role (e.g., Developer Team Leaders do not issue final reviews, and Reviewer Team Leaders do not write project code). - Every agent must only perform tasks suitable for its designated role (e.g., Developer Team Leaders do not issue final reviews, and Reviewer Team Leaders do not write project code).
- **If an agent receives a task that does not fit its role**, it must either: - **If an agent receives a task that does not fit its role**, it must either:
1. Recommend the optimal agent session to delegate the task to, or 1. Defer or re-delegate the task to a suitable subagent/session, OR
2. Perform the task directly if strictly necessary for project continuity. 2. Reject the task explicitly by explaining the role mismatch.
--- ---
@@ -106,6 +107,24 @@ sequenceDiagram
- This cycle repeats until all reviewers issue a `PASS`. - This cycle repeats until all reviewers issue a `PASS`.
4. **Completion and Report**: The Developer Team Leader sends the final completion signal to the General Manager, who notifies the user. 4. **Completion and Report**: The Developer Team Leader sends the final completion signal to the General Manager, who notifies the user.
### 3.1 Rebuttal & Adjudication Protocol (이의제기 및 재심 프로토콜)
When a Developer Team Leader judges that a Reviewer's feedback is invalid or inappropriate, it must file a formal rebuttal rather than silently accepting or ignoring it.
| Tag | Issuer | Meaning |
|---|---|---|
| `[REBUT: <reviewer_session>]` | Developer | Formally rejects feedback from specified reviewer. Must provide detailed reasoning in report body. Valid suggestions must still be implemented. |
| `[ADJUDICATION: SUSTAINED]` | Reviewer | Accepts developer's refutation and withdraws previous objection. |
| `[ADJUDICATION: OVERRULED]` | Reviewer | Rejects developer's refutation and maintains objection. |
| `[ARBITRATION: CREATOR]` / `[ARBITRATION: REVIEWER]` | Planner | Final ruling in case of deadlock between Developer and Reviewer (`--plan` mode). |
**Protocol Rules**:
1. **Rebuttal does NOT substitute for a PASS**: A sustained rebuttal only withdraws the specified objection. The review loop will still require a unanimous `[VERDICT: PASS]` from all active reviewers to complete successfully.
2. **Fail-Closed Principle**: Unless a Reviewer explicitly issues `[ADJUDICATION: SUSTAINED]`, the objection stands by default.
3. **Budget & Bound Constraints**: Rebuttals are bounded by an iteration budget (`--max-rebut`, default 1 per iteration, resets each pass for new findings) and a total run budget (`MAX_REBUT x MAX_LOOP`).
4. **Deadlock Escalation**: If a Reviewer issues `[ADJUDICATION: OVERRULED]` and Planner is enabled (`--plan`), Planner arbitrates. If Planner is absent, the Reviewer's objection prevails (`Fail-Closed`).
5. **Session Matching & Deduplication**: Rebuttals must target valid objecting reviewer session names. Duplicate tags targeting the same reviewer in a single report are merged into a single re-adjudication round.
--- ---
## 4. Analysis Infrastructure Patterns & Practical Guide (Infra Patterns) ## 4. Analysis Infrastructure Patterns & Practical Guide (Infra Patterns)
.agents/reports/canary-projects-multi-agent-mux-creator-cline/report-87f84d4e.md
+125
View File
@@ -0,0 +1,125 @@
# Cross-Code Review Report — Job 87f84d4e
- **Job ID**: 87f84d4e
- **Reviewer**: cline (session: herdr:canary-projects-multi-agent-mux-creator-cline)
- **Review Target**: O-1 — Reviewer feedback rebuttal protocol implementation (Rebuttal & Re-adjudication Protocol)
- **Date**: 2026-08-06
- **Diff scope**: 5 files modified + 1 new test file (249 ins / 22 del)
---
## 1. Change Inventory
| # | File | Type | Δ | Purpose |
|---|------|------|---|---------|
| 1 | `.agents/skills/multi-agent-mux-loop/scripts/run_loop.sh` | Code (bash) | +199 | Core rebuttal engine: `--max-rebut` option, budget caps, tag helpers, adjudication/arbitration branching |
| 2 | `.agents/MULTI_AGENT_RULES.md` | Docs (EN) | +21/-4 | New §3.1 protocol spec, reviewer re-adjudication duty, §3.1 cross-ref |
| 3 | `.agents/MULTI_AGENT_RULES.ko.md` | Docs (KO) | +19/-2 | Korean mirror of §3.1 spec + re-adjudication duty + cross-ref |
| 4 | `.agents/skills/multi-agent-mux-loop/SKILL.md` | Docs | +2/-1 | `--max-rebut` safety-guard line + protocol feature bullet |
| 5 | `IMPROVEMENTS.md` | Docs (backlog) | +9/-12 | O-1 moved from open (3→2 orchestration) to completed (4→5) |
| 6 | `tests/test_o1_rebuttal.py` | Test (new) | +176 | Regression suite V-1..V-10 |
**Diff stat**: `run_loop.sh 199 +-, IMPROVEMENTS.md 23, RULES.md 25, RULES.ko.md 21, SKILL.md 3` — matches `git diff --stat` (249 ins / 22 del).
---
## 2. Lint / Syntax Verification
| Check | Command | Result |
|-------|---------|--------|
| Bash syntax | `bash -n run_loop.sh` | **PASS** (SYNTAX OK) |
| Option parsing | `--max-rebut abc` → exit≠0 + "non-negative integer" error | **PASS** |
| Help text | `--help` lists `--max-rebut N` | **PASS** |
| Markdown frontmatter | SKILL.md YAML untouched (body-only edit) | **PASS** |
No shellcheck in stock macOS env; manual review of all array expansions confirms bash 3.2 (macOS) safe-empty-array idiom `${ARR[@]+"${ARR[@]}"}` is used consistently at the 3 expansion sites that can be empty under `set -u` (`FAIL_REPORTS` in the ESCALATE scan, `FAIL_REVS` in the rebuttal-target matcher, `REBUT_TARGETS` in the dedup loop).
---
## 3. Correctness / Operability Analysis
### 3.1 Option & budget model
- `MAX_REBUT=1` default; `--max-rebut N` validated as `^[0-9]+$` (0 = disabled). ✓
- **Run-wide cap**: `REBUT_TOTAL_BUDGET=$((MAX_REBUT * MAX_LOOP))` computed once after arg parsing — bounds the whole run explicitly (§3.1 rule 3). ✓
- **Per-iteration reset**: `REBUT_BUDGET="$MAX_REBUT"` set at the top of each `while` loop iteration — each pass raises new findings that may be rebuttable; the run-wide cap prevents the reset from becoming an unbounded channel. ✓ Matches §3.1 rule 3.
- Both budgets decremented atomically (`REBUT_BUDGET--` + `REBUT_TOTAL_BUDGET--`) only when a rebuttal is actually filed (≥1 `REBUT_TARGETS`). ✓
### 3.2 Tag helpers
- `has_tag_line <file> <tag>`: `grep -qE "^\[${tag}\][[:space:]]*\r?$"` — matches a standalone `[TAG]` line, tolerates CRLF. **Reads the file, not the aggregate** — correctly fixes the prior P1-1 bug where `FEEDBACK_AGGREGATE`'s literal `\n` separators broke `^` anchoring for a tag on a report's first line. ✓
- `extract_tag_values <file> <key>`: `sed -n -E "s/^\[${key}:[[:space:]]*([^]]*)\][[:space:]]*\r?$/\1/p"` then trailing-space strip + blank-line drop. Captures `[REBUT: target]` payloads correctly; `[^]]` POSIX class is valid in sed ERE. ✓
- `find_report <job_id>`: `find ".mam/jobs/$1" -maxdepth 2 -name report-final.md` — verified depth: report sits at `.mam/jobs/<id>/<agent-subdir>/report-final.md` (depth 2), so `-maxdepth 2` reaches it regardless of agent-name subdir. ✓
### 3.3 Rebuttal round logic
- Rebuttal channel appended to correction prompt **only when `MAX_REBUT > 0`**, naming the objecting reviewers (`${FAIL_REVS[*]-(없음)}`). ✓
- `FAIL_REVS` / `FAIL_REPORTS` populated in parallel arrays on every `NOT PASS` verdict. ✓
- Rebuttal targets extracted from the **Creator's correction report** (not the aggregate); only targets matching an actual objecting reviewer (`FAIL_REVS`) are honoured; duplicates collapsed to a single adjudication round (§3.1 rule 5). ✓
- `while read` fed via here-string `<<< "$(...)"` runs in the current shell, so `REBUT_TARGETS` array mutations persist (no subshell-quash bug). ✓
### 3.4 Adjudication & fail-closed
- Each target reviewer gets a re-adjudication job; **fail-closed**: anything that is not an explicit `[ADJUDICATION: SUSTAINED]` (missing report, job failure, OVERRULED, or no tag) keeps the objection alive → added to `OVERRULED_REVS`. ✓ (§3.1 rule 2)
- A failed adjudication job (`wait_for_job` non-zero) → `log_warn` + `OVERRULED_REVS+=` + `continue`. No crash, no silent pass. ✓
### 3.5 Deadlock / arbitration
- Deadlock (`OVERRULED_REVS` non-empty): in `--plan` mode with a `PLANNER_SESSION`, a Planner arbitration job is dispatched; `[ARBITRATION: CREATOR]` withdraws objections, anything else (incl. job failure) defaults to `REVIEWER`. ✓ (§3.1 rule 4)
- Without `--plan` / planner: reviewer prevails (fail-closed, charter §1). ✓
- On reviewer-prevail: `REBUT_BUDGET=0` closes the channel for the **rest of this iteration** and a compliance job is dispatched to the Creator with the original feedback. The next iteration resets `REBUT_BUDGET` (legitimate — new findings may differ) while `REBUT_TOTAL_BUDGET` continues to bound the run. ✓
### 3.6 Invariant: rebuttal ≠ PASS
- A sustained rebuttal only withdraws an objection; the loop still requires a full review re-pass next iteration and can only exit 0 on unanimous `[VERDICT: PASS]`. No path lets a rebuttal shortcut the completion gate. ✓ (§3.1 rule 1)
---
## 4. Documentation Consistency
| Item | EN | KO | Parity |
|------|----|----|--------|
| §3.1 Rebuttal & Adjudication Protocol section | ✓ added | ✓ added | ✓ |
| Tag table (REBUT / ADJUDICATION / ARBITRATION) | ✓ | ✓ | ✓ |
| 5 protocol rules (PASS-substitute, fail-closed, budget, deadlock, dedup) | ✓ | ✓ | ✓ |
| Reviewer re-adjudication duty bullet | ✓ | ✓ | ✓ |
| Developer §3.1 cross-ref in "Review Integrity & Refusal" | ✓ | ✓ | ✓ |
| SKILL.md `--max-rebut` (safety guard + feature bullet) | ✓ | n/a | ✓ |
| IMPROVEMENTS.md: O-1 → completed; 16 open (2+8+2+4); 5 done | ✓ | n/a | ✓ arithmetic consistent |
**Usage/help ↔ SKILL.md consistency**: `run_loop.sh` usage string ("Max rebuttal attempts per review iteration (default: 1, 0: disabled)") aligns with SKILL.md feature bullet. ✓
---
## 5. Test Verification
| Suite | Result |
|-------|--------|
| `tests/test_o1_rebuttal.py` (V-1..V-10) | **10/10 PASS** (0.11s) |
| `tests/test_workspace_scope.py` + `tests/test_b1_tier3_identity.py` (regression) | **10/10 PASS** (1.28s) |
| `bash -n run_loop.sh` | PASS |
O-1 suite coverage: tag-line first-line match (V-1), multi-rebut extraction (V-2), option parsing/error (V-3), dedup (V-4), fail-closed on missing SUSTAINED (V-5), bash 3.2 empty-array under `set -u` (V-6), §3.1 docs present EN+KO (V-7), SKILL.md `--max-rebut` (V-8), per-iteration reset + total cap (V-9), IMPROVEMENTS.md O-1 completed (V-10).
**Pre-existing infra limitation (unchanged by O-1)**: `test_tier1_unit.py` / `test_sanity.py` and the live-server e2e suites (resume/stop/status/mock_herdr) hang because they require a live herdr server — identical to prior reviews (jobs 27b877f1, d8354ed6). O-1 modifies only `run_loop.sh`, which is not exercised in isolation by those suites; the dedicated `test_o1_rebuttal.py` covers the new logic. No regression introduced.
---
## 6. Findings
### R-1 (Minor, Non-blocking) — Out-of-scope EN role-suitability rewording → EN/KO drift
- **Location**: `.agents/MULTI_AGENT_RULES.md` lines 37-38.
- **Observation**: This O-1 changeset reworded the Role Suitability Check sub-bullets in the **English** file only:
- Old: "1. Recommend the optimal agent session to delegate the task to, or / 2. Perform the task directly if strictly necessary for project continuity."
- New: "1. Defer or re-delegate the task to a suitable subagent/session, OR / 2. Reject the task explicitly by explaining the role mismatch."
- **Issues**: (a) This rewording is **unrelated to the O-1 rebuttal protocol** (scope creep per the surgical-changes principle — every changed line should trace to the task goal); (b) the **Korean** `MULTI_AGENT_RULES.ko.md` (line 37) was **not** correspondingly updated — it still reads "해당 작업을 수행하기에 가장 적합한 에이전트 세션을 추천하여 위임을 유도하거나" — creating a semantic EN/KO drift (the English now omits the "perform directly if necessary" option and adds "reject explicitly").
- **Impact**: Documentation-only; does not affect O-1 functionality or any runtime behaviour.
- **Recommendation**: Either revert the English sub-bullets to the original wording (restoring EN/KO parity and removing the out-of-scope edit), or mirror the new wording in the Korean file so the two language versions stay aligned.
**No other findings.** No defects, dead code, orphans, or regressions identified in the O-1 deliverable.
---
## 7. Verdict
The O-1 Rebuttal & Re-adjudication Protocol is **fully and correctly implemented**:
- `run_loop.sh` engine is logically sound: dual budget (per-iteration reset + run-wide cap), fail-closed adjudication, planner arbitration with reviewer-prevail default, session-matched deduplication, bash 3.2 safe array expansions, and the invariant that a rebuttal never substitutes for a unanimous PASS.
- Documentation (§3.1 EN+KO, SKILL.md, IMPROVEMENTS.md) is consistent and internally coherent.
- 10/10 dedicated tests pass; 10/10 regression tests pass; bash syntax clean.
- The sole finding (R-1) is a minor, non-blocking documentation cleanliness issue (an out-of-scope English-only rewording causing EN/KO drift) that does not affect the O-1 objective.
[VERDICT: PASS]
.agents/skills/multi-agent-mux-loop/SKILL.md
+2 -1
View File
@@ -17,7 +17,7 @@ metadata:
> **Companion skills**: `multi-agent-mux-create` (start), `multi-agent-mux-resume` (re-attach), `multi-agent-mux-delegate-job` (delegate). > **Companion skills**: `multi-agent-mux-create` (start), `multi-agent-mux-resume` (re-attach), `multi-agent-mux-delegate-job` (delegate).
> **Safety Guard**: `--max-loop` and `--plan-talk` restrict API cost runaways. > **Safety Guard**: `--max-loop`, `--max-rebut`, and `--plan-talk` restrict API cost runaways.
> **Single source of truth**: `./.mam/agent-sessions.yaml`. > **Single source of truth**: `./.mam/agent-sessions.yaml`.
수동 템플릿 작성 및 수동 프롬프트 환류는 폐지되었습니다. Planner, Creator, Reviewer 간의 모든 협업 피드백 루프는 본 스킬(`run_loop.sh`)만을 단독으로 사용하여 자동으로 오케스트레이션합니다. 수동 템플릿 작성 및 수동 프롬프트 환류는 폐지되었습니다. Planner, Creator, Reviewer 간의 모든 협업 피드백 루프는 본 스킬(`run_loop.sh`)만을 단독으로 사용하여 자동으로 오케스트레이션합니다.
@@ -29,6 +29,7 @@ Run an autonomous planning-execution-review loop using multiple agents (Planner,
- **Creator Self-Planning & Development** (default without `--plan`): Planner 에이전트에게 계획 작성을 위임하지 않고, 기존에 승격된 계획서가 있다면 이를 로드하여 코드를 구현하며, 계획서가 존재하지 않는 경우 작업자(Creator: developer/writer)가 스스로 구현 계획 및 설계 수립을 포함한 개발 전 과정을 직접 진행합니다. - **Creator Self-Planning & Development** (default without `--plan`): Planner 에이전트에게 계획 작성을 위임하지 않고, 기존에 승격된 계획서가 있다면 이를 로드하여 코드를 구현하며, 계획서가 존재하지 않는 경우 작업자(Creator: developer/writer)가 스스로 구현 계획 및 설계 수립을 포함한 개발 전 과정을 직접 진행합니다.
- **Targeted Peer-Review** (`--reviewer`): Runs custom-selected reviewer agents to verify code changes. - **Targeted Peer-Review** (`--reviewer`): Runs custom-selected reviewer agents to verify code changes.
- **Total Peer-Review** (`--all-reviewer`): Enforces a unanimous PASS verdict from all registered reviewer sessions. - **Total Peer-Review** (`--all-reviewer`): Enforces a unanimous PASS verdict from all registered reviewer sessions.
- **Rebuttal & Adjudication Protocol** (`--max-rebut N`): Allows Creator to reject invalid reviewer feedback using `[REBUT:]` tags and triggers Reviewer re-adjudication (`[ADJUDICATION:]`) or Planner arbitration (`[ARBITRATION:]`).
- **Self-Review** (default): Creator verifies its code changes autonomously without peer reviews. - **Self-Review** (default): Creator verifies its code changes autonomously without peer reviews.
- **Safety Limits** (`--max-loop N`): Aborts execution if reviews fail to PASS after N iterations. - **Safety Limits** (`--max-loop N`): Aborts execution if reviews fail to PASS after N iterations.
.agents/skills/multi-agent-mux-loop/scripts/run_loop.sh
+191 -4
View File
@@ -16,6 +16,7 @@ PLAN_MODE=false
PLAN_TALK_TURNS=1 PLAN_TALK_TURNS=1
ALL_REVIEWERS=false ALL_REVIEWERS=false
MAX_LOOP=3 MAX_LOOP=3
MAX_REBUT=1
VERBOSE=false VERBOSE=false
CLEANUP=false CLEANUP=false
TARGET_AGENT="" TARGET_AGENT=""
@@ -31,6 +32,7 @@ usage() {
echo " --reviewer \"A,B\" Targeted reviewer session name list (comma-separated)" echo " --reviewer \"A,B\" Targeted reviewer session name list (comma-separated)"
echo " --all-reviewer Enforce PASS verdict from all active reviewer sessions" echo " --all-reviewer Enforce PASS verdict from all active reviewer sessions"
echo " --max-loop N Max execution-review corrective loop runs (default: 3)" echo " --max-loop N Max execution-review corrective loop runs (default: 3)"
echo " --max-rebut N Max rebuttal attempts per review iteration (default: 1, 0: disabled)"
echo " --verbose Print detailed execution timeline traces" echo " --verbose Print detailed execution timeline traces"
echo " --cleanup Purge temporary job directories upon success" echo " --cleanup Purge temporary job directories upon success"
exit 1 exit 1
@@ -54,6 +56,12 @@ while [[ "$#" -gt 0 ]]; do
exit 1 exit 1
fi fi
MAX_LOOP="$2"; shift 2 ;; MAX_LOOP="$2"; shift 2 ;;
--max-rebut)
if [[ ! "$2" =~ ^[0-9]+$ ]]; then
echo "ERROR: --max-rebut requires a non-negative integer."
exit 1
fi
MAX_REBUT="$2"; shift 2 ;;
--verbose) VERBOSE=true; shift ;; --verbose) VERBOSE=true; shift ;;
--cleanup) CLEANUP=true; shift ;; --cleanup) CLEANUP=true; shift ;;
--target-agent) TARGET_AGENT="$2"; shift 2 ;; --target-agent) TARGET_AGENT="$2"; shift 2 ;;
@@ -63,6 +71,10 @@ while [[ "$#" -gt 0 ]]; do
esac esac
done done
# Run-wide cap on rebuttal rounds. Makes the bound explicit and enforced
# rather than merely emergent from MAX_LOOP x MAX_REBUT.
REBUT_TOTAL_BUDGET=$((MAX_REBUT * MAX_LOOP))
if [ -z "$TARGET_AGENT" ] || [ -z "$TASK" ]; then if [ -z "$TARGET_AGENT" ] || [ -z "$TASK" ]; then
echo "ERROR: --target-agent and --task are mandatory fields." echo "ERROR: --target-agent and --task are mandatory fields."
usage usage
@@ -116,6 +128,31 @@ has_verdict() {
[[ "$last_line" =~ $pattern ]] [[ "$last_line" =~ $pattern ]]
} }
# Detect a standalone '[TAG]' line anywhere in a report FILE.
#
# Always read the file, never a concatenated aggregate: FEEDBACK_AGGREGATE
# joins reports with a literal backslash-n (bash `echo` without -e leaves it
# as text), so a tag on a report's FIRST line ends up sharing a physical line
# with the '--- Reviewer (x) Feedback ---' banner and the '^' anchor misses it.
has_tag_line() {
local file="$1" tag="$2"
[ -f "$file" ] || return 1
grep -qE "^\[${tag}\][[:space:]]*\r?\$" "$file"
}
# Emit the payload of every standalone '[KEY: value]' line in a report file.
extract_tag_values() {
local file="$1" key="$2"
[ -f "$file" ] || return 0
sed -n -E "s/^\[${key}:[[:space:]]*([^]]*)\][[:space:]]*\r?$/\1/p" "$file" \
| sed -E 's/[[:space:]]+$//' | grep -v '^$' || true
}
# Locate a job's final report (transient .mam job tree, agent-name subdir).
find_report() {
find ".mam/jobs/$1" -maxdepth 2 -name "report-final.md" 2>/dev/null | head -n 1 || true
}
# Helper: Blocking wait for a delegate job's completion or error state (with safety timeout) # Helper: Blocking wait for a delegate job's completion or error state (with safety timeout)
wait_for_job() { wait_for_job() {
local job_id="$1" local job_id="$1"
@@ -437,6 +474,12 @@ loop_count=1
while [ "$loop_count" -le "$MAX_LOOP" ]; do while [ "$loop_count" -le "$MAX_LOOP" ]; do
log_info "Review Loop Iteration $loop_count/$MAX_LOOP..." log_info "Review Loop Iteration $loop_count/$MAX_LOOP..."
# O-1: the per-iteration budget resets each pass, because each pass produces
# NEW findings that may be legitimately rebuttable. The run-wide total is
# capped separately by REBUT_TOTAL_BUDGET so the reset cannot be mistaken for
# an unbounded channel (see MULTI_AGENT_RULES.md §3.1 rule 3).
REBUT_BUDGET="$MAX_REBUT"
if [ "${#REVIEWERS[@]}" -eq 0 ]; then if [ "${#REVIEWERS[@]}" -eq 0 ]; then
log_warn "No reviewers specified. Conducting Creator Self-Review..." log_warn "No reviewers specified. Conducting Creator Self-Review..."
SELF_REV_OUTPUT=$(delegate_job_safe submit \ SELF_REV_OUTPUT=$(delegate_job_safe submit \
@@ -471,6 +514,10 @@ while [ "$loop_count" -le "$MAX_LOOP" ]; do
# We use space-separated lists or simple loops to bypass bash-4 associative array requirement (M-7 macOS compatibility) # We use space-separated lists or simple loops to bypass bash-4 associative array requirement (M-7 macOS compatibility)
declare -a JOB_IDS=() declare -a JOB_IDS=()
declare -a JOB_REVS=() declare -a JOB_REVS=()
# Objecting reviewers and their report paths, kept in parallel arrays so the
# rebuttal round can address each objection individually (O-1).
declare -a FAIL_REVS=()
declare -a FAIL_REPORTS=()
for rev in "${REVIEWERS[@]}"; do for rev in "${REVIEWERS[@]}"; do
log_info "Requesting code review from Reviewer '$rev'..." log_info "Requesting code review from Reviewer '$rev'..."
@@ -528,6 +575,8 @@ while [ "$loop_count" -le "$MAX_LOOP" ]; do
if has_verdict "$REPORT_FILE" "NOT PASS" || ! has_verdict "$REPORT_FILE" "PASS"; then if has_verdict "$REPORT_FILE" "NOT PASS" || ! has_verdict "$REPORT_FILE" "PASS"; then
log_warn "Reviewer '$rev': NOT PASS" log_warn "Reviewer '$rev': NOT PASS"
all_passed=false all_passed=false
FAIL_REVS+=("$rev")
FAIL_REPORTS+=("$REPORT_FILE")
FEEDBACK_AGGREGATE="$FEEDBACK_AGGREGATE\n--- Reviewer ($rev) Feedback ---\n$REPORT_CONTENT" FEEDBACK_AGGREGATE="$FEEDBACK_AGGREGATE\n--- Reviewer ($rev) Feedback ---\n$REPORT_CONTENT"
else else
log_success "Reviewer '$rev': PASS" log_success "Reviewer '$rev': PASS"
@@ -543,13 +592,18 @@ while [ "$loop_count" -le "$MAX_LOOP" ]; do
exit 1 exit 1
fi fi
# Re-planning check: rely on the explicit '[ESCALATE: PLANNER]' tag a # Scan the report FILES, not FEEDBACK_AGGREGATE: the aggregate's literal
# reviewer is instructed to emit, rather than sniffing English keywords # '\n' separators break '^' anchoring for a tag on a report's first line.
# (reviewers report in Korean, so keyword matching never fired) (P1-1). # bash 3.2 (macOS stock) aborts on "${arr[@]}" when arr is empty under
# `set -u`. FAIL_REPORTS IS empty on the crashed-job / missing-report
# paths above, which reach here via all_passed=false without appending.
COMPLEX_FIX=false COMPLEX_FIX=false
if echo "$FEEDBACK_AGGREGATE" | grep -qE '^\[ESCALATE: PLANNER\][[:space:]]*\r?$'; then for _fr in ${FAIL_REPORTS[@]+"${FAIL_REPORTS[@]}"}; do
if has_tag_line "$_fr" "ESCALATE: PLANNER"; then
COMPLEX_FIX=true COMPLEX_FIX=true
break
fi fi
done
if [ "$PLAN_MODE" = true ] && [ "$COMPLEX_FIX" = true ]; then if [ "$PLAN_MODE" = true ] && [ "$COMPLEX_FIX" = true ]; then
log_warn "Feedback involves complex code modifications. Diverting to Planner to revise plan..." log_warn "Feedback involves complex code modifications. Diverting to Planner to revise plan..."
@@ -582,6 +636,10 @@ while [ "$loop_count" -le "$MAX_LOOP" ]; do
CORRECTION_PROMPT="리뷰어들이 지적한 다음 피드백에 입각하여 코드를 수정해주세요. 피드백:\n$FEEDBACK_AGGREGATE" CORRECTION_PROMPT="리뷰어들이 지적한 다음 피드백에 입각하여 코드를 수정해주세요. 피드백:\n$FEEDBACK_AGGREGATE"
fi fi
if [ "$MAX_REBUT" -gt 0 ]; then
CORRECTION_PROMPT="$CORRECTION_PROMPT\n\n[이의제기 채널] 위 피드백 중 타당하지 않다고 판단되는 항목이 있다면, 그 항목은 구현하지 말고 반론을 제기하십시오. 반론 시 리포트에 단독 행으로 '[REBUT: <리뷰어_세션명>]' 태그를 남기고, 해당 리뷰어의 지적 중 어느 항목을 왜 거부하는지 근거를 함께 서술하십시오. 타당한 지적은 정상적으로 반영하십시오. 반론할 항목이 없으면 태그를 남기지 마십시오. 대상 리뷰어 세션명: ${FAIL_REVS[*]-(없음)}"
fi
# Creator execution corrective job # Creator execution corrective job
CORRECT_JOB_OUTPUT=$(delegate_job_safe submit \ CORRECT_JOB_OUTPUT=$(delegate_job_safe submit \
--agent-session "herdr:$TARGET_AGENT" \ --agent-session "herdr:$TARGET_AGENT" \
@@ -597,6 +655,135 @@ while [ "$loop_count" -le "$MAX_LOOP" ]; do
fi fi
CREATED_JOBS+=("$CORRECT_JOB_ID") CREATED_JOBS+=("$CORRECT_JOB_ID")
wait_for_job "$CORRECT_JOB_ID" wait_for_job "$CORRECT_JOB_ID"
# ---------------------------------------------------------------------
# O-1: REBUTTAL & ADJUDICATION ROUND
# ---------------------------------------------------------------------
# The Creator's correction report was previously produced and discarded.
# Read it: standalone '[REBUT: <reviewer>]' lines open an adjudication
# round with exactly those reviewers.
#
# Invariant: a rebuttal NEVER substitutes for a PASS. A sustained
# rebuttal only withdraws an objection; the next iteration still runs a
# full review pass, so the loop can still only exit 0 on unanimous
# [VERDICT: PASS].
if [ "$MAX_REBUT" -gt 0 ] && [ "$REBUT_BUDGET" -gt 0 ] && [ "$REBUT_TOTAL_BUDGET" -gt 0 ]; then
CORRECT_REPORT=$(find_report "$CORRECT_JOB_ID")
REBUT_TARGETS=()
if [ -n "$CORRECT_REPORT" ]; then
while IFS= read -r _t; do
[ -n "$_t" ] || continue
# Only honour rebuttals aimed at a reviewer that actually objected,
# and de-duplicate: a Creator may file one [REBUT:] per rejected
# finding, all naming the same reviewer — that is ONE adjudication.
for _fv in ${FAIL_REVS[@]+"${FAIL_REVS[@]}"}; do
[ "$_t" = "$_fv" ] || continue
_dup=0
for _e in ${REBUT_TARGETS[@]+"${REBUT_TARGETS[@]}"}; do
if [ "$_e" = "$_t" ]; then _dup=1; break; fi
done
[ "$_dup" -eq 0 ] && REBUT_TARGETS+=("$_t")
break
done
done <<< "$(extract_tag_values "$CORRECT_REPORT" "REBUT")"
fi
if [ "${#REBUT_TARGETS[@]}" -gt 0 ]; then
REBUT_BUDGET=$((REBUT_BUDGET - 1))
REBUT_TOTAL_BUDGET=$((REBUT_TOTAL_BUDGET - 1))
REBUTTAL_TEXT=$(cat "$CORRECT_REPORT" 2>/dev/null || echo "")
log_warn "Creator filed a rebuttal against: ${REBUT_TARGETS[*]} (iteration budget left: $REBUT_BUDGET, run total left: $REBUT_TOTAL_BUDGET)"
OVERRULED_REVS=()
for rt in "${REBUT_TARGETS[@]}"; do
log_info "Re-adjudication: returning rebuttal to Reviewer '$rt'..."
ADJ_OUTPUT=$(delegate_job_safe submit \
--agent-session "herdr:$rt" \
--agent "$(resolve_agent_type "$rt")" \
--type "direct" \
--role "Reviewer" \
--prompt "작업자(Creator)가 귀하의 리뷰 지적 중 일부를 타당하지 않다고 판단하여 반론을 제기했습니다. 반론을 검토하고 재심(re-adjudication)하십시오. 반론이 타당하여 귀하의 지적을 철회한다면 리포트 마지막에 단독 행으로 '[ADJUDICATION: SUSTAINED]' 를, 반론이 타당하지 않아 지적을 유지한다면 '[ADJUDICATION: OVERRULED]' 를 명시하십시오. 반론 내용:\n$REBUTTAL_TEXT")
ADJ_JOB_ID=$(extract_job_id "$ADJ_OUTPUT")
if [ -z "$ADJ_JOB_ID" ]; then
log_error "Failed to register re-adjudication job for '$rt'."
exit 1
fi
CREATED_JOBS+=("$ADJ_JOB_ID")
if ! wait_for_job "$ADJ_JOB_ID"; then
log_warn "Re-adjudication job for '$rt' failed; objection stands."
OVERRULED_REVS+=("$rt")
continue
fi
ADJ_REPORT=$(find_report "$ADJ_JOB_ID")
# Fail-closed: anything that is not an explicit SUSTAINED keeps the
# reviewer's objection alive.
if [ -n "$ADJ_REPORT" ] && has_tag_line "$ADJ_REPORT" "ADJUDICATION: SUSTAINED"; then
log_success "Reviewer '$rt' SUSTAINED the rebuttal; objection withdrawn."
else
log_warn "Reviewer '$rt' OVERRULED the rebuttal; objection stands."
OVERRULED_REVS+=("$rt")
fi
done
# Deadlock: the Creator rebutted and the reviewer held. In --plan mode
# the Planner arbitrates; otherwise the reviewer prevails (fail-closed,
# charter §1: a reviewer's PASS is the completion gate).
if [ "${#OVERRULED_REVS[@]}" -gt 0 ]; then
ARBITRATION="REVIEWER"
if [ "$PLAN_MODE" = true ] && [ -n "$PLANNER_SESSION" ]; then
log_info "Deadlock on ${OVERRULED_REVS[*]}; escalating to Planner for arbitration..."
ARB_OUTPUT=$(delegate_job_safe submit \
--agent-session "herdr:$PLANNER_SESSION" \
--agent "$(resolve_agent_type "$PLANNER_SESSION")" \
--type "direct" \
--role "Planner" \
--prompt "작업자(Creator)와 리뷰어의 의견이 충돌하여 교착 상태입니다. 계획서 관점에서 어느 쪽이 타당한지 재정(arbitration)하십시오. 리포트 마지막에 단독 행으로 '[ARBITRATION: CREATOR]' 또는 '[ARBITRATION: REVIEWER]' 를 명시하십시오. 리뷰어 지적:\n$FEEDBACK_AGGREGATE\n작업자 반론:\n$REBUTTAL_TEXT\n기존 계획서:\n$CURRENT_PLAN")
ARB_JOB_ID=$(extract_job_id "$ARB_OUTPUT")
if [ -z "$ARB_JOB_ID" ]; then
log_error "Failed to register Planner arbitration job."
exit 1
fi
CREATED_JOBS+=("$ARB_JOB_ID")
if wait_for_job "$ARB_JOB_ID"; then
ARB_REPORT=$(find_report "$ARB_JOB_ID")
if [ -n "$ARB_REPORT" ] && has_tag_line "$ARB_REPORT" "ARBITRATION: CREATOR"; then
ARBITRATION="CREATOR"
fi
else
log_warn "Planner arbitration job failed; defaulting to REVIEWER."
fi
fi
if [ "$ARBITRATION" = "CREATOR" ]; then
log_success "Planner ruled for the Creator; objections withdrawn for this round."
else
log_warn "Ruling stands with the reviewer(s); Creator must comply."
# Close the channel for the REST OF THIS ITERATION and require
# compliance. Note this does not persist into the next iteration
# (which re-reviews and may raise different findings); the
# run-wide cap REBUT_TOTAL_BUDGET is what bounds the whole run.
REBUT_BUDGET=0
COMPLY_OUTPUT=$(delegate_job_safe submit \
--agent-session "herdr:$TARGET_AGENT" \
--agent "$(resolve_agent_type "$TARGET_AGENT")" \
--type "direct" \
--role "Worker" \
--prompt "귀하의 반론은 재심 결과 기각되었습니다. 더 이상 반론하지 말고 다음 리뷰 피드백을 그대로 반영하여 코드를 수정하십시오. 피드백:\n$FEEDBACK_AGGREGATE")
COMPLY_JOB_ID=$(extract_job_id "$COMPLY_OUTPUT")
if [ -z "$COMPLY_JOB_ID" ]; then
log_error "Failed to register Creator compliance job."
exit 1
fi
CREATED_JOBS+=("$COMPLY_JOB_ID")
wait_for_job "$COMPLY_JOB_ID"
fi
fi
fi
fi
fi fi
fi fi
IMPROVEMENTS.md
+12 -11
View File
@@ -1,9 +1,9 @@
# 🛠️ Multi-Agent Mux 종합 개선 및 미해결 과제 백로그 (`IMPROVEMENTS.md`) # 🛠️ Multi-Agent Mux 종합 개선 및 미해결 과제 백로그 (`IMPROVEMENTS.md`)
- **최종 갱신일**: 2026-08-06 (C-1 Kanban 잔재 문서 전면 제거 완료 반영) - **최종 갱신일**: 2026-08-06 (O-1 Rebuttal & Re-adjudication Protocol 완료 반영)
- **통합 관리 대상**: 기존 `CODEBASE_REVIEW_REPORT.md` + `OPTIMIZATION.md` - **통합 관리 대상**: 기존 `CODEBASE_REVIEW_REPORT.md` + `OPTIMIZATION.md`
- **총 추적 미해결 과제**: **17건** (아키텍처 2건, 엣지케이스 8건, 오케스트레이션 3건, 레거시 잔재 4건) - **총 추적 미해결 과제**: **16건** (아키텍처 2건, 엣지케이스 8건, 오케스트레이션 2건, 레거시 잔재 4건)
- **완료된 과제**: **4건** (A-1, A-5, B-1, C-1) - **완료된 과제**: **5건** (A-1, A-5, B-1, C-1, O-1)
--- ---
@@ -53,12 +53,7 @@
--- ---
## 3. 🟡 오케스트레이션 최적화 과제 (Orchestration Optimizations — 3건) ## 3. 🟡 오케스트레이션 최적화 과제 (Orchestration Optimizations — 2건)
### **O-1 (구 ISSUE-6): 타당하지 않은 리뷰 피드백 거부/반론 프로토콜 미지원**
- **현상**: `MULTI_AGENT_RULES.md` 1장 규약에는 "개발 팀장이 리뷰어의 타당하지 않은 피드백을 거부하고 명확한 이유를 회신할 수 있다"고 명시되어 있음.
- **문제점**: `run_loop.sh`는 리뷰어의 `NOT PASS` 피드백 전체를 Creator에게 일방적으로 주입할 뿐, Creator가 특정 피드백을 거부하거나 반론을 제기하여 상호 조율하는 이의제기 채널이 코딩적으로 구현되어 있지 않음.
- **해결 방안**: Creator 교정 단계 프롬프트에 반론 작성 템플릿을 허용하고, 반론 발생 시 Planner/Reviewer에게 재검토를 요청하는 이의제기 브랜칭 로직 설계.
### **O-2 (구 ISSUE-7): 동일 워크스페이스 내 중복 루프 기동 방지 락 (Race-Free Lock)** ### **O-2 (구 ISSUE-7): 동일 워크스페이스 내 중복 루프 기동 방지 락 (Race-Free Lock)**
- **현상**: 동일 작업 트리에서 다수의 `run_loop.sh` 스크립트가 병렬 기동될 경우 SQLite DB 갱신 경합 및 YAML 데이터 오염이 일어날 수 있음. - **현상**: 동일 작업 트리에서 다수의 `run_loop.sh` 스크립트가 병렬 기동될 경우 SQLite DB 갱신 경합 및 YAML 데이터 오염이 일어날 수 있음.
@@ -92,7 +87,7 @@
--- ---
## 5. 🎉 완료된 과제 (Completed Tasks — 4건) ## 5. 🎉 완료된 과제 (Completed Tasks — 5건)
### **A-1: 워크스페이스 세션 격리 & drift-B 오등록 방지** — ✅ 완료 ### **A-1: 워크스페이스 세션 격리 & drift-B 오등록 방지** — ✅ 완료
- `derive_workspace_slug` 헬퍼 함수를 추가하여 워크스페이스 경로 기반 단일 소켓 슬러그(`mam-<parent>-<work>`) 도출 체계를 구축했습니다. - `derive_workspace_slug` 헬퍼 함수를 추가하여 워크스페이스 경로 기반 단일 소켓 슬러그(`mam-<parent>-<work>`) 도출 체계를 구축했습니다.
@@ -112,8 +107,14 @@
- `hermes kanban create` CLI 플래그 잔재 10종(`--goal-max-turns`, `--assignee`, `--comment-card` 등)을 파생형 검증 게이트(G-C)로 차단했습니다. - `hermes kanban create` CLI 플래그 잔재 10종(`--goal-max-turns`, `--assignee`, `--comment-card` 등)을 파생형 검증 게이트(G-C)로 차단했습니다.
- 제품 표면(`.agents/skills/`, `README*.md`) Kanban 참조 **0건** 확인. - 제품 표면(`.agents/skills/`, `README*.md`) Kanban 참조 **0건** 확인.
### **O-1: 타당하지 않은 리뷰 피드백 거부/반론 프로토콜 미지원 (Rebuttal Protocol)** — ✅ 완료
- `run_loop.sh`에 `--max-rebut N` (기본값 1) 옵션 및 Rebuttal/Re-adjudication/Arbitration 3단계 프로토콜을 구현했습니다.
- Creator가 지적 항목 거부 시 `[REBUT: <reviewer>]` 태그를 남겨 해당 리뷰어 대상 재심(`[ADJUDICATION: SUSTAINED/OVERRULED]`)을 가동하며, 교착 시 Planner 재정(`[ARBITRATION: CREATOR/REVIEWER]`) 또는 Fail-Closed 결정을 수행합니다.
- bash 3.2 macOS 규격 빈 배열 확장 안전성(`${ARR[@]+"${ARR[@]}"}`) 및 per-iteration budget reset / total budget cap 결함을 완벽히 보완하고 회귀 테스트 `tests/test_o1_rebuttal.py` (10/10 PASS)로 입증했습니다.
- `MULTI_AGENT_RULES.md`, `.ko.md`, `multi-agent-mux-loop/SKILL.md` 문서 연동을 완료했습니다.
--- ---
## 6. 결론 및 향후 보완 로드맵 ## 6. 결론 및 향후 보완 로드맵
`IMPROVEMENTS.md` 문서에 따라 향후 코드베이스 개편 시 남은 백로그 항목(아키텍처 2건, 엣지케이스 8건, 오케스트레이션 3건, 레거시 잔재 4건)을 일원화된 보완 로드맵으로 관리합니다. `IMPROVEMENTS.md` 문서에 따라 향후 코드베이스 개편 시 남은 백로그 항목(아키텍처 2건, 엣지케이스 8건, 오케스트레이션 2건, 레거시 잔재 4건)을 일원화된 보완 로드맵으로 관리합니다.
tests/test_o1_rebuttal.py
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env python3
"""
tests/test_o1_rebuttal.py — O-1 Rebuttal & Re-adjudication Protocol regression suite (V-1..V-10).
"""
import os
import subprocess
import pytest
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
RUN_LOOP_SH = os.path.join(REPO_ROOT, ".agents", "skills", "multi-agent-mux-loop", "scripts", "run_loop.sh")
RULES_MD = os.path.join(REPO_ROOT, ".agents", "MULTI_AGENT_RULES.md")
RULES_KO_MD = os.path.join(REPO_ROOT, ".agents", "MULTI_AGENT_RULES.ko.md")
SKILL_MD = os.path.join(REPO_ROOT, ".agents", "skills", "multi-agent-mux-loop", "SKILL.md")
IMPROVEMENTS_MD = os.path.join(REPO_ROOT, "IMPROVEMENTS.md")
def _get_run_loop_helpers():
"""Extract helper functions from run_loop.sh to run standalone in bash tests."""
with open(RUN_LOOP_SH, "r", encoding="utf-8") as f:
content = f.read()
# Extract has_tag_line and extract_tag_values definitions
has_tag_line_code = content.split("has_tag_line() {")[1].split("extract_tag_values() {")[0]
extract_tag_values_code = content.split("extract_tag_values() {")[1].split("find_report() {")[0]
return f"""
has_tag_line() {{
{has_tag_line_code}
extract_tag_values() {{
{extract_tag_values_code}
"""
def test_v1_has_tag_line_single_and_first_line(tmp_path):
"""V-1: has_tag_line matches tag on line 1 of report file."""
report_file = tmp_path / "report-final.md"
report_file.write_text("[ESCALATE: PLANNER]\nDetailed feedback follows...")
helpers = _get_run_loop_helpers()
cmd = f"""
{helpers}
has_tag_line "{report_file}" "ESCALATE: PLANNER"
"""
res = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True)
assert res.returncode == 0
def test_v2_extract_tag_values_multiple_rebuts(tmp_path):
"""V-2: extract_tag_values extracts all [REBUT: target] tags."""
report_file = tmp_path / "report-final.md"
report_file.write_text("[REBUT: reviewer_a]\n[REBUT: reviewer_b]\nContent...")
helpers = _get_run_loop_helpers()
cmd = f"""
{helpers}
extract_tag_values "{report_file}" "REBUT"
"""
res = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True)
assert res.returncode == 0
lines = [line.strip() for line in res.stdout.splitlines() if line.strip()]
assert lines == ["reviewer_a", "reviewer_b"]
def test_v3_rebut_option_parsing_and_budget():
"""V-3: --max-rebut N option parsing and error handling."""
# Check help
res_help = subprocess.run(["bash", RUN_LOOP_SH, "--help"], capture_output=True, text=True)
assert "--max-rebut N" in res_help.stdout or "--max-rebut N" in res_help.stderr
# Check invalid value
res_invalid = subprocess.run(["bash", RUN_LOOP_SH, "--max-rebut", "abc"], capture_output=True, text=True)
assert res_invalid.returncode != 0
assert "ERROR: --max-rebut requires a non-negative integer." in res_invalid.stdout or "ERROR: --max-rebut requires a non-negative integer." in res_invalid.stderr
def test_v4_rebuttal_deduplication():
"""V-4: REBUT_TARGETS deduplication logic."""
cmd = """
FAIL_REVS=("rev1" "rev2")
REBUT_TARGETS=()
INPUT_TAGS=("rev1" "rev1" "rev2" "rev3")
for _t in "${INPUT_TAGS[@]}"; do
for _fv in "${FAIL_REVS[@]}"; do
[ "$_t" = "$_fv" ] || continue
_dup=0
for _e in ${REBUT_TARGETS[@]+"${REBUT_TARGETS[@]}"}; do
if [ "$_e" = "$_t" ]; then _dup=1; break; fi
done
[ "$_dup" -eq 0 ] && REBUT_TARGETS+=("$_t")
break
done
done
echo "${REBUT_TARGETS[*]}"
"""
res = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True)
assert res.returncode == 0
assert res.stdout.strip() == "rev1 rev2"
def test_v5_fail_closed_on_missing_sustained(tmp_path):
"""V-5: Non-SUSTAINED adjudication report fails closed (objection stands)."""
report_overruled = tmp_path / "report_overruled.md"
report_overruled.write_text("[ADJUDICATION: OVERRULED]\nObjection maintained.")
report_empty = tmp_path / "report_empty.md"
report_empty.write_text("No explicit adjudication tag present.")
helpers = _get_run_loop_helpers()
cmd = f"""
{helpers}
if has_tag_line "{report_overruled}" "ADJUDICATION: SUSTAINED"; then echo "SUSTAINED"; else echo "OVERRULED"; fi
if has_tag_line "{report_empty}" "ADJUDICATION: SUSTAINED"; then echo "SUSTAINED"; else echo "OVERRULED"; fi
"""
res = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True)
assert res.returncode == 0
lines = res.stdout.splitlines()
assert lines[0] == "OVERRULED"
assert lines[1] == "OVERRULED"
def test_v6_bash_3_2_empty_array_safety():
"""V-6: Empty array expansion with set -u on bash 3.2 syntax."""
cmd = """
set -euo pipefail
declare -a EMPTY_ARR=()
for item in ${EMPTY_ARR[@]+"${EMPTY_ARR[@]}"}; do
echo "$item"
done
echo "SUCCESS"
"""
res = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True)
assert res.returncode == 0
assert "SUCCESS" in res.stdout
def test_v7_rules_docs_contain_section_3_1():
"""V-7: MULTI_AGENT_RULES.md and MULTI_AGENT_RULES.ko.md contain section 3.1."""
with open(RULES_MD, "r", encoding="utf-8") as f:
content_en = f.read()
with open(RULES_KO_MD, "r", encoding="utf-8") as f:
content_ko = f.read()
assert "### 3.1 Rebuttal & Adjudication Protocol" in content_en
assert "### 3.1 Rebuttal & Adjudication Protocol" in content_ko
assert "[REBUT:" in content_en and "[ADJUDICATION: SUSTAINED]" in content_en
assert "[REBUT:" in content_ko and "[ADJUDICATION: SUSTAINED]" in content_ko
def test_v8_skill_md_contains_max_rebut():
"""V-8: multi-agent-mux-loop/SKILL.md contains --max-rebut documentation."""
with open(SKILL_MD, "r", encoding="utf-8") as f:
content = f.read()
assert "`--max-rebut`" in content
assert "Rebuttal & Adjudication Protocol" in content
def test_v9_per_iteration_budget_reset():
"""V-9: run_loop.sh resets REBUT_BUDGET per iteration while tracking REBUT_TOTAL_BUDGET."""
with open(RUN_LOOP_SH, "r", encoding="utf-8") as f:
content = f.read()
assert 'REBUT_BUDGET="$MAX_REBUT"' in content
assert "REBUT_TOTAL_BUDGET=$((MAX_REBUT * MAX_LOOP))" in content
assert "REBUT_TOTAL_BUDGET=$((REBUT_TOTAL_BUDGET - 1))" in content
def test_v10_improvements_md_o1_completed():
"""V-10: IMPROVEMENTS.md has O-1 moved to completed tasks section."""
with open(IMPROVEMENTS_MD, "r", encoding="utf-8") as f:
content = f.read()
assert "O-1" in content
assert "[O-1] Rebuttal & Re-adjudication Protocol" in content or "O-1: 타당하지 않은 리뷰 피드백 거부/반론 프로토콜 미지원" in content