multi-agent-mux/.agents/MULTI_AGENT_RULES.md

MULTI_AGENT_RULES.md

This document serves as the common guidelines and protocol for introducing the MQTT messaging backplane and Tmux-based multi-agent orchestration workflow to a new project. It defines the rules and architecture to ensure collaborating agents perform tasks safely, robustly, and consistently.

All agents working on a new project must read this document thoroughly and comply with the defined protocols before starting any tasks.

Note

This repository uses two separate guides: the general LLM behavioral guidelines (AGENTS.md) and the project-specific multi-agent orchestration guidelines (.agents/MULTI_AGENT_RULES.md).

---

1. Agent Roles Definition (Agent Roles)

We clearly separate responsibilities and permissions between roles to reduce bottlenecks and enhance the quality of execution.

👑 General Manager (Orchestrator)

• Core Responsibility: Interact directly with the user, receive high-level requirements, establish task plans, delegate tasks to Team Leaders, control the overall workflow, and report completion back to the user.
• Ambiguity Resolution: If a user's requirements contain ambiguous details, do not guess. Immediately ask the user for clarification (we recommend using the /grill-me slash command).

👥 Team Leaders (팀장)

Newly spawned agents (e.g., antigravity, claude, cline, hermes) act as Team Leaders of their respective groups. They receive delegated tasks from the General Manager and manage implementation or review workflows.

• Developer Team Leader (개발 팀장):
  • Receives tasks from the General Manager.
  • Task Breakdown & Planning: Thoroughly analyzes the task, breaks it down into small units, and creates a plan.
  • Internal Parallelism: Can run subagents in parallel internally to handle the delegated work.
  • Review Integrity & Refusal: Thoroughly reviews feedback from Reviewers. Adopts/implements recommendations if valid. If any recommendation is judged invalid, the Developer Team Leader must not implement it, but instead return the refutation along with detailed reasons to the Reviewer.
  • Completion Signal: Once all reviewers yield a PASS and changes are verified, the Developer Team Leader who first received the task sends a completion signal back to the General Manager.
• Reviewer Team Leader (리뷰어 팀장):
  • Receives review requests from the Developer Team Leader.
  • Detailed Feedback with Directions: Simply rejecting changes (NOT PASS) is forbidden. Reviewers must specify the exact reason for the issue and provide a concrete, stable, and verified alternative direction for improvement.
  • Consensus Loop: Engages in the review cycle until all objections are resolved and a final PASS is issued.

🛡️ Role Suitability Check Principle (자신의 역할 범위 수행 원칙)

• Every agent must only perform tasks suitable for its designated role (e.g., Developer Team Leaders do not issue final reviews, and Reviewer Team Leaders do not write project code).
• If an agent receives a task that does not fit its role, it must either:
  1. Recommend the optimal agent session to delegate the task to, or
  2. Perform the task directly if strictly necessary for project continuity.

---

2. Messaging Backplane & Registry Protocol

Asynchronous communication and state management between agents are controlled via distributed event channels and file/DB registries.

📡 MQTT Backplane

• Event Lifecycle:
  • started (Job execution starts) ➡️ progress/permission_required (Share intermediate progress) ➡️ completed (Successful termination) or error (Failed termination)
  • completed and error are terminal events that are published exactly once.
• Publish/Subscribe Rules:
  • Since MQTT does not guarantee persistent queues, the subscriber (job_subscriber.py) must be running in the background before the agent starts (the Subscribe-before-Publish principle).
  • When publishing terminal events, publish with retain=True on the broker so that subscribers joining late can still read the final state.
  • Generalize all transmitted data to ensure that sensitive secrets like passwords, private keys, or absolute system paths are not included.

🗃️ Registry & State Management

• This architecture maintains two distinct registries based on their purpose:
  • Job Registry: The metadata and lifecycle of each asynchronous job are recorded in individual JSON files (.mam/jobs/<id>.json). Concurrency conflicts (claiming races) across multiple sessions are prevented via file-based fcntl advisory locks (registry_lock via registry.py).
  • Session Registry: TMUX monitoring states and running agent metadata are consistently controlled using a SQLite WAL database (.mam/agent-sessions.db) to support reliable concurrent transactions on a single host. However, since SQLite WAL mode does not guarantee complete file locking in Network File System (NFS) environments, we recommend using a local file system.

🛡️ Security Protocol (HMAC-SHA256)

• Unauthenticated PoC Mode: If the auth_token in the job registry is set to null (the default PoC mode), signature verification is skipped and all events are accepted (verify_hmac always returns True).
• Authenticated Production Mode: In production environments or integrations requiring authentication, a unique cryptographic token (auth_token) is issued for each job. The publisher must include an hmac_sig signature in the payload keyed by this token, and the receiving end (verify_hmac) will immediately drop messages that lack a signature or have mismatching signatures to prevent downgrade attacks.
• Rollout Strategy: To avoid event drops caused by inconsistencies between publishing and receiving nodes when updating security schemes, hybrid transition formats (which risk leaking plaintext tokens) must not be used. Instead, adopt a "Simultaneous Rollout" where all nodes are updated at once.

---

3. Collaborative Workflow Execution Loop (Workflow Loop)

sequenceDiagram
    autonumber
    actor User as User
    participant GM as General Manager
    participant DTL as Developer Team Leader
    participant RTL as Reviewer Team Leaders
    participant M as MQTT Backplane

    User->>GM: Hand over requirements
    GM->>DTL: Delegate task (e.g., create landing page)
    Note over DTL: Analyze, breakdown & spawn parallel subagents
    DTL->>M: Publish 'started' event
    Note over DTL: Modify code & implement
    DTL->>M: Publish 'completed'
    DTL->>RTL: Request review (I created landing page. Please review it)
    Note over RTL: Cross-analysis & verification
    alt Defect Found (Reviewer feedback)
        RTL->>DTL: NOT PASS / Feedback (Must include reason & improvement direction)
        Note over DTL: DTL checks validity of suggestions
        alt Valid feedback
            Note over DTL: DTL adopts and modifies code
        else Invalid feedback
            DTL->>RTL: Send refutation & reasons (Did not reflect inappropriate parts)
        end
        DTL->>RTL: Request review again (Modified review items)
    else Verification Pass
        RTL->>DTL: PASS
    end
    DTL->>GM: Send completion signal
    GM->>User: Notify task completion

1. Planning and Allocation: The General Manager delegates the task to the Developer Team Leader.
2. Analysis and Internal Execution: The Developer Team Leader analyzes the task, breaks it down, plans execution, and optionally spawns parallel subagents. It publishes started, completes the task, and requests review from the Reviewer Team Leader.
3. Objection & Refinement Loop:
   • The Reviewer Team Leader must provide clear reasons and improvement directions for any issues.
   • The Developer Team Leader validates the feedback. Valid suggestions are implemented; invalid ones are refuted with reasons and returned to the reviewer.
   • This cycle repeats until all reviewers issue a PASS.
4. Completion and Report: The Developer Team Leader sends the final completion signal to the General Manager, who notifies the user.

---

4. Analysis Infrastructure Patterns & Practical Guide (Infra Patterns)

These are critical instructions for preventing data loss and infrastructure-level failures during long-running agent analyses.

📸 Preventing TUI Viewport Truncation (The 3 Pane Snapshotting Rules)

To ensure that agents running in TMUX environments do not lose debug logs or previous outputs due to screen scrollback limits, the following snapshotting pattern must be enforced:

1. Pre-brief Capture: Capture the pane (capture-pane -S -200) immediately after sending the task instruction (Brief) to back up the starting point of the input history.
2. Loop Snapshot: For long-running agent sessions (5 minutes or more), periodically (e.g., every 30 seconds) scan the viewport and append the incremental data to /tmp/pane-snap.txt.
3. Post-job Capture: Capture the complete pane state one final time immediately after a job completes or returns an error to preserve the entire execution trajectory.

📄 Markdown-Based Workflow & Communication (마크다운 기반 협업 및 결과 전달)

• Core Principle: To prevent TUI character loss, truncation, and layout breakage during sequential input typing, all collaborative workflows must favor file-based markdown communication.
• Rules & Protocols:
  • Exception: Extremely simple prompts (e.g., "Re-evaluate", "Check status", "Proceed") of 1 or 2 lines may be sent directly via tmux input buffers.
  • Task Delegation:
    • Manual path: Detailed task briefs may be written to a local Markdown file (e.g., .mam/reports/brief-<job_id>.md or a workspace path) first. The sender then issues a simple trigger command: "Read <file_path> and execute."
    • Automated path: The automated job runner (multi-agent-mux-delegate-job submit) automatically provisions the brief at .mam/jobs/<job_id>/brief.md and sends a short pointer instruction to the agent.
  • Result Reporting & Feedback:
    • Manual/Durable reviews: Detailed reviews, design proposals, or audit reports must be saved under .mam/reports/<tmux_session_name>/report-<job_id>.md.
    • Automated job reports: Automated execution results are saved directly to .mam/jobs/<job_id>/<agent_name>-reports/report-final.md (or <clean_session_name>-reports/ for loops) as transient files.
    • Versioned promotions: Any final design plans, review verdicts, or security audit reports that require version control must be explicitly copied to tracked directory paths (specifically under .agents/reports/<tmux_session_name>/ or docs/reports/).
  • Cleanup & Retention Contract: Files under .mam/jobs/<job_id>/ and .mam/reports/ are transient audit-trail artifacts. While durable outcomes are committed to version control under .agents/reports/, ephemeral directory trees can be cleaned up manually as needed; stop_session.sh does not automatically purge these report trees during session exit.

⏱️ Timeout Configuration & Alignment Rules

• Job Execution Limits (timeout_sec & idle_timeout_sec): Each job independently manages its overall execution timeout (timeout_sec, default 3600s) and idle timeout without receiving messages (idle_timeout_sec, default 120s).
• Monitor Idle Waiting (SUB_IDLE_TIMEOUT): The idle timeout for the monitor script (reconcile.sh), SUB_IDLE_TIMEOUT, must always be set generously to 3600s (1 hour) or more to align with the maximum job budget. This prevents the monitor from terminating early due to idle detection, which would lose control over background tasks before they finish.

---

5. Setup Checklist for New Projects (Setup Checklist)

Use this checklist when deploying this agent orchestration model to a new project:

• Virtualenv Dependencies: Are required Python packages like pyyaml and paho-mqtt included in .venv or requirements.txt?
• Configuration File: Are the MQTT broker address and security credentials safely loaded and shared via the .env file?
• Directory Convention: Are the registry path (.mam/jobs/) and logging path (.mam/delegate_job_logs/) added to .gitignore?
• Core Scripts: Are the core scripts (mqtt_common.py, publish_event.py, job_subscriber.py, and registry.py) in place?
• HMAC Enablement: When a new registry job is created, is a random auth_token correctly injected, and is signature-based mutual authentication active?
• Charter Placement: Is this protocol file (MULTI_AGENT_RULES.md) placed in the .agents/ directory of the new project? (Placing it in .agents/ is essential to keep the project root clean while allowing onboarding agents to align on the rules.)

---

6. Onboarding Handshake Protocol (Onboarding Handshake)

Newly spawned Team Leader agents must align their context using the --onboard mechanism before receiving any real work. This ensures they operate with the correct design rules and repository history.

🔄 The Onboarding Handshake Sequence

1. Creation with Onboard flag: The General Manager spawns a new session with:

   bash .agents/skills/multi-agent-mux-create/scripts/create_session.sh --workspace "$(pwd)" --agent <agent> --role <role> --onboard
   

   This automated onboarding workflow registers a job, provisioning the brief under .mam/jobs/<job_id>/brief.md and sending a short pointer to the agent session.
2. Orienting the Agent: The agent session starts up and automatically receives the registered onboarding brief instructing it to:
   • Read README.md and .agents/MULTI_AGENT_RULES.md to align with design principles and constraints.
   • Run git status and git diff to analyze active modifications.
   • Read .mam/agent-sessions.yaml to identify other running agents and verify its own assigned role.
3. Handshake Publication: The agent executes these orientation tasks and publishes a completed event to the broker (using publish_event.py) with:
   • Detail: "Onboarding complete; aligned with role <role>"
4. GM Gating Check: The General Manager polls or subscribes to this completed event. The GM must wait for this onboarding handshake to succeed before delegating any implementation or review jobs to the new session.

---

This guide balances collaboration efficiency with strict code security. Any required changes must be discussed and agreed upon by the General Manager and all Team Leaders before updating this document.