multi-agent-mux/BOOTSTRAP.md

BOOTSTRAP.md

This document guides you through the setup and initialization procedures required to adopt the herdr_agent_orchestration orchestration and messaging backplane workflow in a new project, enabling a new developer or agent to get up and running quickly.

A new agent can follow the steps in this guide sequentially to establish a stable and reliable initial environment.

---

1. Scaffolding Overview (Project Structure)

Before cloning this project into a new environment, you must first understand the locations and roles of its core components:

• .agents/: Orchestration and custom agent skills root.
  • MULTI_AGENT_RULES.md: Definition of agent roles (PM, Worker, Reviewer) and event publication rules.
  • MULTI_AGENT_RULES.ko.md: Definition of agent roles (PM, Worker, Reviewer) and event publication rules (Korean).
  • skills/: A collection of shell scripts that execute multi-agent coordination and asynchronous job processing.
    • lib.sh: The core orchestration shell functions and virtual environment (venv) auto-loading library.
    • multi-agent-mux-create/: Script to launch isolated herdr agent sessions.
    • multi-agent-mux-stop/: Script to gracefully stop agent sessions and update states.
    • multi-agent-mux-resume/: Script to restore stopped agent sessions back to their previous conversation state.
    • multi-agent-mux-status/: Script to query the current running state of all agent sessions.
    • multi-agent-mux-monitor/: Monitor script to sync herdr states with the registry.
    • multi-agent-mux-delegate-job/: Asynchronous job splitting and delegation module.
      • requirements.txt: Python dependency list (paho-mqtt, pyyaml).
      • scripts/: Python scripts running the core business logic.
        • registry.py: Job registration, claiming, and atomic file lock control (CLI supported).
        • job_subscriber.py: Background event subscriber and audit log generator.
        • publish_event.py: Event publisher for runtime states and error traps.
        • mqtt_common.py: Common utility for connecting to the MQTT broker.
    • multi-agent-mux-loop/: Autonomous planning-execution-review loop orchestrator.
    • multi-agent-mux-orc-onboard/: Orchestrator session UUID onboarding and sub-agent isolation helper.
• MESSAGING.md: Messaging scheme and wire protocol guidelines for MQTT communication between agents.

---

2. Workspace Provisioning (Step 0 — Reference Only)

Before any agent can read this guide, the workspace must be provisioned. This step has already been performed by the human developer using the installer. It is documented here strictly as a reference for verifying agents to understand how the assets arrived.

Step 0.1: Run the Installer One-Liner (Pre-Cloned/Provisioned)

The human developer bootstrapped the workspace by running:

curl -fsSL https://git.godopu.com/tmpl/multi-agent-mux/raw/branch/main/deploy/install.sh | bash

The installer verified system dependencies, staged the repository, copied the runtime .agents/ scaffolding and documentation files (including this BOOTSTRAP.md), and set up the default .mam.env and .venv environments.

Note

Verify Only: AI agents reading this guide must not attempt to run the installation script again. Proceed directly to configuration and verification steps.

---

3. Environment Configuration (.mam.env)

To set up the messaging broker and execution paths, you must create and modify a local environment configuration file (.mam.env).

Note

Legacy Config Migration: If you are updating from an earlier version that used .env, MAM will continue to read .env as a fallback. However, we recommend migrating to .mam.env via ./deploy/generate-env.sh --migrate-legacy.

Step 3.1: Run the Generation Script

Run the environment template copy script provided in the project root:

# Automatically copy .mam.env.example to .mam.env (does not overwrite if it already exists)
./deploy/generate-env.sh

# To force overwrite and create a backup of the existing .mam.env:
./deploy/generate-env.sh --force

# To explicitly migrate an existing legacy .env to .mam.env:
./deploy/generate-env.sh --migrate-legacy

Step 3.2: Modify Environment Variables

Open the generated .mam.env file to configure settings as needed.

Note

The default .mam.env file generated by generate-env.sh has all environment variables commented out. If left commented out, the system defaults to using relative paths (.mam/, etc.) relative to the local project root, and the public MQTT broker. You can use it as-is without uncommenting anything.

1. MQTT Broker Setup (MQTT_BROKER):
   • The default broker is HiveMQ's public sandbox broker (broker.hivemq.com). However, for production work where security and privacy are critical, we strongly recommend changing this to a private broker address.
2. Authentication Credentials (MQTT_USERNAME, MQTT_PASSWORD):
   • If using a secured broker, change the placeholders marked replace_me to your actual broker credentials.
3. Path Variables (Optional):
   • Uncomment and specify absolute paths for variables like AGENT_SESSIONS_YAML and DELEGATE_JOB_LOGS_DIR only if you need to override the default relative paths to align with specific build systems or host directories.

Warning

Security Mode Default Warning: The system's default setting is the unauthenticated PoC mode. If an auth_token is not explicitly provided (or is null) during job registration, HMAC signature verification is skipped. In a public broker environment or production phase, you must generate and inject a unique random auth_token during job registration to enable HMAC signature security. (For detailed security protocols, refer to section 2.3 Security Protocol in MESSAGING.md and MULTI_AGENT_RULES.md. Automated token generation and injection via CLI is on the roadmap under task FW-N6.)

---

4. Dependency and Virtualenv Setup

Set up the Python 3 dependencies required to run the orchestration and MQTT messaging backplane.

Step 4.1: Build Python Virtual Environment

Create and activate a .venv virtual environment in the project root:

# Create virtual environment
python3 -m venv .venv

# Activate virtual environment
source .venv/bin/activate

Step 4.2: Install Dependency Packages

Install the required packages listed in requirements.txt under multi-agent-mux-delegate-job:

# Install dependencies (pyyaml, paho-mqtt, etc.)
pip install -r .agents/skills/multi-agent-mux-delegate-job/requirements.txt

---

5. Directory Structure and Security Audit Guide

Ensure that the local registry directories required to track agent states and jobs are successfully created:

1. Required Directory Structure:
   • .mam/jobs/: Holds detailed metadata files for registered asynchronous jobs.
   • .mam/delegate_job_logs/: Holds the audit logs (events.ndjson) for all backplane events published by agents.
2. Git Ignore Configuration (.gitignore):
   • When initializing a new project, verify that the following entries are configured in .gitignore to prevent committing local runtimes to the repository. The exception !.mam.env.example must be kept to preserve the template:

     .env
     .env.*
     !.env.example
     .mam.env
     .mam.env.*
     !.mam.env.example
     .mam/
     .venv/
     __pycache__/
     *.pyc
     

---

6. Execution Verification and Bootstrap Tests

To verify that the environment has been successfully built without runtime errors, run the following verification checklist.

Important

All verification commands below must be executed from the project root directory (where the .mam/ directory is directly visible). This is because the default job registry path resolved by scripts is relative to the current working directory under ./.mam/jobs.

Verification Test 1: Registry Script Load Check

Verify that the Python scripts and virtual environment libraries load correctly by listing jobs:

# Run using the python interpreter in the virtual environment
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/registry.py list

• Expected Output: The command should exit successfully and print an empty JSON array [] (or a list of pending/running jobs if any exist) without any python traceback errors.

Verification Test 2: MQTT Broker Connection Handshake Test

Test the end-to-end communication through the broker to verify that events are published and received correctly:

# 1. Register a temporary test job and capture its 8-character Hex Job ID
JID=$(.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/registry.py register \
  --agent "test-agent" \
  --prompt "Bootstrap check command" \
  --timeout 120)
echo "Generated Job ID: $JID"

# 2. Run the background event subscriber (Subscriber) for this Job ID
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/job_subscriber.py --job "$JID" &

# 3. Wait 2 seconds to allow the Subscriber to establish its MQTT socket connection
sleep 2

# 4. Publish a start event (adhering to the Subscribe-before-Publish rule)
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/publish_event.py \
  --job "$JID" \
  --event started \
  --detail "Bootstrap MQTT verification connection check"

# 5. Verify that the event is printed to stdout and written to the audit log:
#    .mam/delegate_job_logs/events.ndjson

# 6. Stop the background subscriber and clean up the test job records
kill %1
rm -f ".mam/jobs/$JID.json" ".mam/jobs/$JID.lock"

---

7. Onboarding Collaborating Agents (New Agent Onboarding)

Once the setup is verified, onboarding agents (claude, agy, hermes, grok, opencode, codex) should immediately read the MULTI_AGENT_RULES.md guidelines in the .agents/ directory.

The guidelines describe essential workflows—such as surgical change constraints, cross-verification review loops, and pane snapshotting to prevent viewport truncation—allowing new agents to quickly and safely integrate with the multi-agent workflow.