# BOOTSTRAP.md

This document guides you through the setup and initialization procedures required to adopt the `tmux_agent_orchestration` orchestration and messaging backplane workflow in a new project, enabling a new developer or agent to get up and running quickly.

A new agent can follow the steps in this guide sequentially to establish a stable and reliable initial environment.

---

## 1. Scaffolding Overview (Project Structure)

Before cloning this project into a new environment, you must first understand the locations and roles of its core components:

*   `.agents/skills/`: A collection of shell scripts that execute multi-agent coordination and asynchronous job processing.
    *   `lib.sh`: The core orchestration shell functions and virtual environment (venv) auto-loading library.
    *   `multi-agent-mux-create/`: Script to launch isolated tmux agent sessions.
    *   `multi-agent-mux-stop/`: Script to gracefully stop agent sessions and update states.
    *   `multi-agent-mux-resume/`: Script to restore stopped agent sessions back to their previous conversation state.
    *   `multi-agent-mux-status/`: Script to query the current running state of all agent sessions.
    *   `multi-agent-mux-monitor/`: Monitor script to sync tmux states with the registry.
    *   `multi-agent-mux-delegate-job/`: Asynchronous job splitting and delegation module.
        *   `requirements.txt`: Python dependency list (`paho-mqtt`, `pyyaml`).
        *   `scripts/`: Python scripts running the core business logic.
            *   `registry.py`: Job registration, claiming, and atomic file lock control (CLI supported).
            *   `job_subscriber.py`: Background event subscriber and audit log generator.
            *   `publish_event.py`: Event publisher for runtime states and error traps.
            *   `mqtt_common.py`: Common utility for connecting to the MQTT broker.
*   `AGENT.md`: Definition of agent roles (PM, Worker, Reviewer) and event publication rules.
*   `MESSAGING.md`: Messaging scheme and wire protocol guidelines for MQTT communication between agents.

---

## 2. Environment Configuration (.env)

To set up the messaging broker and execution paths, you must create and modify a local environment configuration file (`.env`).

### Step 2.1: Run the Generation Script
Run the environment template copy script provided in the project root:

```bash
# Automatically copy .env.example to .env (does not overwrite if it already exists)
./scripts/generate-env.sh

# To force overwrite and create a backup of the existing .env:
./scripts/generate-env.sh --force
```

### Step 2.2: Modify Environment Variables
Open the generated `.env` file to configure settings as needed.

> [!NOTE]
> The default `.env` file generated by `generate-env.sh` has all environment variables commented out. If left commented out, the system defaults to using relative paths (`.mam/`, etc.) relative to the local project root, and the public MQTT broker. You can use it as-is without uncommenting anything.

1.  **MQTT Broker Setup (`MQTT_BROKER`)**:
    *   The default broker is HiveMQ's public sandbox broker (`broker.hivemq.com`). However, for production work where security and privacy are critical, we strongly recommend changing this to a private broker address.
2.  **Authentication Credentials (`MQTT_USERNAME`, `MQTT_PASSWORD`)**:
    *   If using a secured broker, change the placeholders marked `replace_me` to your actual broker credentials.
3.  **Path Variables (Optional)**:
    *   Uncomment and specify absolute paths for variables like `AGENT_SESSIONS_YAML` and `DELEGATE_JOB_LOGS_DIR` only if you need to override the default relative paths to align with specific build systems or host directories.

> [!WARNING]
> **Security Mode Default Warning**:
> The system's default setting is the **unauthenticated PoC mode**. If an `auth_token` is not explicitly provided (or is `null`) during job registration, HMAC signature verification is skipped.
> In a public broker environment or production phase, you must generate and inject a unique random `auth_token` during job registration to enable HMAC signature security. (For detailed security protocols, refer to section `2.3 Security Protocol` in [MESSAGING.md](./MESSAGING.md) and [AGENT.md](./AGENT.md). Automated token generation and injection via CLI is on the roadmap under task `FW-N6`.)

---

## 3. Dependency and Virtualenv Setup

Set up the Python 3 dependencies required to run the orchestration and MQTT messaging backplane.

### Step 3.1: Build Python Virtual Environment
Create and activate a `.venv` virtual environment in the project root:

```bash
# Create virtual environment
python3 -m venv .venv

# Activate virtual environment
source .venv/bin/activate
```

### Step 3.2: Install Dependency Packages
Install the required packages listed in `requirements.txt` under `multi-agent-mux-delegate-job`:

```bash
# Install dependencies (pyyaml, paho-mqtt, etc.)
pip install -r .agents/skills/multi-agent-mux-delegate-job/requirements.txt
```

---

## 4. Directory Structure and Security Audit Guide

Ensure that the local registry directories required to track agent states and jobs are successfully created:

1.  **Required Directory Structure**:
    *   `.mam/jobs/`: Holds detailed metadata files for registered asynchronous jobs.
    *   `.mam/delegate_job_logs/`: Holds the audit logs (`events.ndjson`) for all backplane events published by agents.
2.  **Git Ignore Configuration (`.gitignore`)**:
    *   When initializing a new project, verify that the following entries are configured in `.gitignore` to prevent committing local runtimes to the repository. The exception `!.env.example` must be kept to preserve the template:
        ```text
        .env
        .env.*
        !.env.example
        .mam/
        .venv/
        __pycache__/
        *.pyc
        ```

---

## 5. Execution Verification and Bootstrap Tests

To verify that the environment has been successfully built without runtime errors, run the following verification checklist.

> [!IMPORTANT]
> All verification commands below must be executed from the **project root directory** (where the `.mam/` directory is directly visible). This is because the default job registry path resolved by scripts is relative to the current working directory under `./.mam/jobs`.

### Verification Test 1: Registry Script Load Check
Verify that the Python scripts and virtual environment libraries load correctly by listing jobs:

```bash
# Run using the python interpreter in the virtual environment
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/registry.py list
```
*   **Expected Output**: The command should exit successfully and print an empty JSON array `[]` (or a list of pending/running jobs if any exist) without any python traceback errors.

### Verification Test 2: MQTT Broker Connection Handshake Test
Test the end-to-end communication through the broker to verify that events are published and received correctly:

```bash
# 1. Register a temporary test job and capture its 8-character Hex Job ID
JID=$(.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/registry.py register \
  --agent "test-agent" \
  --prompt "Bootstrap check command" \
  --timeout 120)
echo "Generated Job ID: $JID"

# 2. Run the background event subscriber (Subscriber) for this Job ID
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/job_subscriber.py --job "$JID" &

# 3. Wait 2 seconds to allow the Subscriber to establish its MQTT socket connection
sleep 2

# 4. Publish a start event (adhering to the Subscribe-before-Publish rule)
.venv/bin/python3 .agents/skills/multi-agent-mux-delegate-job/scripts/publish_event.py \
  --job "$JID" \
  --event started \
  --detail "Bootstrap MQTT verification connection check"

# 5. Verify that the event is printed to stdout and written to the audit log:
#    .mam/delegate_job_logs/events.ndjson

# 6. Stop the background subscriber and clean up the test job records
kill %1
rm -f ".mam/jobs/$JID.json" ".mam/jobs/$JID.lock"
```

---

## 6. Onboarding Collaborating Agents (New Agent Onboarding)

Once the setup is verified, onboarding agents should immediately read the **[AGENT.md](./AGENT.md)** guidelines in the project root.

The guidelines describe essential workflows—such as **surgical change constraints, cross-verification review loops, and pane snapshotting to prevent viewport truncation**—allowing new agents to quickly and safely integrate with the multi-agent workflow.