130 lines
5.3 KiB
Markdown
130 lines
5.3 KiB
Markdown
# Test Ready Report
|
|
|
|
## Test Runner
|
|
- **Command**: `.venv/bin/pytest tests/`
|
|
- **Expected**: All tests pass with exit code 0
|
|
|
|
## Coverage Summary
|
|
- **1. Feature Coverage (Tier 1)**: 29 tests
|
|
- **2. Boundary & Corner (Tier 2)**: 26 tests
|
|
- **3. Cross-Feature (Tier 3)**: 5 tests
|
|
- **4. Real-World Application (Tier 4)**: 5 tests
|
|
- **Sanity Checks**: 2 tests
|
|
- **Challenger/M2 Unit**: 7 tests
|
|
- **Total**: 74 tests
|
|
|
|
## Feature Checklist
|
|
|
|
### 1. Create Session
|
|
- **Tier 1 (Unit Checks)**
|
|
- [x] `derive_session_name` slug generation checks
|
|
- [x] Workspace-to-slug character translation
|
|
- [x] Invalid workspace path filtering
|
|
- [x] Role parameter sanity validations
|
|
- [x] Session override string generation
|
|
- **Tier 2 (Component Checks)**
|
|
- [x] Verification of state serialization to YAML schema
|
|
- [x] Isolation home directory structure validation
|
|
- [x] SQLite DB connection verification
|
|
- [x] Concurrency check for database registration lock
|
|
- [x] Database schema validation on write
|
|
- **Tier 3 (Integration Checks)**
|
|
- [x] Spawn session execution with mock `herdr` and mock agent
|
|
- [x] TUI readiness wait check
|
|
- [x] Cleanup trap execution on crash
|
|
- [x] Argument validation logic verification
|
|
- [x] Isolation directory creation checks
|
|
- **Tier 4 (Real-World Application Scenarios)**
|
|
- [x] Standard Agent Session Lifecycle E2E test (Scenario 1)
|
|
- [x] Parallel Session Operations with flock Locking E2E test (Scenario 4)
|
|
|
|
### 2. Resume Session
|
|
- **Tier 1 (Unit Checks)**
|
|
- [x] Workspace UUID resolution order unit tests
|
|
- [x] CLI session ID parser validations
|
|
- [x] Check prioritization (yaml file -> disk scan -> cache)
|
|
- [x] Workspace path boundary check
|
|
- [x] Empty UUID handling logic
|
|
- **Tier 2 (Component Checks)**
|
|
- [x] Configuration restore verification
|
|
- [x] Environment overrides assertion
|
|
- [x] Integrity check on retrieved SQLite metadata
|
|
- [x] Validation of session ownership verification
|
|
- [x] Config parsing for resume options
|
|
- **Tier 3 (Integration Checks)**
|
|
- [x] Run `resume_session.sh` with mock agents
|
|
- [x] Intercept agent command structure inside mock `herdr`
|
|
- [x] Verify agent receives correct conversation UUID flag
|
|
- [x] Invalid/missing UUID recovery path test
|
|
- [x] Workspace resume CLI args verification
|
|
- **Tier 4 (Real-World Application Scenarios)**
|
|
- [x] Session Disconnect and Resume E2E test (Scenario 2)
|
|
|
|
### 3. Stop Session
|
|
- **Tier 1 (Unit Checks)**
|
|
- [x] Session name verification check
|
|
- [x] Purge verification confirmations logic
|
|
- [x] Command derivation format validation
|
|
- [x] Timeout calculation helper tests
|
|
- [x] Reason logging serializer test
|
|
- **Tier 2 (Component Checks)**
|
|
- [x] Safe folder path validation (shutil protection)
|
|
- [x] Database status field mutation serialization
|
|
- [x] Isolation folder cleanup check
|
|
- [x] Lock file release checks on stop
|
|
- [x] Concurrency handling of stop mutations
|
|
- **Tier 3 (Integration Checks)**
|
|
- [x] Execute `stop_session.sh` with graceful key delivery (`/exit`)
|
|
- [x] Fallback to forcible termination (`herdr kill-session`) check
|
|
- [x] Fallback to PID termination (`kill -9`) verify
|
|
- [x] Purge files verification on disk (`--purge-conversation`)
|
|
- [x] CLI flag verification with yes/no confirmation
|
|
- **Tier 4 (Real-World Application Scenarios)**
|
|
- [x] Standard Agent Session Lifecycle E2E test (Scenario 1)
|
|
- [x] Parallel Session Operations with flock Locking E2E test (Scenario 4)
|
|
|
|
### 4. Status Query
|
|
- **Tier 1 (Unit Checks)**
|
|
- [x] JSON converter unit tests
|
|
- [x] Diff formatter text generators
|
|
- [x] Output alignment tests
|
|
- [x] Table grid column math verify
|
|
- [x] CLI status argument parse tests
|
|
- **Tier 2 (Component Checks)**
|
|
- [x] Status read locks verification
|
|
- [x] Parsing of drift status classifications
|
|
- [x] Concurrency read protection test
|
|
- [x] Registry YAML-to-JSON structural translation
|
|
- [x] Verification of database read access checks
|
|
- **Tier 3 (Integration Checks)**
|
|
- [x] Running `status.sh` with `--json`
|
|
- [x] Verify console output match formatting rules
|
|
- [x] Verify exit status codes on different states
|
|
- [x] Integration test with `reconcile.sh` read-only diff emission
|
|
- [x] Verify status command doesn't trigger side effects
|
|
- **Tier 4 (Real-World Application Scenarios)**
|
|
- [x] Standard Agent Session Lifecycle E2E test (Scenario 1)
|
|
- [x] Drift Detection and Auto-Reconciliation E2E test (Scenario 3)
|
|
|
|
### 5. Monitor/Reconcile
|
|
- **Tier 1 (Unit Checks)**
|
|
- [x] Drift state classification unit tests
|
|
- [x] Signature verification checks
|
|
- [x] Subscription topic parsing tests
|
|
- [x] MQTT message structure validator
|
|
- [x] HMAC validation logic tests
|
|
- **Tier 2 (Component Checks)**
|
|
- [x] Concurrency lock checks (`.mam/monitor.lock`)
|
|
- [x] Verify YAML and SQLite database reconciliation logic
|
|
- [x] DB validation on drift updates
|
|
- [x] HMAC signature signature verification
|
|
- [x] SQLite journal mode fallback check (WAL vs DELETE)
|
|
- **Tier 3 (Integration Checks)**
|
|
- [x] Execute `reconcile.sh` in single-pass mode (`--once`)
|
|
- [x] MQTT subscription execution with mock messages
|
|
- [x] Verify auto-termination of orphaned herdr sessions
|
|
- [x] Verify auto-registration of untracked herdr sessions
|
|
- [x] Lock contention handling testing
|
|
- **Tier 4 (Real-World Application Scenarios)**
|
|
- [x] Drift Detection and Auto-Reconciliation E2E test (Scenario 3)
|