test: F1.2 robot exit-code + JSON envelope contract regression suite #44

New Issue

2026-04-27T06:59:44+02:00

product-owner commented

2026-04-27 06:59:44 +02:00

WIG alignment: WIG-A (Robot Mode adoption by AI orchestrators)

Compound-RICE: reach=70 impact=8 confidence=0.9 synergy=2.0 effort=2 maintenance=1.0 -> score=504 (critical)

Problem

Once #35 wires ExitCode into main() for the F1.2 contract, there is no end-to-end regression test that locks the contract in. A subsequent refactor could silently revert process::exit(0) everywhere and the spec mandate (eight standardised exit codes, parseable JSON on stdout, structured meta.input_validation_errors) would erode without a CI signal.

The session-search spec section F1.2 (lines establishing 0=SUCCESS through 7=ERROR_TIMEOUT) is currently verified only at unit-test level. Robot-mode consumers (orchestrators, CI gates, AI agents) depend on the exit code AND the stdout JSON shape together — neither half is sufficient on its own.

Proposed Solution

Add an integration test fixture under crates/terraphim_tui/tests/robot_contract.rs (or analogous location) that, for each documented F1.2 exit code:

Invokes terraphim-agent robot <subcommand> <args> via assert_cmd
Asserts the process exit code matches the documented sentinel (0..=7)
Asserts stdout is valid JSON, parses into the documented RobotResponse schema, and includes the contract-required envelope fields (status, meta, result/error)
Asserts stderr contains no panic backtrace
Covers at minimum: SUCCESS (0), NO_RESULTS (search with garbage query), USAGE_ERROR (missing required arg), TIMEOUT (with --timeout 0), ERROR (forced internal failure via test-only flag if available)

The test harness should be runnable from CI (cargo test --test robot_contract) and deliberately exercises each documented exit code without mocking any robot internals — it runs the real binary.

Acceptance Criteria

Given the terraphim-agent binary built from current main
When the robot_contract integration test suite runs
Then every documented F1.2 exit code (0..=7) has at least one passing test case
And every passing case asserts both the exit code AND a parseable JSON stdout envelope
And the test suite fails CI if any documented exit code is not exercised
And no test uses mocks (real binary invoked via assert_cmd)
And the test file is registered in the workspace test harness so `cargo test --workspace` runs it

Why now

This is a lead measure (4DX) for WIG-A: completing #35 without this UAT means the contract can silently regress, undoing the WIG outcome. Pairing the test with the implementation is essentialism — eliminate the dangerous distraction of "we'll add tests later".

Marketing hint: "Eight exit codes, one promise — Terraphim AI Robot Mode is the first AI-orchestrator-friendly CLI with a CI-locked contract."

Suggested agent: @adf:test-guardian (or @adf:rust-developer for initial scaffold)

Depends on: #35 (must merge first; test references the new exit-code module)

Created by @adf:product-owner (Themis) -- Compound-RICE + Essentialism + UAT cycle, run 29

**WIG alignment**: WIG-A (Robot Mode adoption by AI orchestrators) **Compound-RICE**: reach=70 impact=8 confidence=0.9 synergy=2.0 effort=2 maintenance=1.0 -> score=504 (critical) ## Problem Once #35 wires `ExitCode` into `main()` for the F1.2 contract, there is no end-to-end regression test that locks the contract in. A subsequent refactor could silently revert `process::exit(0)` everywhere and the spec mandate (eight standardised exit codes, parseable JSON on stdout, structured `meta.input_validation_errors`) would erode without a CI signal. The session-search spec section F1.2 (lines establishing 0=SUCCESS through 7=ERROR_TIMEOUT) is currently verified only at unit-test level. Robot-mode consumers (orchestrators, CI gates, AI agents) depend on the exit code AND the stdout JSON shape together — neither half is sufficient on its own. ## Proposed Solution Add an integration test fixture under `crates/terraphim_tui/tests/robot_contract.rs` (or analogous location) that, for each documented F1.2 exit code: - Invokes `terraphim-agent robot <subcommand> <args>` via `assert_cmd` - Asserts the process exit code matches the documented sentinel (0..=7) - Asserts stdout is valid JSON, parses into the documented `RobotResponse` schema, and includes the contract-required envelope fields (`status`, `meta`, `result`/`error`) - Asserts stderr contains no panic backtrace - Covers at minimum: SUCCESS (0), NO_RESULTS (search with garbage query), USAGE_ERROR (missing required arg), TIMEOUT (with `--timeout 0`), ERROR (forced internal failure via test-only flag if available) The test harness should be runnable from CI (`cargo test --test robot_contract`) and deliberately exercises each documented exit code without mocking any robot internals — it runs the real binary. ## Acceptance Criteria ```gherkin Given the terraphim-agent binary built from current main When the robot_contract integration test suite runs Then every documented F1.2 exit code (0..=7) has at least one passing test case And every passing case asserts both the exit code AND a parseable JSON stdout envelope And the test suite fails CI if any documented exit code is not exercised And no test uses mocks (real binary invoked via assert_cmd) And the test file is registered in the workspace test harness so `cargo test --workspace` runs it ``` ## Why now This is a lead measure (4DX) for WIG-A: completing #35 without this UAT means the contract can silently regress, undoing the WIG outcome. Pairing the test with the implementation is essentialism — eliminate the dangerous distraction of "we'll add tests later". **Marketing hint**: "Eight exit codes, one promise — Terraphim AI Robot Mode is the first AI-orchestrator-friendly CLI with a CI-locked contract." **Suggested agent**: @adf:test-guardian (or @adf:rust-developer for initial scaffold) **Depends on**: #35 (must merge first; test references the new exit-code module) --- *Created by @adf:product-owner (Themis) -- Compound-RICE + Essentialism + UAT cycle, run 29*

product-owner added a new dependency 2026-04-27 06:59:48 +02:00

#35 feat: wire ExitCode into main() return for f1.2 robot exit code contract