feat: lesson extractor for F5.3 cross-session learning bullet 1 #42

New Issue

2026-04-25T11:05:54+02:00

product-owner commented

2026-04-25 11:05:54 +02:00

Problem Statement

Spec section F5.3 Cross-Session Learning in docs/specifications/terraphim-agent-session-search-spec.md lists three bullets:

Successful solutions become lessons learned -- when a session contains a fail→fix pattern, the fix should be auto-captured as a CapturedLearning correction so the next time the failing command appears, the agent already knows the answer.
Recommendation engine for similar past sessions (tracked in #9 /sessions recommend).
Knowledge graph weighted by session frequency (tracked in #37 rolegraph weighting).

Bullets 2 and 3 have issues. Bullet 1 has none. No code path scans imported sessions for paired failure-then-success commands and emits CapturedLearning records with corrections pre-filled. Today, learnings are only captured live via the PostToolUse hook (terraphim-agent learn hook) -- past sessions imported via sessions import contain rich fail→fix history that goes unmined.

Verification:

rg -n "lesson_extractor|extract_lessons|fail.*then.*fix|fix_after_fail" crates/terraphim_sessions/   -> 0 matches
rg -n "lesson_extractor|extract_lessons" crates/terraphim_agent/                                     -> 0 matches
rg -n "fail.*fix|extract.*correction" crates/terraphim_agent/src/learnings/                          -> 0 matches

Spec text (verbatim, F5.3):

Successful solutions become lessons learned

This is the highest-leverage F5.3 bullet because it bootstraps the entire learning store from existing session history rather than requiring users to re-encounter every failure live.

Acceptance Criteria

New module crates/terraphim_agent/src/learnings/lesson_extractor.rs exposing:

pub struct ExtractedLesson {
    pub failed_command: String,
    pub failed_at: DateTime<Utc>,
    pub correction: String,        // command that succeeded after the failure
    pub corrected_at: DateTime<Utc>,
    pub session_id: String,
    pub confidence: f32,           // 0..1, see scoring rules below
}

pub fn extract_lessons_from_session(session: &Session) -> Vec<ExtractedLesson>;
pub async fn extract_lessons_from_all_sessions(store: &SessionStore) -> Result<Vec<ExtractedLesson>>;

Pair detection rules (deterministic, no LLM):
- A BashTool invocation with non-zero exit code (or stderr containing error:/Error:/fatal:) followed within the same session by a successful BashTool invocation whose command shares a normalised prefix (first token, e.g. npm/bun/cargo) is a candidate pair.
- Confidence = 1.0 if the second command runs within 5 minutes of the first, 0.7 if 5-30 min, 0.4 if 30+ min, 0.0 discarded if a different command-class succeeded between them.
- Discard pairs where failed and corrected commands are byte-identical (no actual learning).
New REPL command /sessions extract-lessons [--session ID] [--min-confidence 0.5] [--dry-run]:
- Without --dry-run, calls LearningStore::add_correction() for each extracted lesson with confidence ≥ threshold.
- With --dry-run, prints proposed lessons as a table.
CLI subcommand mirror: terraphim-agent learn extract-lessons [--session ID] [--min-confidence 0.5] [--dry-run].
When extraction inserts a correction, the resulting CapturedLearning.notes field gets extracted-from-session:<session-id> prefix so a reviewer can trace provenance.
Idempotency: re-running extraction on the same session does not duplicate corrections (check (failed_command, session_id) already present in the store before inserting).
Unit tests:
- Session with one fail→fix pair (npm install fail → bun install success) produces one lesson at confidence 1.0.
- Session with fail then unrelated success then matching success produces zero lessons (intervening success disqualifies).
- Session with fail→fix at 10 min gap produces one lesson at confidence 0.7.
- Session with fail→fix at 45 min gap produces one lesson at confidence 0.4.
- Idempotency test: extract twice, store has one entry.
Integration test: import a fixture .jsonl session under crates/terraphim_sessions/fixtures/, run extractor, assert expected lessons.
cargo test -p terraphim_agent learnings::lesson_extractor passes.
cargo clippy -p terraphim_agent -- -D warnings passes.

Proposed Approach

Crates affected: crates/terraphim_agent (new module + REPL/CLI wiring), read-only access to crates/terraphim_sessions.

Files to modify / add:

crates/terraphim_agent/src/learnings/lesson_extractor.rs (new, ~150 LoC)
crates/terraphim_agent/src/learnings/mod.rs (add pub mod lesson_extractor)
crates/terraphim_agent/src/repl/commands.rs (add ExtractLessons to SessionsSubcommand enum and dispatch arm)
crates/terraphim_agent/src/main.rs (add ExtractLessons to LearnSub enum and dispatch arm)
crates/terraphim_agent/tests/lesson_extractor_test.rs (new integration test, ~80 LoC)
crates/terraphim_sessions/fixtures/fail_then_fix_npm_to_bun.jsonl (new test fixture, ~20 lines)

Why this approach: Pure-read pass over already-imported sessions, deterministic confidence scoring, no LLM calls, no new deps. Reuses existing LearningStore::add_correction() so corrections appear in learn list / learn query immediately.

Dependencies

Pairs with #9 (/sessions recommend) and #37 (rolegraph session weighting) to complete F5.3 (3/3 bullets covered).
Independent of #5 (Tantivy index) -- the extractor scans raw session JSONL, not the index.
Lesson capture format compatible with #41 (null-byte truncation) -- since we only extract command strings, not stderr blobs, null-byte handling is not relevant here.

Verification

# Unit tests
cargo test -p terraphim_agent learnings::lesson_extractor

# Integration test (fixture-driven)
cargo test -p terraphim_agent --test lesson_extractor_test

# Clippy
cargo clippy -p terraphim_agent -- -D warnings

# Manual smoke test
./target/debug/terraphim-agent sessions import
./target/debug/terraphim-agent learn extract-lessons --dry-run
./target/debug/terraphim-agent learn extract-lessons --min-confidence 0.7
./target/debug/terraphim-agent learn list   # expect new entries with [extracted-from-session:*] notes

Scope

~150 LoC core + ~80 LoC tests + ~30 LoC REPL/CLI wiring + ~20 lines fixture = under 300 lines total. Single PR, well under the 500-line cap.

Upstream: This issue tracks a feature for terraphim/terraphim-ai (mirror in terraphim/agent-tasks because the canonical repo currently 404s).

## Problem Statement Spec section **F5.3 Cross-Session Learning** in `docs/specifications/terraphim-agent-session-search-spec.md` lists three bullets: 1. **Successful solutions become lessons learned** -- when a session contains a fail→fix pattern, the fix should be auto-captured as a `CapturedLearning` correction so the *next* time the failing command appears, the agent already knows the answer. 2. Recommendation engine for similar past sessions (tracked in #9 `/sessions recommend`). 3. Knowledge graph weighted by session frequency (tracked in #37 rolegraph weighting). Bullets 2 and 3 have issues. **Bullet 1 has none.** No code path scans imported sessions for paired failure-then-success commands and emits `CapturedLearning` records with corrections pre-filled. Today, learnings are only captured live via the `PostToolUse` hook (`terraphim-agent learn hook`) -- past sessions imported via `sessions import` contain rich fail→fix history that goes unmined. Verification: ``` rg -n "lesson_extractor|extract_lessons|fail.*then.*fix|fix_after_fail" crates/terraphim_sessions/ -> 0 matches rg -n "lesson_extractor|extract_lessons" crates/terraphim_agent/ -> 0 matches rg -n "fail.*fix|extract.*correction" crates/terraphim_agent/src/learnings/ -> 0 matches ``` Spec text (verbatim, F5.3): > Successful solutions become lessons learned This is the highest-leverage F5.3 bullet because it bootstraps the entire learning store from existing session history rather than requiring users to re-encounter every failure live. ## Acceptance Criteria - [ ] New module `crates/terraphim_agent/src/learnings/lesson_extractor.rs` exposing: ```rust pub struct ExtractedLesson { pub failed_command: String, pub failed_at: DateTime<Utc>, pub correction: String, // command that succeeded after the failure pub corrected_at: DateTime<Utc>, pub session_id: String, pub confidence: f32, // 0..1, see scoring rules below } pub fn extract_lessons_from_session(session: &Session) -> Vec<ExtractedLesson>; pub async fn extract_lessons_from_all_sessions(store: &SessionStore) -> Result<Vec<ExtractedLesson>>; ``` - [ ] Pair detection rules (deterministic, no LLM): - A `BashTool` invocation with non-zero exit code (or stderr containing `error:`/`Error:`/`fatal:`) followed within the same session by a successful `BashTool` invocation whose command shares a normalised prefix (first token, e.g. `npm`/`bun`/`cargo`) is a candidate pair. - Confidence = `1.0` if the second command runs within 5 minutes of the first, `0.7` if 5-30 min, `0.4` if 30+ min, `0.0` discarded if a different command-class succeeded between them. - Discard pairs where failed and corrected commands are byte-identical (no actual learning). - [ ] New REPL command `/sessions extract-lessons [--session ID] [--min-confidence 0.5] [--dry-run]`: - Without `--dry-run`, calls `LearningStore::add_correction()` for each extracted lesson with confidence ≥ threshold. - With `--dry-run`, prints proposed lessons as a table. - [ ] CLI subcommand mirror: `terraphim-agent learn extract-lessons [--session ID] [--min-confidence 0.5] [--dry-run]`. - [ ] When extraction inserts a correction, the resulting `CapturedLearning.notes` field gets `extracted-from-session:<session-id>` prefix so a reviewer can trace provenance. - [ ] Idempotency: re-running extraction on the same session does not duplicate corrections (check `(failed_command, session_id)` already present in the store before inserting). - [ ] Unit tests: - Session with one fail→fix pair (npm install fail → bun install success) produces one lesson at confidence 1.0. - Session with fail then unrelated success then matching success produces zero lessons (intervening success disqualifies). - Session with fail→fix at 10 min gap produces one lesson at confidence 0.7. - Session with fail→fix at 45 min gap produces one lesson at confidence 0.4. - Idempotency test: extract twice, store has one entry. - [ ] Integration test: import a fixture `.jsonl` session under `crates/terraphim_sessions/fixtures/`, run extractor, assert expected lessons. - [ ] `cargo test -p terraphim_agent learnings::lesson_extractor` passes. - [ ] `cargo clippy -p terraphim_agent -- -D warnings` passes. ## Proposed Approach **Crates affected:** `crates/terraphim_agent` (new module + REPL/CLI wiring), read-only access to `crates/terraphim_sessions`. **Files to modify / add:** - `crates/terraphim_agent/src/learnings/lesson_extractor.rs` (new, ~150 LoC) - `crates/terraphim_agent/src/learnings/mod.rs` (add `pub mod lesson_extractor`) - `crates/terraphim_agent/src/repl/commands.rs` (add `ExtractLessons` to `SessionsSubcommand` enum and dispatch arm) - `crates/terraphim_agent/src/main.rs` (add `ExtractLessons` to `LearnSub` enum and dispatch arm) - `crates/terraphim_agent/tests/lesson_extractor_test.rs` (new integration test, ~80 LoC) - `crates/terraphim_sessions/fixtures/fail_then_fix_npm_to_bun.jsonl` (new test fixture, ~20 lines) **Why this approach:** Pure-read pass over already-imported sessions, deterministic confidence scoring, no LLM calls, no new deps. Reuses existing `LearningStore::add_correction()` so corrections appear in `learn list` / `learn query` immediately. ## Dependencies - Pairs with #9 (`/sessions recommend`) and #37 (rolegraph session weighting) to complete F5.3 (3/3 bullets covered). - Independent of #5 (Tantivy index) -- the extractor scans raw session JSONL, not the index. - Lesson capture format compatible with #41 (null-byte truncation) -- since we only extract command strings, not stderr blobs, null-byte handling is not relevant here. ## Verification ```bash # Unit tests cargo test -p terraphim_agent learnings::lesson_extractor # Integration test (fixture-driven) cargo test -p terraphim_agent --test lesson_extractor_test # Clippy cargo clippy -p terraphim_agent -- -D warnings # Manual smoke test ./target/debug/terraphim-agent sessions import ./target/debug/terraphim-agent learn extract-lessons --dry-run ./target/debug/terraphim-agent learn extract-lessons --min-confidence 0.7 ./target/debug/terraphim-agent learn list # expect new entries with [extracted-from-session:*] notes ``` ## Scope ~150 LoC core + ~80 LoC tests + ~30 LoC REPL/CLI wiring + ~20 lines fixture = under 300 lines total. Single PR, well under the 500-line cap. Upstream: This issue tracks a feature for terraphim/terraphim-ai (mirror in `terraphim/agent-tasks` because the canonical repo currently 404s).

product-owner referenced this issue

2026-04-25 11:06:25 +02:00

feat: criterion benchmark suite for G1 performance targets #43

product-owner referenced this issue

2026-04-25 11:09:06 +02:00

feat: criterion benchmark suite for G1 performance targets #43

product-owner referenced this issue

2026-04-27 07:00:23 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

product-owner added a new dependency 2026-04-27 11:03:37 +02:00

#37 feat: rolegraph session-frequency edge weighting from concept enrichment (f5.3 bullet 3)

product-owner added a new dependency 2026-04-27 11:03:37 +02:00