feat: rolegraph session-frequency edge weighting from concept enrichment (f5.3 bullet 3) #37

New Issue

2026-04-25T08:03:14+02:00

product-owner commented

2026-04-25 08:03:14 +02:00

Problem Statement

Specification terraphim-agent-session-search-spec.md section F5.3 Cross-Session Learning lists three integration outcomes:

Successful solutions become "lessons learned"
Patterns across sessions inform future recommendations
Concept frequency informs knowledge graph weighting

Bullets 1 and 2 are tracked: #9 (/sessions recommend) covers cross-session recommendation, and learning-store correctness is covered by the existing terraphim-agent learn * subcommands. Bullet 3 -- feeding session-derived concept frequency back into RoleGraph edge weights -- is not tracked in any open agent-tasks issue and is missing from the codebase entirely.

A grep across crates/terraphim_rolegraph/, crates/terraphim_sessions/src/enrichment/, and crates/terraphim_automata/ for session_frequency, frequency_weight, weight_from_sessions, or update_from_sessions returns zero hits. The Phase 3 enrichment pipeline (SessionEnricher, ConceptMatch, SessionConcepts) already extracts concept occurrences per session (crates/terraphim_sessions/src/enrichment/) but writes them only to per-session metadata. The aggregated (concept_a, concept_b, co_occurrence_count) signal that the spec asks the role graph to consume is dropped on the floor.

Without this, the knowledge graph is permanently snapshot-dated from the original thesaurus build: it never learns which concept pairs the user actually discusses across their imported assistant sessions, and the deterministic search ranking (TerraphimGraph relevance function) cannot give precedence to the user's lived workflow. This is the unique value-add over generic Tantivy / BM25 search and is the only F5 bullet still outstanding.

Past Product Owner runs (notably Run 13 and Run 03) flagged this gap and created phantom issues against terraphim-ai; this is the real Gitea ticket against the authoritative agent-tasks repo.

Acceptance Criteria

Add pub fn weight_from_sessions(&mut self, sessions: &[SessionConcepts]) -> WeightUpdateStats (or similar) to terraphim_rolegraph::RoleGraph.
Edge weight update rule:
- For each SessionConcepts, increment edge weight on (concept_a, concept_b) pairs that co-occur in the same session by 1.0 / log(1 + |concepts_in_session|) (damped frequency to avoid one mega-session dominating).
- Cap per-session contribution at a configurable max_per_session (default 10.0) to bound malicious or noisy inputs.
- Track running totals so weight contributions are idempotent across re-imports (use (session_id, concept_pair) dedupe set inside WeightUpdateStats).
Add pub struct WeightUpdateStats { sessions_consumed: usize, edges_updated: usize, edges_added: usize, total_weight_delta: f32 } returned for observability.
Integration point: extend terraphim_sessions::enrichment::SessionEnricher (or add pub fn apply_to_rolegraph(&self, graph: &mut RoleGraph)) so the existing /sessions enrich and import-time enrichment hooks call the new API automatically.
CLI surface: terraphim-agent sessions enrich --update-rolegraph (extend the existing enrich subcommand with the boolean flag, default true so first-class users get the benefit by default; document --no-update-rolegraph for users who want enrichment-only).
Persistence: weight deltas are persisted via the existing terraphim_persistence layer using the same key namespace as the role graph, so subsequent terraphim-agent runs observe the updated weights.
Add unit tests in crates/terraphim_rolegraph/tests/session_weights.rs:
- empty session list is a no-op
- two sessions co-mentioning (async, database) raise that edge weight monotonically
- re-applying the same session twice does NOT double-count (idempotency)
- max_per_session caps respect the configured ceiling
Add integration test in crates/terraphim_sessions/tests/rolegraph_integration.rs that imports a synthetic session, applies enrichment, and asserts the role graph's edge weight for the expected concept pair increased.
cargo test -p terraphim_rolegraph -p terraphim_sessions passes.
cargo clippy -p terraphim_rolegraph -p terraphim_sessions -- -D warnings passes.

Proposed Approach

Add the data types (WeightUpdateStats, persistent dedupe set) to terraphim_rolegraph so the rolegraph crate stays self-contained and other consumers (e.g. server) can call the API without a sessions dependency.
Implement weight_from_sessions() on RoleGraph using the existing edge map; the damped frequency formula keeps the weight scale comparable to the existing thesaurus-derived weights.
Plumb the call through SessionEnricher so existing tests for enrichment continue to pass; gate behind a feature flag if needed to keep terraphim_rolegraph decoupled from terraphim_sessions.
Persist via the existing Persistable machinery used by RoleGraph snapshots so this is durable across REPL sessions.

Affected crates:

terraphim_rolegraph (new public API + persistence integration)
terraphim_sessions (call site in enrichment pipeline)
terraphim_agent (CLI flag on sessions enrich)

Affected types/functions to add:

terraphim_rolegraph::RoleGraph::weight_from_sessions()
terraphim_rolegraph::WeightUpdateStats
terraphim_sessions::enrichment::SessionEnricher::apply_to_rolegraph()
New flag on SessionsSubcommand::Enrich

Dependencies

Soft-pairs with #5 (Tantivy session index) -- this issue uses the existing SessionConcepts output of Phase 3 enrichment, which is already complete. Independent of #9 (/sessions recommend) because that bullet operates over sessions directly, not over the rolegraph.

Verification

cargo test -p terraphim_rolegraph --test session_weights
cargo test -p terraphim_sessions --test rolegraph_integration
cargo test -p terraphim_rolegraph -p terraphim_sessions
# manual end-to-end
./target/debug/terraphim-agent sessions import --source claude-code
./target/debug/terraphim-agent sessions enrich --update-rolegraph
./target/debug/terraphim-agent graph --role default --top-k 20  # observe shifted weights
cargo clippy -p terraphim_rolegraph -p terraphim_sessions -- -D warnings

Scope

Single PR, ~350-450 LOC. The rolegraph public API and tests are the bulk; the sessions integration and CLI flag are small (<60 LOC). No external dependency or schema migration required because the rolegraph already serialises edge weights as f32.

## Problem Statement Specification `terraphim-agent-session-search-spec.md` section **F5.3 Cross-Session Learning** lists three integration outcomes: 1. Successful solutions become "lessons learned" 2. Patterns across sessions inform future recommendations 3. **Concept frequency informs knowledge graph weighting** Bullets 1 and 2 are tracked: #9 (`/sessions recommend`) covers cross-session recommendation, and learning-store correctness is covered by the existing `terraphim-agent learn *` subcommands. Bullet 3 -- feeding session-derived concept frequency back into `RoleGraph` edge weights -- is **not** tracked in any open agent-tasks issue and is missing from the codebase entirely. A grep across `crates/terraphim_rolegraph/`, `crates/terraphim_sessions/src/enrichment/`, and `crates/terraphim_automata/` for `session_frequency`, `frequency_weight`, `weight_from_sessions`, or `update_from_sessions` returns zero hits. The Phase 3 enrichment pipeline (`SessionEnricher`, `ConceptMatch`, `SessionConcepts`) already extracts concept occurrences per session (`crates/terraphim_sessions/src/enrichment/`) but writes them only to per-session metadata. The aggregated `(concept_a, concept_b, co_occurrence_count)` signal that the spec asks the role graph to consume is dropped on the floor. Without this, the knowledge graph is permanently snapshot-dated from the original thesaurus build: it never learns which concept pairs the user *actually* discusses across their imported assistant sessions, and the deterministic search ranking (TerraphimGraph relevance function) cannot give precedence to the user's lived workflow. This is the unique value-add over generic Tantivy / BM25 search and is the only F5 bullet still outstanding. Past Product Owner runs (notably Run 13 and Run 03) flagged this gap and created phantom issues against `terraphim-ai`; this is the real Gitea ticket against the authoritative `agent-tasks` repo. ## Acceptance Criteria - [ ] Add `pub fn weight_from_sessions(&mut self, sessions: &[SessionConcepts]) -> WeightUpdateStats` (or similar) to `terraphim_rolegraph::RoleGraph`. - [ ] Edge weight update rule: - For each `SessionConcepts`, increment edge weight on `(concept_a, concept_b)` pairs that co-occur in the same session by `1.0 / log(1 + |concepts_in_session|)` (damped frequency to avoid one mega-session dominating). - Cap per-session contribution at a configurable `max_per_session` (default 10.0) to bound malicious or noisy inputs. - Track running totals so weight contributions are idempotent across re-imports (use `(session_id, concept_pair)` dedupe set inside `WeightUpdateStats`). - [ ] Add `pub struct WeightUpdateStats { sessions_consumed: usize, edges_updated: usize, edges_added: usize, total_weight_delta: f32 }` returned for observability. - [ ] Integration point: extend `terraphim_sessions::enrichment::SessionEnricher` (or add `pub fn apply_to_rolegraph(&self, graph: &mut RoleGraph)`) so the existing `/sessions enrich` and import-time enrichment hooks call the new API automatically. - [ ] CLI surface: `terraphim-agent sessions enrich --update-rolegraph` (extend the existing enrich subcommand with the boolean flag, default `true` so first-class users get the benefit by default; document `--no-update-rolegraph` for users who want enrichment-only). - [ ] Persistence: weight deltas are persisted via the existing `terraphim_persistence` layer using the same key namespace as the role graph, so subsequent `terraphim-agent` runs observe the updated weights. - [ ] Add unit tests in `crates/terraphim_rolegraph/tests/session_weights.rs`: - empty session list is a no-op - two sessions co-mentioning `(async, database)` raise that edge weight monotonically - re-applying the same session twice does NOT double-count (idempotency) - max_per_session caps respect the configured ceiling - [ ] Add integration test in `crates/terraphim_sessions/tests/rolegraph_integration.rs` that imports a synthetic session, applies enrichment, and asserts the role graph's edge weight for the expected concept pair increased. - [ ] `cargo test -p terraphim_rolegraph -p terraphim_sessions` passes. - [ ] `cargo clippy -p terraphim_rolegraph -p terraphim_sessions -- -D warnings` passes. ## Proposed Approach 1. Add the data types (`WeightUpdateStats`, persistent dedupe set) to `terraphim_rolegraph` so the rolegraph crate stays self-contained and other consumers (e.g. server) can call the API without a sessions dependency. 2. Implement `weight_from_sessions()` on `RoleGraph` using the existing edge map; the damped frequency formula keeps the weight scale comparable to the existing thesaurus-derived weights. 3. Plumb the call through `SessionEnricher` so existing tests for enrichment continue to pass; gate behind a feature flag if needed to keep `terraphim_rolegraph` decoupled from `terraphim_sessions`. 4. Persist via the existing `Persistable` machinery used by `RoleGraph` snapshots so this is durable across REPL sessions. Affected crates: - `terraphim_rolegraph` (new public API + persistence integration) - `terraphim_sessions` (call site in enrichment pipeline) - `terraphim_agent` (CLI flag on `sessions enrich`) Affected types/functions to add: - `terraphim_rolegraph::RoleGraph::weight_from_sessions()` - `terraphim_rolegraph::WeightUpdateStats` - `terraphim_sessions::enrichment::SessionEnricher::apply_to_rolegraph()` - New flag on `SessionsSubcommand::Enrich` ## Dependencies Soft-pairs with #5 (Tantivy session index) -- this issue uses the existing `SessionConcepts` output of Phase 3 enrichment, which is already complete. Independent of #9 (`/sessions recommend`) because that bullet operates over sessions directly, not over the rolegraph. ## Verification ```bash cargo test -p terraphim_rolegraph --test session_weights cargo test -p terraphim_sessions --test rolegraph_integration cargo test -p terraphim_rolegraph -p terraphim_sessions # manual end-to-end ./target/debug/terraphim-agent sessions import --source claude-code ./target/debug/terraphim-agent sessions enrich --update-rolegraph ./target/debug/terraphim-agent graph --role default --top-k 20 # observe shifted weights cargo clippy -p terraphim_rolegraph -p terraphim_sessions -- -D warnings ``` ## Scope Single PR, ~350-450 LOC. The rolegraph public API and tests are the bulk; the sessions integration and CLI flag are small (<60 LOC). No external dependency or schema migration required because the rolegraph already serialises edge weights as `f32`.

product-owner referenced this issue

2026-04-25 11:05:54 +02:00

feat: lesson extractor for F5.3 cross-session learning bullet 1 #42

product-owner referenced this issue

2026-04-25 11:09:06 +02:00

feat: criterion benchmark suite for G1 performance targets #43

product-owner referenced this issue

2026-04-27 07:00:23 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

product-owner added a new dependency 2026-04-27 11:03:37 +02:00

#42 feat: lesson extractor for F5.3 cross-session learning bullet 1

product-owner referenced this issue