feat: CodeSnippet extraction and Message.snippets in session model (F4.2) #39

New Issue

2026-04-25T10:01:09+02:00

product-owner commented

2026-04-25 10:01:09 +02:00

Problem Statement

Spec section F4.2 Session Data Model defines a first-class CodeSnippet type alongside Message.snippets: Vec<CodeSnippet>:

pub struct CodeSnippet {
    pub language: Option<String>,
    pub content: String,
    pub file_path: Option<String>,
    pub line_range: Option<(usize, usize)>,
}

This shape is what F4.3 relies on: the Tantivy code_content field (separate from content) needs canonical code blocks, and the spec's edge-n-gram tokenizer for code patterns is meaningless without them. Issue #5 ("Tantivy session index") therefore depends on this data being present.

Today, crates/terraphim_sessions/src/model.rs lacks a CodeSnippet struct entirely. Message carries blocks: Vec<ContentBlock> where Text { text } blends prose and fenced code, and ToolUse/ToolResult lose code structure. Connectors (Aider markdown, Claude JSONL, Cursor SQLite, Cline JSON) all parse code fences, throw the language tag away, and append the raw text into content. Downstream concept enrichment (SessionEnricher) sees no language signal, and the planned code_content index has nothing to consume.

Acceptance Criteria

Add pub struct CodeSnippet in crates/terraphim_sessions/src/model.rs with the four fields above plus Serialize/Deserialize/Debug/Clone
Add Message.snippets: Vec<CodeSnippet> (default empty via #[serde(default)]) without removing blocks
Add helper Message::extract_snippets_from_blocks(&self) -> Vec<CodeSnippet> that pulls fenced code from Text blocks (use existing terraphim_markdown_parser for fence parsing)
Wire NativeClaudeConnector, AiderConnector, and ClineConnector to populate snippets after parsing each message
code_blocks: bool in SessionMetadata (already present as has_code in spec) becomes metadata.has_code = !snippets.is_empty() in the connectors that didn't set it
cargo test -p terraphim_sessions model::tests::code_snippet* passes (fence parsing, language detection for rust/python/none)
cargo test -p terraphim_sessions passes
cargo clippy -p terraphim_sessions -- -D warnings passes

Proposed Approach

Crates affected: crates/terraphim_sessions only. No public API breakage outside the workspace because the field is additive with #[serde(default)].

Files to add/modify:

crates/terraphim_sessions/src/model.rs -- add CodeSnippet, extend Message, add extraction helper
crates/terraphim_sessions/src/connector/native.rs -- populate snippets when parsing Claude JSONL text content
crates/terraphim_sessions/src/connector/aider.rs -- already calls terraphim_markdown_parser; capture fenced blocks into CodeSnippet
crates/terraphim_sessions/src/connector/cline.rs -- detect code fences in message content (Cline messages are markdown)

Key extraction logic: Use a regex like (?s)```(\w*)\n(.*?)```` to split fenced blocks; first capture is the language tag (empty -> None), second is the content. file_pathandline_rangestayNone` for now -- they require connector-specific tool-use parsing which is a follow-up.

Dependencies

None. Blocks #5 (Tantivy index) -- when this lands, #5 can index code_content from Message.snippets[].content joined per session.

After merge, run gitea-robot add-dep --owner terraphim --repo agent-tasks --issue 5 --blocks <THIS> to record the edge.

Verification

cargo test -p terraphim_sessions model
cargo test -p terraphim_sessions connector::native
cargo test -p terraphim_sessions connector::aider --features aider-connector
cargo clippy -p terraphim_sessions -- -D warnings

Manual smoke test:

terraphim-agent sessions import --source claude
terraphim-agent sessions show <id> --format json | jq '.messages[].snippets'
# Expect non-empty arrays for messages with fenced code

Scope

Single PR, ~300-400 lines including tests and connector wiring. Do not change the Tantivy index, do not add file_path/line_range connector parsing -- both are explicit follow-ups to keep this PR reviewable.

## Problem Statement Spec section **F4.2 Session Data Model** defines a first-class `CodeSnippet` type alongside `Message.snippets: Vec<CodeSnippet>`: ```rust pub struct CodeSnippet { pub language: Option<String>, pub content: String, pub file_path: Option<String>, pub line_range: Option<(usize, usize)>, } ``` This shape is what F4.3 relies on: the Tantivy `code_content` field (separate from `content`) needs canonical code blocks, and the spec's edge-n-gram tokenizer for code patterns is meaningless without them. Issue #5 ("Tantivy session index") therefore depends on this data being present. Today, `crates/terraphim_sessions/src/model.rs` lacks a `CodeSnippet` struct entirely. `Message` carries `blocks: Vec<ContentBlock>` where `Text { text }` blends prose and fenced code, and `ToolUse`/`ToolResult` lose code structure. Connectors (Aider markdown, Claude JSONL, Cursor SQLite, Cline JSON) all parse code fences, throw the language tag away, and append the raw text into `content`. Downstream concept enrichment (`SessionEnricher`) sees no language signal, and the planned `code_content` index has nothing to consume. ## Acceptance Criteria - [ ] Add `pub struct CodeSnippet` in `crates/terraphim_sessions/src/model.rs` with the four fields above plus `Serialize`/`Deserialize`/`Debug`/`Clone` - [ ] Add `Message.snippets: Vec<CodeSnippet>` (default empty via `#[serde(default)]`) without removing `blocks` - [ ] Add helper `Message::extract_snippets_from_blocks(&self) -> Vec<CodeSnippet>` that pulls fenced code from `Text` blocks (use existing `terraphim_markdown_parser` for fence parsing) - [ ] Wire `NativeClaudeConnector`, `AiderConnector`, and `ClineConnector` to populate `snippets` after parsing each message - [ ] `code_blocks: bool` in `SessionMetadata` (already present as `has_code` in spec) becomes `metadata.has_code = !snippets.is_empty()` in the connectors that didn't set it - [ ] `cargo test -p terraphim_sessions model::tests::code_snippet*` passes (fence parsing, language detection for `rust`/`python`/none) - [ ] `cargo test -p terraphim_sessions` passes - [ ] `cargo clippy -p terraphim_sessions -- -D warnings` passes ## Proposed Approach **Crates affected:** `crates/terraphim_sessions` only. No public API breakage outside the workspace because the field is additive with `#[serde(default)]`. **Files to add/modify:** - `crates/terraphim_sessions/src/model.rs` -- add `CodeSnippet`, extend `Message`, add extraction helper - `crates/terraphim_sessions/src/connector/native.rs` -- populate snippets when parsing Claude JSONL `text` content - `crates/terraphim_sessions/src/connector/aider.rs` -- already calls `terraphim_markdown_parser`; capture fenced blocks into `CodeSnippet` - `crates/terraphim_sessions/src/connector/cline.rs` -- detect code fences in message content (Cline messages are markdown) **Key extraction logic:** Use a regex like `(?s)```(\w*)\n(.*?)```` to split fenced blocks; first capture is the language tag (empty -> `None`), second is the content. `file_path` and `line_range` stay `None` for now -- they require connector-specific tool-use parsing which is a follow-up. ## Dependencies None. Blocks #5 (Tantivy index) -- when this lands, #5 can index `code_content` from `Message.snippets[].content` joined per session. After merge, run `gitea-robot add-dep --owner terraphim --repo agent-tasks --issue 5 --blocks <THIS>` to record the edge. ## Verification ``` cargo test -p terraphim_sessions model cargo test -p terraphim_sessions connector::native cargo test -p terraphim_sessions connector::aider --features aider-connector cargo clippy -p terraphim_sessions -- -D warnings ``` Manual smoke test: ``` terraphim-agent sessions import --source claude terraphim-agent sessions show <id> --format json | jq '.messages[].snippets' # Expect non-empty arrays for messages with fenced code ``` ## Scope Single PR, ~300-400 lines including tests and connector wiring. Do not change the Tantivy index, do not add `file_path`/`line_range` connector parsing -- both are explicit follow-ups to keep this PR reviewable.

product-owner added a new dependency 2026-04-25 10:02:06 +02:00

#5 feat: Implement Tantivy-based session index for full-text search

product-owner referenced this issue

2026-04-27 07:00:23 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

root referenced this issue

2026-04-27 13:05:00 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

product-owner referenced this issue

2026-05-09 03:03:13 +02:00

ci: enforce F1.2 robot-contract regression job on every PR #49

Sign in to join this conversation.