feat: OpenCode and Codex JSONL connectors (F4.1 source-table completion) #28

New Issue

2026-04-25T05:03:00+02:00

product-owner commented

2026-04-25 05:03:00 +02:00

Problem Statement

The session-search spec F4.1 (terraphim-agent-session-search-spec.md, lines 269-280) lists six supported sources, but only four have connectors today:

Source	Format	Location	Status
Terraphim AI	JSONL	`~/.claude/`	done (`NativeClaudeConnector`, `ClaClaudeConnector`)
Cursor	SQLite	`~/.cursor/`	done (`ClaCursorConnector`)
Aider	Markdown	`.aider.chat.history.md`	planned (Task 2.5.1)
Cline	JSON	`~/.cline/`	tracked (#3)
OpenCode	JSONL	`~/.opencode/`	not tracked
Codex	JSONL	`~/.codex/`	not tracked

Verification:

rg -l "OpenCode|opencode|codex" crates/terraphim_sessions/src/connector/   -> 0 matches
rg -l "OpenCode|opencode|codex" crates/terraphim-session-analyzer/src/connectors/ -> 0 matches

OpenCode and Codex are JSONL log formats nearly identical in shape to Terraphim AI's ~/.claude/projects/<slug>/<session-id>.jsonl. They are mentioned by name in the spec source-list but no code or open issue exists for them. Without these, agents that use OpenCode (opencode/minimax-m2.5) or ChatGPT/Codex CLI on this very repo (per CLAUDE.md "Model Rules for Bigbox Agents") cannot have their sessions searched -- the headline cross-agent capability gap.

Acceptance Criteria

Add crates/terraphim_sessions/src/connector/opencode.rs implementing SessionConnector for OpenCode JSONL
Add crates/terraphim_sessions/src/connector/codex.rs implementing SessionConnector for Codex JSONL
Both connectors:
- source_id() returns "opencode" and "codex" respectively
- display_name() returns "OpenCode" and "ChatGPT Codex"
- default_path() resolves to ~/.opencode/ and ~/.codex/ (XDG-aware on Linux: prefer $XDG_DATA_HOME/opencode then ~/.opencode)
- detect() returns Available { sessions_estimate } only when the directory exists AND contains at least one .jsonl file
- import(opts) walks <root>/**/*.jsonl, parses each line as a JSON value, normalises into Session { id, source, source_id, created_at, updated_at, messages, metadata }
Schema mapping (best-effort, document assumptions in module rustdoc):
- OpenCode lines: { "role": "user|assistant|system", "content": "...", "timestamp": "..." } -> map directly to Message
- Codex lines: { "type": "message", "role": "user|assistant", "content": [...] } -> flatten content blocks to text; ignore type != "message" lines
- Unknown line shapes are logged at warn level and skipped (not fatal); count exposed via import() result
Reuse MessageRole::from_str (case-insensitive) for role mapping; default to MessageRole::User if missing with a debug log
Register both connectors in Sessionservice::with_default_connectors() so /sessions sources and /sessions import discover them automatically
Each line has at least created_at/updated_at -- if absent, fall back to file mtime; document the fallback in code comments
Unit tests with fixture JSONL files in crates/terraphim_sessions/tests/fixtures/opencode/ and tests/fixtures/codex/:
- happy path: 3 messages each, all parse, role mapping correct
- malformed line in middle: parse continues, count of skipped lines emitted
- empty file: returns 0 sessions, no error
- missing directory: detect() returns NotFound
cargo test -p terraphim_sessions connector::opencode connector::codex passes
cargo test -p terraphim_agent --features repl-sessions passes (existing /sessions sources/import tests still green)
cargo clippy -p terraphim_sessions -- -D warnings passes
Update crates/terraphim_sessions/README.md (or add) with a "Supported Sources" table that matches the spec verbatim plus the new connectors

Proposed Approach

Mirror the structure of NativeClaudeConnector (existing): a small JSONL walker + line parser per source.
No new external deps -- reuse serde_json, tokio::fs (for async dir walking), and walkdir (already in workspace).
Treat the two as a single PR because they share ~80% of code; factor a private helper parse_jsonl_messages(path, mapper) and pass per-source mappers.
For Codex's nested content: [{type: "text", text: ...}] shape, keep the mapper local to the codex module so future Codex schema changes do not affect OpenCode.
File mtime fallback uses tokio::fs::metadata(path).await?.modified() (already used elsewhere in the connector module).

Dependencies

None. Independent of #3 (Cline), #13 (Generic MCP). Should land alongside (or after) the secret redaction issue so the new connectors pick up redaction by default.

Verification

cargo test -p terraphim_sessions connector::opencode connector::codex
cargo test -p terraphim_agent --features repl-sessions sessions_sources sessions_import
cargo clippy -p terraphim_sessions -- -D warnings
mkdir -p ~/.opencode/test-session && cat > ~/.opencode/test-session/log.jsonl <<EOF
{"role":"user","content":"hi","timestamp":"2026-04-25T10:00:00Z"}
{"role":"assistant","content":"hello","timestamp":"2026-04-25T10:00:01Z"}
EOF
./target/debug/terraphim-agent --robot sessions sources | jq '.data.sources[] | select(.id=="opencode")'
./target/debug/terraphim-agent --robot sessions import --source opencode | jq '.data.imported'

Expected: sources.opencode.status == "Available"; imported >= 1.

Scope

Single PR, ~350-450 LOC across 2 new modules + service registration + 2 fixture sets + tests. No public-api churn outside Sessionservice::with_default_connectors.

Upstream: terraphim-agent-session-search-spec.md F4.1 source-table completion (lines 269-280).

## Problem Statement The session-search spec F4.1 (terraphim-agent-session-search-spec.md, lines 269-280) lists six supported sources, but only four have connectors today: | Source | Format | Location | Status | |--------|--------|----------|--------| | Terraphim AI | JSONL | `~/.claude/` | done (`NativeClaudeConnector`, `ClaClaudeConnector`) | | Cursor | SQLite | `~/.cursor/` | done (`ClaCursorConnector`) | | Aider | Markdown | `.aider.chat.history.md` | planned (Task 2.5.1) | | Cline | JSON | `~/.cline/` | tracked (#3) | | **OpenCode** | **JSONL** | **`~/.opencode/`** | **not tracked** | | **Codex** | **JSONL** | **`~/.codex/`** | **not tracked** | Verification: ``` rg -l "OpenCode|opencode|codex" crates/terraphim_sessions/src/connector/ -> 0 matches rg -l "OpenCode|opencode|codex" crates/terraphim-session-analyzer/src/connectors/ -> 0 matches ``` OpenCode and Codex are JSONL log formats nearly identical in shape to Terraphim AI's `~/.claude/projects/<slug>/<session-id>.jsonl`. They are mentioned by name in the spec source-list but no code or open issue exists for them. Without these, agents that use OpenCode (`opencode/minimax-m2.5`) or ChatGPT/Codex CLI on this very repo (per CLAUDE.md "Model Rules for Bigbox Agents") cannot have their sessions searched -- the headline cross-agent capability gap. ## Acceptance Criteria - [ ] Add `crates/terraphim_sessions/src/connector/opencode.rs` implementing `SessionConnector` for OpenCode JSONL - [ ] Add `crates/terraphim_sessions/src/connector/codex.rs` implementing `SessionConnector` for Codex JSONL - [ ] Both connectors: - `source_id()` returns `"opencode"` and `"codex"` respectively - `display_name()` returns `"OpenCode"` and `"ChatGPT Codex"` - `default_path()` resolves to `~/.opencode/` and `~/.codex/` (XDG-aware on Linux: prefer `$XDG_DATA_HOME/opencode` then `~/.opencode`) - `detect()` returns `Available { sessions_estimate }` only when the directory exists AND contains at least one `.jsonl` file - `import(opts)` walks `<root>/**/*.jsonl`, parses each line as a JSON value, normalises into `Session { id, source, source_id, created_at, updated_at, messages, metadata }` - [ ] Schema mapping (best-effort, document assumptions in module rustdoc): - OpenCode lines: `{ "role": "user|assistant|system", "content": "...", "timestamp": "..." }` -> map directly to `Message` - Codex lines: `{ "type": "message", "role": "user|assistant", "content": [...] }` -> flatten content blocks to text; ignore `type != "message"` lines - Unknown line shapes are logged at warn level and skipped (not fatal); count exposed via `import()` result - [ ] Reuse `MessageRole::from_str` (case-insensitive) for role mapping; default to `MessageRole::User` if missing with a debug log - [ ] Register both connectors in `Sessionservice::with_default_connectors()` so `/sessions sources` and `/sessions import` discover them automatically - [ ] Each line has at least `created_at`/`updated_at` -- if absent, fall back to file mtime; document the fallback in code comments - [ ] Unit tests with fixture JSONL files in `crates/terraphim_sessions/tests/fixtures/opencode/` and `tests/fixtures/codex/`: - happy path: 3 messages each, all parse, role mapping correct - malformed line in middle: parse continues, count of skipped lines emitted - empty file: returns 0 sessions, no error - missing directory: `detect()` returns `NotFound` - [ ] `cargo test -p terraphim_sessions connector::opencode connector::codex` passes - [ ] `cargo test -p terraphim_agent --features repl-sessions` passes (existing /sessions sources/import tests still green) - [ ] `cargo clippy -p terraphim_sessions -- -D warnings` passes - [ ] Update `crates/terraphim_sessions/README.md` (or add) with a "Supported Sources" table that matches the spec verbatim plus the new connectors ## Proposed Approach - Mirror the structure of `NativeClaudeConnector` (existing): a small JSONL walker + line parser per source. - No new external deps -- reuse `serde_json`, `tokio::fs` (for async dir walking), and `walkdir` (already in workspace). - Treat the two as a single PR because they share ~80% of code; factor a private helper `parse_jsonl_messages(path, mapper)` and pass per-source mappers. - For Codex's nested `content: [{type: "text", text: ...}]` shape, keep the mapper local to the codex module so future Codex schema changes do not affect OpenCode. - File mtime fallback uses `tokio::fs::metadata(path).await?.modified()` (already used elsewhere in the connector module). ## Dependencies None. Independent of #3 (Cline), #13 (Generic MCP). Should land alongside (or after) the secret redaction issue so the new connectors pick up redaction by default. ## Verification ``` cargo test -p terraphim_sessions connector::opencode connector::codex cargo test -p terraphim_agent --features repl-sessions sessions_sources sessions_import cargo clippy -p terraphim_sessions -- -D warnings mkdir -p ~/.opencode/test-session && cat > ~/.opencode/test-session/log.jsonl <<EOF {"role":"user","content":"hi","timestamp":"2026-04-25T10:00:00Z"} {"role":"assistant","content":"hello","timestamp":"2026-04-25T10:00:01Z"} EOF ./target/debug/terraphim-agent --robot sessions sources | jq '.data.sources[] | select(.id=="opencode")' ./target/debug/terraphim-agent --robot sessions import --source opencode | jq '.data.imported' ``` Expected: `sources.opencode.status == "Available"`; `imported >= 1`. ## Scope Single PR, ~350-450 LOC across 2 new modules + service registration + 2 fixture sets + tests. No public-api churn outside `Sessionservice::with_default_connectors`. Upstream: terraphim-agent-session-search-spec.md F4.1 source-table completion (lines 269-280).

product-owner added a new dependency 2026-04-25 05:03:12 +02:00

#26 feat: redact api keys/secrets/JWTs during session import (Privacy)

root referenced this issue

2026-04-25 05:07:16 +02:00

feat: redact api keys/secrets/JWTs during session import (Privacy) #26

product-owner referenced this issue

2026-04-27 07:00:23 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

product-owner referenced this issue

2026-04-27 11:04:34 +02:00

feat: wire ExitCode into main() return for f1.2 robot exit code contract #35

root referenced this issue