FORGE_DFORWARD_SESSION_COACCESS_v1 · persist session linkage on the store-fill WRITE path · forward-only
Co-access backfill (option B) was falsified — 0.9% live coverage, confirmed by an independent Grok+Gemini+Codex panel and the operator's own 4-model panel. Root cause is a marking problem: live-capture chunks had session grouping at capture, but the store drops it on write. D-forward fixes it forward — new captures persist session linkage so the co-access plane accrues as a real behavioral signal. Runs parallel to vectors (revive-frontier); it is not the near-term recall win — its value is future accrual.
{session_id, transcript_path, ts} → ~/.forge/store-fill/spool.jsonlsessionId IS in scopeadmitSpan TIER-1/2 secret+PII gate passes the spanEvictionAsPut.consolidate → store.put(raw)put(raw) takes ONLY raw; INSERT INTO records has no session column → linkage LOST{session_id, ordinal, capture_batch_id} + redacted transcript ref through consolidateadmitSpan passes — order preserved (persist only for the admitted content_hash)record_session substrate row (composite PK) — the freq co-access LOGfreq-only related_hashes → SELECTComposed lane, single-writer, hard-ordered (fleet-ruled): D-forward writes the record_session substrate ONLY; buildRelatedHashIndex is the sole related_hashes writer. It runs as a freq-only projection (knn + dag excluded) — because at the default fused weights freq (0.05) is drowned by knn (0.6), so a blend stays a partial vector-dup; a freq-only index_version is the pure orthogonal co-access axis the SELECT routes to. Substrate must populate FIRST, then reindex — building the indexer before the substrate = a knn/vector duplicate. Gate order: substrate row written ONLY after admitSpan passes, for the admitted content_hash only.
CREATE TABLE record_session ( content_hash TEXT NOT NULL, -- FK records session_id TEXT NOT NULL, transcript_ref TEXT, -- REDACTED (hash/token, not raw path) ordinal INTEGER, -- position within session capture_batch_id TEXT, created_at INTEGER NOT NULL, PRIMARY KEY (content_hash, session_id) -- many-to-many );
related_hashes( source_hash, related_hash, freq_score REAL [0,1], -- co-access weight index_version, computed_at) -- edges derived from same-session -- membership, proximity-bounded -- (NOT full-session clique)
FORGE_DFORWARD_SESSION_COACCESS = 0 (default — byte-identical) = 1 writes sidecar + edges Additive · reversible · serving unchanged until enabled.
// signature widened, provenance OPTIONAL async put( raw: Uint8Array | string, prov?: SessionProvenance // NEW, optional ): Promise<PutResult> // existing callers unaffected (prov undefined) // when prov + flag on → write record_session
Backward-compatible: prov optional; all current put(raw) callers keep working.
// thread session context that is ALREADY
// in scope (feedSessionContent(sessionId))
consolidate(span, {
sessionId,
ordinal: acc.ordinal++,
captureBatchId,
transcriptRef: redact(transcriptPath)
})
No new data source — sessionId is already tracked per-accumulator; we stop discarding it.
redactTranscriptPath(p): string // keyed hash + display-safe basename; // NEVER store raw /Users/<name>/… path emitCoAccessEdges(sessionRows) // cap fan-out K; weight∈[0,1]; // idempotent upsert; off hot-path
The two security/perf-critical helpers the SEED panel required.
| Scenario | Expected behavior |
|---|---|
Identical content in two sessions (same content_hash) | Two rows via composite PK (content_hash, session_id) — many-to-many; never overwrite provenance. |
transcript_path contains PII (/Users/<name>/…) | Redacted to keyed hash + basename before persist; raw path never stored. BLOCKER folded |
Span abstained by admitSpan (secret detected) | No sidecar row, no edge — persistence strictly downstream of admission. |
| Same content twice within one session | Idempotent upsert on the PK; ordinal of first occurrence retained. |
| Large session (N chunks) | Sidecar O(1)/chunk; edges proximity-bounded (cap fan-out K), NOT N² clique. perf |
| Flag OFF (default) | No sidecar writes, no edges — store bytes + serving identical. |
| Historical 2,822 records | Untouched. Forward-only — no created_at reconstruction (interleaved-session false positives). |
| Drainer restart mid-session | capture_batch_id marks partial batch; re-drain idempotent via PK. |
| Layer | Test |
|---|---|
| unit | redactTranscriptPath never emits a raw home path; deterministic token; basename display-safe. |
| unit | composite-PK many-to-many: same content, 2 sessions → 2 rows; same (content,session) twice → 1 row (idempotent). |
| unit | edge fan-out cap: N-chunk session emits ≤ K·N edges (no clique); every freq_score ∈ [0,1]. |
| unit | admission-order: abstained span → 0 sidecar rows, 0 edges. |
| integration | real spool → drainer → sidecar + edges land for admitted content_hash only (fixture transcript). |
| integration | flag OFF → identical store bytes vs baseline (byte-diff assertion). |
| migration | additive migration applies to an existing populated store; no data loss; re-runnable. |
| File | Change |
|---|---|
total-recall-mw/src/store-fill-drainer.ts | Thread sessionId/ordinal/batch/redacted-ref through consolidate (already in scope). |
total-recall/src/store.ts | Widen put() with optional provenance; write record_session; call emitCoAccessEdges. |
total-recall/src/eviction-as-put.ts | Pass provenance through consolidate → put. |
total-recall/src/related-hash.ts | Idempotent bounded edge upsert path (freq_score). |
total-recall/…/migrations/*.sql (new) | Canonical additive migration: record_session table + index. |
transcript_path is filesystem metadata highly likely to carry PII. admitSpan gates content, not metadata — the path must be redacted (keyed hash + basename) before persistence. Persistence stays strictly downstream of TIER-1/2 admission; the secret-boundary-guard (599c890361) is preserved. Gemini SEED blocker, folded.put() provenance is optional; every existing caller is unaffected. Behind default-OFF FORGE_DFORWARD_SESSION_COACCESS; store bytes + serving are byte-identical until explicitly enabled and coord-verified.