diff --git a/PLAN.md b/PLAN.md index 97be3b9..d2421a5 100644 --- a/PLAN.md +++ b/PLAN.md @@ -185,3 +185,96 @@ See `ARCHITECTURE.md` §14. Rule: if a unit test can assert the exact output, it - No npm toolchain; UI assets hand-written and embedded. - Bedrock deferred until a SigV4 client is justified; interface supports it. - Remote repository: none configured. Development continues locally; pushing will be documented when a remote exists. + +--- + +## 7. Review pass: epistemic honesty, security reconciliation, reproducibility, privacy (v0.2, started 2026-09-15) + +Baseline recorded before any change: `go vet ./...` clean; `go test -race -count=1 ./...` passes (36 packages with tests, exit 0) at commit `4693dcb`, tagged `v0.1`. Working tree carried one untracked, user-created file (`docs.tgz`), left untouched. + +### 7.1 Findings and evidence + +Each finding was checked against the implementation, its tests and the documentation. Classification: **D** implementation defect, **DD** documentation defect, **AW** architectural weakness, **P** partially implemented, **OK** already correct, **X** incorrect assumption. + +| # | Topic | Class | Evidence | +|---|---|---|---| +| 1 | Assertions vs beliefs | **AW** | `internal/cognition/respond.go:95-111`: every claim extracted from a message becomes a `claim` belief whose only evidence is the message event (`Source: "said-by:"+actor`, weight 0.6, 0.8 for corrections). No structure records *who asserted what, with what stance, modality or certainty*; reported speech, hypotheticals, quotation, disagreement and sarcasm all collapse into the same path. `docs/conversation.md` states the message is the evidence. No test covers reported/hypothetical/negated speech. | +| 2 | Evidence provenance | **AW** | `internal/beliefs/confidence.go:91-117`: diversity counts distinct `Source` strings; `belief_evidence` has no ancestry columns (`migrations/0006`). Consolidation (`consolidation.go:310`) cites episodes as `memory` evidence with source `conversation/user` while the same episodes' claims already cite the message event as `said-by:actor`; a belief holding both counts two "independent" sources for one utterance. Hypotheses (`handlers.go:75`) key each event as its own source (`co-occurrence:`+id), so two events from one actor count twice. Replay preserves event ids, so provenance can be reconstructed once recorded. | +| 3 | Meaning of confidence | **DD/P** | `docs/beliefs.md` and the UI present `confidence` as a bounded score with a formula, never as calibrated probability, but the word "confidence" and the `conf` helper (percent-like display) invite that reading. `predictions.Calibrate` (`predictions.go:379-425`) already computes buckets and a Brier score per mind; nothing links belief scores to it. No documented meaning for `credibility`, `weight`, `diversity`. | +| 4 | "Gap-free" sequences | **DD** | `docs/adr/0002-event-immutability.md:12` says "bigserial seq for gap-free cursors". Code assumes only monotonic order: cursor reads `seq > cursor` (`brain/cycle.go:387`, `events/store.go:388-403`), SSE resumes with `seq > Last-Event-ID` (`api/events.go:283-303`). No `+1` assumption found (grep). No regression test for gaps. | +| 5 | Security docs vs code | **DD** (one **D**) | T3 says archives are "extracted with path normalisation…" and, in the same paragraph, "Archives are never extracted" (`THREAT_MODEL.md:26-27`); code rejects archives (`integrations/research.go`). T16 says "JSON in a zip with per-file SHA-256 manifest" then contradicts itself; code is a single JSON document with a whole-archive SHA-256 (`snapshots/archive.go`). T16 also says "policies are not imported" then "policies are copied as state": code copies `policies` on **import** as well as fork (`snapshots.Tables`). Importing policies from an untrusted archive is an authority-bearing import: **defect**; fork (same operator, same instance) may keep them. `SECURITY.md` says the research folder "treats archive members as hostile paths": archives are rejected outright. | +| 6 | Security test claims | **DD/P** | T9 (SQL injection) cites `TestStoredXSSIsEscaped`, which only sends metacharacters through search routes and checks status 200. No test asserts that injected SQL had no effect on other rows or that identifiers are allow-listed. Dynamic SQL exists in `snapshots/archive.go` (table/column names from `Tables` and `information_schema`) and in filter builders (`events/store.go`, `memory/memory.go`, `beliefs/store.go`) that use positional parameters. T17–T25 are marked ☐ although T19 (`router` budget tests), T20 (`brain.TestLoopDetectorSuppressesRepeats`), T21 (`secrets` tests), T22 (`integrations` signature/replay tests), T23 (git hooks disabled, tested) have tests; T17, T18, T24, T25 are partially covered. | +| 7 | Capability boundary | **OK/P** | Authority is enforced in `capabilities.Service.Invoke` (`service.go:258-`) outside cognition: forbidden contexts, registry lookup, parameter validation, policy `Decide`, approval or execution. `cognition.Act` only proposes `http.fetch` from conversation requests (`action.go:31-57`); model output never reaches `Invoke` directly. Missing: a test that a *forged or malicious cognitive output* (an `action` thought naming `sudo_shell`, or a perception JSON containing a tool call) cannot execute, and an explicit statement of the principle in `SECURITY.md`. | +| 8 | Immutable history vs privacy | **AW** | No redaction path exists. `cognitive_events` payloads, `messages.content`, `memories.content`, `llm_calls.request/response`, `audit_log.details`, snapshot archives and exports all hold plaintext; ADR-0002 mentions a "dedicated, audited maintenance path" that was never built. Logs do not write payload content (checked `slog` call sites). | +| 9 | Replay reproducibility | **P** | `replays` record snapshot, range, steps and labels (`replay.go`) but no build identity: `internal/version.Version` exists (ldflags, default `dev`) and nothing else is captured. `experiments` record LLM routing but not build, schema or prompt versions. Labels are exact/approximate/diverged; no "incompatible" and no first-divergence component analysis. | +| 10 | Cognitive forks per model | **P** | Snapshots, forks and experiments exist; experiments run one subject. Model routing is daemon-wide (`router.routing`, `llm_roles`), so two forks cannot use different providers. `llm.Request.MindID` is already carried, so per-mind role overrides can be resolved in the router without touching callers. | +| 11 | Causal timeline | **P** | Event page shows one hop (`api/events.go:151-190`: parents, children ≤50, evidence, derived, correlated). No multi-hop, bounded, labelled traversal; no distinction between recorded causation (`causation_id`, `parent_ids`, evidence refs) and inferred (same cycle). | +| 12 | Temporal beliefs | **AW** | `beliefs` has `first_created`, `last_evaluated`, `superseded_by` only; no validity interval. Contradiction detection (`store.go:~500-530`) marks both contested. "door open at 10:00" then "door closed at 10:05" becomes a contradiction. | +| 13 | Explicit unknowns | **P** | Statuses: hypothesis, tentative, likely, accepted, contested, rejected, superseded (`confidence.go:18-24`). A belief with no evidence sits at its prior as `tentative`; there is no `unknown`. Introspection answers "I have no belief about X" when nothing matches (`respond.go` intent), which is correct but not represented as state. | +| 14 | Contextual reliability | **AW** | Credibility is a fixed function of trust level (`pkg/events/events.go:221-236`); no per-source or per-domain adjustment, and no place an operator correction could lower it. Nothing lowers global credibility either (no defect, but no mechanism). | +| 15 | Association vs causation | **P** | `concepts.Relation` has `related_to`, `caused_by`, `supports`, `contradicts`, `derived_from`, `similar_to`, `follows`; entity co-mention writes `related_to` (`entities.go:51`); association hypotheses say "may have caused" and are `hypothesis` beliefs with alternatives (`handlers.go`), which is correct; no `co_occurs_with`, `possibly_caused_by`, `temporally_precedes`, `asserted_by` relations; nothing writes `caused_by` today. | +| 16 | Interpretation language | **OK/P** | `docs/interpretation.md` states the boundary and gives two operational examples. Missing: a table of operational meanings for the major terms. | +| 17 | Docs consistency | **X/DD** | README's "all sixteen phases" is true (PLAN §4). "T17–T25 unchecked" refers to `THREAT_MODEL.md` threats, not phases: they are verification items, several of which have tests today (see #6). | +| 18 | AWS Bedrock (user request) | feature | ADR-0004 anticipated Bedrock "behind the same interface later"; `llm.ProviderKind` has openai, anthropic, ollama. Bedrock needs SigV4 signing (stdlib crypto) over the Converse API, credentials as secret references. | + +### 7.2 Design decisions (smallest coherent change per finding) + +1. **Assertions.** New package `internal/epistemics`: an `Assertion` record (actor, actor kind, proposition, subject, stance `asserts|denies|doubts|questions|supposes`, speaker certainty `asserted|hedged|uncertain`, modality `factual|hypothetical|reported|quoted|imperative|question`, negated, reported actor, interpretation confidence with notes, source event, context, trust) and a deterministic interpreter over the perception output. Conversation creates an assertion for every claim; a proposition belief is created **only** for factual or reported stances, with evidence of kind `assertion`, family rooted at the source event, weight scaled by certainty, modality and interpretation confidence; hypotheticals and questions create no belief; `doubts` adds counter-evidence from the doubter. "Andy believes X" is queryable from assertions with the assertion's own confidence (interpretation confidence), separate from the proposition's score. Migration `0017_assertions`. +2. **Provenance.** `belief_evidence` gains `derivation` (direct, assertion, consolidation, inference, reflection, dream, retrieval, prediction, operator, external), `family` (independent-source key) and `root_ids uuid[]` with `provenance_status` (`recorded` | `legacy_unknown`, default legacy for existing rows). `Compute` groups by family: a family contributes its strongest member plus a diminishing share of the rest (documented), so three derived copies of one source ≈ one, and diversity counts families. Producers (conversation, consolidation, association, predictions, operator, dreaming) supply ancestry; a bounded helper derives roots for memories through `derived_from` edges (depth ≤ 8). Migration `0017` (shared with assertions, or `0018`). +3. **Confidence semantics.** Rename in documentation and UI labels: *epistemic score* (raw, uncalibrated) everywhere the number appears; add `beliefs.Calibration` interface backed by `predictions.Calibrate` per mind (overall and per prediction category), returning a calibrated estimate only when the bucket has ≥ 20 decided predictions and "insufficient data" otherwise; `Breakdown` gains explicit component meanings. No stored precision beyond what exists. +4. **Sequences.** Correct ADR-0002 and comments; add `TestCursorsToleratePlaceholderGaps` that burns sequence values in a rolled-back transaction and verifies cursor, query and SSE resume unchanged. +5. **Security docs.** Rewrite T3, T7, T16 and `SECURITY.md` to describe the code; **stop importing policies from archives** (fork keeps them), record the omission in the import result and audit; T17–T25 each rewritten to name a real test or stay ☐ with an honest note. +6. **Security tests.** Add `TestSQLInjectionHasNoEffect` (metacharacters through every search and filter route with row-count and content assertions before/after) and a static test that dynamic SQL identifiers come only from allow-lists; add tests for T25 (drive and goal vocabularies exclude self-preservation; minds cannot reach admin routes) and T24 (schema-invalid model output rejected). Rewrite each threat's V line to the test that actually proves it. +7. **Capability boundary.** Add `TestDeceivedMindHasNoAuthority`: forged action thoughts for locked, unknown and ASK capabilities and a perception JSON containing a tool call all end in deny/approval, never execution; document "A mind may be deceived. Deception must not imply authority." with the boundary's location. +8. **Privacy.** Cryptographic erasure is not adopted now: search, retrieval, replay and the UI read plaintext from these columns, and key-per-row envelopes would touch every read path; recorded as future work with the threat model. Implement the audited maintenance path ADR-0002 promised: a `redactions` ledger and `noema_redact_*` SQL functions (SECURITY DEFINER, callable only through `internal/privacy`) that replace the sensitive columns of an event, memory, message or model call with a fixed marker, keep a SHA-256 of the removed content, set `redacted_at/reason/policy`, write an audit row and a `redaction` event; cascade helpers find dependants (memories by origin event, messages by event, model calls by correlation); stored snapshot archives are scrubbed of the same ids; export and replay skip redacted content and label it. Backups are documented as outside the system's reach. Migration `0019_redaction`. +9. **Replay identity.** `internal/version` gains `Info()` from `debug.ReadBuildInfo` (module version, VCS revision, modified flag, time, Go version, OS/arch) plus schema version (max applied migration), canonical non-secret config digest, timezone, and per-run prompt versions, personality version, policy digest and LLM routing. Snapshots store the identity at capture; replays store it at run and compare: same commit and schema → candidate for exact; different schema → **incompatible**; different build → approximate at best with the differing fields listed. Divergence analysis records the first diverging step and the event categories that differed. Migration `0020_runtime_identity`. +10. **Forks per model.** `CognitiveConfig.LLMRoles map[role]model-id` honoured by the router when `Request.MindID` is set; experiments gain `variants` (name + config/personality/LLM overrides) run from one snapshot against the same stimuli, with a comparison report: first divergent stimulus, belief-set overlap, mean confidence difference, attention overlap, memory and goal counts, tokens and cost, capability requests. Existing single-subject experiments keep working. +11. **Causal timeline.** API `GET /api/v1/events/{id}/chain?direction=back|forward&depth≤6` and UI `/events/{id}/timeline`: bounded BFS over `parent_ids`, `causation_id`, evidence references and reverse references, ≤ 200 nodes, edge kinds `caused`, `parent`, `cited`, `derived`, and `same-cycle (inferred)` clearly labelled. +12. **Temporal beliefs.** `beliefs` gains `valid_from`, `valid_until`, `observed_at`, `temporal_scope` (`persistent|transient|unknown`); a deterministic scope classifier for state propositions; a newer contradicting observation of a *transient* proposition closes the older belief (`valid_until`, status `superseded`, reason "state changed") instead of contesting it; persistent contradictions behave as today. Migration `0018_temporal_beliefs`. +13. **Unknowns.** New status `unknown` for beliefs with no live evidence on either side (except hypotheses); introspection reports it as "I do not know"; CHECK constraint widened by migration `0018`. +14. **Source reliability.** `source_reliability(mind_id, source, domain, correct, wrong, score)` updated only by prediction outcomes and operator corrections **in the domain of the belief concerned**; evidence credibility = trust credibility adjusted by the (source, domain) score when it has ≥ 3 observations. Exposed on the belief page. Migration `0018`. +15. **Association vs causation.** Add relations `co_occurs_with`, `possibly_caused_by`, `temporally_precedes`, `asserted_by`; co-mention writes `co_occurs_with`; association hypotheses record `possibly_caused_by` between the two memories/entities; `caused_by` is reserved for explicit operator statements. Tests. +16. **Interpretation.** Add an operational-meaning table to `docs/interpretation.md`. +17. **Docs pass** after implementation; README status line clarified; THREAT_MODEL statuses verified. +18. **Bedrock.** `internal/llm/providers/bedrock`: SigV4 (HMAC-SHA256, stdlib) over the Converse API for chat and Titan/Cohere embeddings via InvokeModel; credentials as secret references (`access_key_ref`, `secret_key_ref`, optional session token) and region on the provider row; same SSRF-safe client; wire-format tests with recorded fixtures. + +### 7.3 Affected packages and files + +`internal/beliefs` (confidence, store, migrations), new `internal/epistemics`, `internal/cognition` (respond, handlers, entities, associate, action tests), `internal/consolidation`, `internal/dreaming`, `internal/predictions`, `internal/events` (gap test), `internal/snapshots` (policy exclusion, identity, scrub), `internal/replay`, `internal/experiments`, `internal/llm` (+`router`, new `providers/bedrock`), `internal/version`, `internal/config` (digest), new `internal/privacy`, `internal/api` (evidence/assertion/chain/redaction/variant routes, openapi.json), `internal/ui` (belief detail, timeline, experiment comparison, redaction admin), `pkg/concepts`, `migrations/0017–0020`, `tests/security`, `docs/*`, `THREAT_MODEL.md`, `SECURITY.md`, `README.md`, ADRs 0002 and new 0009–0012. + +### 7.4 Database migrations + +All additive: new tables (`assertions`, `source_reliability`, `redactions`, `experiment_variants` if needed), new nullable or defaulted columns (`belief_evidence.derivation/family/root_ids/provenance_status`, `beliefs.valid_from/valid_until/observed_at/temporal_scope`, `replays.identity`, `snapshots.identity`, `cognitive_events.redacted_at/redaction_id` plus `redacted_hash`), a widened CHECK on `beliefs.status`, and SQL functions for redaction. Existing evidence rows are marked `provenance_status='legacy_unknown'` with `family = source`; no ancestry is invented. Down migrations drop only what they added. Redaction functions are the only path that modifies append-only rows and run inside one transaction with an audit row. + +### 7.5 Security implications + +Policies are no longer imported from archives (fail closed; forks unaffected). Redaction is administrator-only, audited, CSRF-protected and irreversible by design. New endpoints are read-only except redaction and experiment variants (operator). Bedrock credentials are secret references; the signing key never leaves the process; endpoint is derived from the region, not user input. No new attack surface accepts file uploads or shell arguments. + +### 7.6 Backwards compatibility + +APIs are extended, not changed; existing fields keep their names (`confidence` stays as the JSON field, documented as the epistemic score, with `calibrated` added where available). Existing beliefs keep their statuses and scores until re-evaluated; `Compute` on legacy rows behaves exactly as before because legacy families equal legacy sources. Archives from 0.1 import unchanged except that policies are skipped and reported. Replays without identity are shown as "identity not recorded". + +### 7.7 Tests required + +Per change: unit tests for the interpreter (eight speech patterns), provenance grouping (independent vs derived vs repeated vs consolidated), calibration gating, gap tolerance, policy import exclusion, SQL injection effect test, static identifier scan, deceived-mind authority test, temporal supersession, unknown status, per-domain reliability isolation, relation semantics, identity capture and incompatibility, variant comparison metrics, chain traversal bounds and cycle safety, redaction (content gone, hash kept, references intact, snapshots scrubbed, replay labelled), Bedrock signing vectors and wire formats. Full `go test -race ./...`, `go vet`, staticcheck, gosec, govulncheck before each commit that touches code. + +### 7.8 Documentation affected + +`docs/beliefs.md`, `docs/conversation.md`, new `docs/epistemics.md` (final deliverable: architecture changes, epistemic, security, replay and privacy models, tests, limitations), `docs/snapshots-replay.md`, `docs/graph.md`, `docs/interpretation.md`, `docs/llm-providers.md`, `docs/predictions.md`, `SECURITY.md`, `THREAT_MODEL.md`, `README.md`, ADR-0002 correction, new ADRs (assertions and provenance; redaction; replay identity; Bedrock signing), `CHANGELOG.md`, the OpenAPI document, and the published site. + +### 7.9 Implementation order + +1. Sequence terminology + gap regression test (docs, tiny code). +2. Security reconciliation: policy import exclusion, SQL-injection and identifier tests, deceived-mind test, T17–T25 verification, `SECURITY.md`/`THREAT_MODEL.md` rewrite. +3. Evidence provenance schema and grouped `Compute`. +4. Assertions and the conversation pipeline. +5. Confidence semantics and calibration interface; unknown status; temporal beliefs; contextual reliability; relation semantics (four small commits). +6. Replay identity and divergence analysis; per-mind LLM routing and multi-variant experiments. +7. Causal timeline. +8. Privacy redaction. +9. Bedrock provider. +10. Interpretation table, documentation pass, final deliverable, site rebuild. + +### 7.10 Rollback considerations + +Each commit is independently revertible; migrations are additive with down scripts, and no migration rewrites existing values other than defaulting new columns. Reverting provenance leaves extra columns unread. Reverting redaction leaves the ledger table; redacted content is gone by design and cannot be rolled back, which is documented on the redaction screen. Reverting per-mind LLM routing leaves the config key ignored.