chore: initialise Noema repository
Foundation for the persistent, inspectable cognitive runtime: README, implementation plan with phases and risk register, architecture, security model, threat model, contributing guide, eight ADRs covering the global workspace, event immutability, memory representation, LLM abstraction, capability system, authentication, frontend and storage. Includes Makefile, env example, compose stack (pgvector) and Dockerfile skeleton, plus a record of the development environment assumptions.
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
# ADR-NNNN: Title
|
||||
|
||||
Date: YYYY-MM-DD
|
||||
Status: Proposed | Accepted | Superseded by ADR-XXXX
|
||||
|
||||
## Context
|
||||
## Decision
|
||||
## Consequences
|
||||
## Alternatives considered
|
||||
@@ -0,0 +1,20 @@
|
||||
# ADR-0001: Global-workspace cognitive cycle with deterministic attention
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Noema needs many concurrent cognitive processes to compete for limited "attention" in a way an operator can inspect and reproduce. Letting an LLM decide what to think about would be opaque, expensive and non-deterministic.
|
||||
|
||||
## Decision
|
||||
Each mind runs a cycle loop. Registered `Process` implementations observe a `BrainState` and return `ThoughtCandidate`s. A pure-Go attention scorer computes a weighted sum over named features with per-mind weights, records every candidate and its component scores as events, and admits the top-K into a capacity-limited workspace. Admitted thoughts are broadcast (as state and as an event) and dispatched to executive handlers. Cycles are triggered by new events or mode timers, never by a fixed hot loop.
|
||||
|
||||
## Consequences
|
||||
- Attention is explainable ("why did A win?") and unit-testable for determinism.
|
||||
- LLM cost is bounded per cycle because processes, not the cycle, decide when to call models within budgets.
|
||||
- Adding a cognitive faculty means adding a `Process`; the core does not change.
|
||||
- Workspace capacity and weights are tunables exposed in the UI with audit trails.
|
||||
|
||||
## Alternatives considered
|
||||
- Single LLM agent loop with tools: opaque, vendor-bound, cannot show why.
|
||||
- Actor-per-process with message passing: more machinery for no inspectability gain; concurrency is still bounded by a pool in the chosen design.
|
||||
@@ -0,0 +1,21 @@
|
||||
# ADR-0002: Append-only cognitive events with database-enforced immutability
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Provenance and replay require that history never changes. Application discipline alone is fragile.
|
||||
|
||||
## Decision
|
||||
`cognitive_events` (and `belief_versions`, `personality_versions`, `audit_log`, `approval_decisions`) get triggers that raise on UPDATE and DELETE. Belief changes, corrections, consolidations and retractions are new events referencing old ones. Retention purges, if ever configured, run through a dedicated, audited maintenance path that disables the trigger inside one transaction and records what was purged.
|
||||
|
||||
Events use UUIDv7 ids (time-ordered) plus a `bigserial seq` for gap-free cursors. Durable consumers read by `seq`; the in-memory bus is only a wake-up.
|
||||
|
||||
## Consequences
|
||||
- History is trustworthy by construction; provenance links never dangle.
|
||||
- Storage grows monotonically; retention policy is an explicit operator decision.
|
||||
- Cursor-based consumption makes restarts and replay simple.
|
||||
|
||||
## Alternatives considered
|
||||
- Event sourcing with projections rebuilt from events only: elegant but forces all reads through replay; we keep mutable projection tables (beliefs, memories) with immutable version tables beside them.
|
||||
- Kafka or similar: unnecessary operational weight for a single-process daemon.
|
||||
@@ -0,0 +1,23 @@
|
||||
# ADR-0003: Memory representation
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Noema needs episodic, semantic, procedural and autobiographical memory, working memory, decay without deletion, several retrieval mechanisms, provenance and relationships, and must not collapse epistemic categories.
|
||||
|
||||
## Decision
|
||||
- One `memories` table with a `kind` discriminator and shared columns (importance, salience, activation, retrieval stats, source, credibility, epistemic kind, timestamps) plus a typed `content` JSON document per kind. Go exposes distinct types; the shared table keeps relationships, search and retrieval uniform.
|
||||
- `epistemic_kind` is a separate enum (observation, fact, memory, claim, hypothesis, belief, prediction, assumption, inference, opinion) present on memories, beliefs and evidence links.
|
||||
- Full-text search via a generated `tsvector` column. Embeddings live in `memory_embeddings(memory_id, model, dims, embedding)` created only if pgvector is available, so representations from different models are never mixed.
|
||||
- Relationships live in `memory_relationships` and the general `entity_relationships` graph with confidence, provenance and timestamps.
|
||||
- Decay is a scheduled deterministic update of `activation`; rows are never deleted by decay.
|
||||
- Working memory is an in-process bounded structure persisted in `workspace_states` after each cycle.
|
||||
|
||||
## Consequences
|
||||
- Retrieval is a deterministic fusion of text, vector, recency, activation, importance and graph distance with per-component scores recorded.
|
||||
- Loss of the embedding service degrades retrieval quality but never availability.
|
||||
|
||||
## Alternatives considered
|
||||
- Table per memory kind: cleaner typing but duplicated retrieval/relationship code and cross-kind queries become unions.
|
||||
- Vector-only store: rejected by requirement; loses provenance and structure.
|
||||
@@ -0,0 +1,19 @@
|
||||
# ADR-0004: LLM as a replaceable cognitive service
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Minds must survive provider changes; different tasks want different models; every call must be inspectable and replayable; secrets must not leak.
|
||||
|
||||
## Decision
|
||||
`internal/llm` defines `LanguageModel` (`Complete(ctx, CognitiveRequest) (CognitiveResponse, error)`) and `Embedder`. Providers implement raw HTTP clients (OpenAI-compatible, Anthropic Messages, Ollama; Bedrock later behind the same interface). Provider rows in the database hold endpoint, model, parameters and a *secret reference*. **Roles** map cognitive tasks to a primary model and ordered fallbacks. A `Recorder` wrapper writes every request/response (prompt name+version, rendered messages, parameters, seed, model, usage, latency, estimated cost) and enforces budgets; a `Replayer` serves recorded responses during replay. Prompts are versioned rows rendered with structured inputs; untrusted text is enclosed in explicit data blocks.
|
||||
|
||||
## Consequences
|
||||
- Cognitive code depends only on roles; swapping vendors is configuration.
|
||||
- Replay of LLM steps is exact when a recorded response exists and approximate otherwise, and is labelled as such.
|
||||
- Budget enforcement and cost accounting live in one place.
|
||||
|
||||
## Alternatives considered
|
||||
- Vendor SDKs: heavy dependency trees, inconsistent behaviour; raw HTTP is small and controllable.
|
||||
- LangChain-style frameworks in Go: unnecessary abstraction.
|
||||
@@ -0,0 +1,19 @@
|
||||
# ADR-0005: Capability and policy layer as the only path to action
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
A reasoning system must not gain infrastructure access because it can describe an action. Operators need ALLOW/ASK/DENY control at several scopes with auditability.
|
||||
|
||||
## Decision
|
||||
Tools register a `Capability` (name, description, parameter schema, risk class, target kind). The only execution entry point is `capabilities.Invoke`, which: validates parameters, evaluates policies (scopes: global, mind, capability, target pattern, action type, time window; most specific wins; DENY > ASK > ALLOW on ties), records a `policy_decision` event, and either executes, creates an approval request, or denies. Approvals are bound to a hash of the exact request; "approve for session" grants a time-boxed, hash-pattern-bound exception. Contexts flagged as dreaming or replay are refused unconditionally. `DENY`-locked capabilities (sudo shell, database deletion) cannot be relaxed via UI or API.
|
||||
|
||||
## Consequences
|
||||
- Every external effect has a policy decision and an event behind it.
|
||||
- New tools are small: implement the capability, register it, choose a default policy.
|
||||
- Minds have no capability that edits policies, so escalation is structurally impossible.
|
||||
|
||||
## Alternatives considered
|
||||
- Per-tool ad-hoc checks: inevitably bypassed.
|
||||
- OS-level sandboxing only: valuable later as defence in depth, not a replacement for policy.
|
||||
@@ -0,0 +1,18 @@
|
||||
# ADR-0006: Local authentication design
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
The admin interface controls a system that can act on infrastructure. It needs strong local authentication first, with room for OIDC.
|
||||
|
||||
## Decision
|
||||
Argon2id (64 MiB, t=3, p=2, 16-byte salt) password hashes; opaque 256-bit session tokens stored as SHA-256 hashes with idle and absolute expiry; cookies `HttpOnly; SameSite=Lax; Path=/` and `Secure` outside development; per-session CSRF token checked on all state-changing cookie-authenticated requests plus Origin check; login limiter per IP (token bucket) and per account (exponential backoff persisted in the database); optional TOTP (RFC 6238, implemented with stdlib HMAC) with hashed recovery codes; session listing and revocation; roles `administrator`, `operator`, `observer` enforced by route middleware. Personal API tokens (hashed, role-bound) authenticate `noemactl` and scripts. `auth.Authenticator` is an interface so OIDC can be added.
|
||||
|
||||
## Consequences
|
||||
- No third-party auth dependencies beyond `x/crypto`.
|
||||
- First-run bootstrap admin from environment or `noemactl user create`.
|
||||
|
||||
## Alternatives considered
|
||||
- OIDC-only: blocks single-operator deployments.
|
||||
- JWT sessions: revocation and listing become awkward; opaque tokens are simpler and safer.
|
||||
@@ -0,0 +1,19 @@
|
||||
# ADR-0007: Server-rendered UI with dependency-free enhancement
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
The UI must be a rich cognitive observatory yet maintainable by Go developers, secure against XSS, and free of supply-chain risk.
|
||||
|
||||
## Decision
|
||||
Go `html/template` pages embedded in the binary, one layout, CSS custom properties with light/dark themes (system preference plus toggle), and a small amount of hand-written vanilla JavaScript: an SSE client that inserts server-rendered HTML fragments, form enhancements (fetch + swap where it helps, plain POST-redirect-GET otherwise), keyboard navigation, and a canvas force-directed graph for the relationship explorer. Charts are inline SVG rendered by templates. No npm, no bundler, no frameworks, no CDN. Strict CSP with no inline scripts.
|
||||
|
||||
## Consequences
|
||||
- Every view is a Go handler + template; live updates reuse the same templates.
|
||||
- Graph rendering is bounded (node caps, filters) by design.
|
||||
- If a page later needs heavier interactivity, a single vendored, hash-pinned library can be added under `web/static/vendor/` with an ADR update.
|
||||
|
||||
## Alternatives considered
|
||||
- React/Vue SPA: doubles the codebase and toolchain for little gain in an operator tool.
|
||||
- HTMX: close to what is needed, but a 30-line SSE/fragment helper covers our usage without a third-party script.
|
||||
@@ -0,0 +1,18 @@
|
||||
# ADR-0008: PostgreSQL with explicit SQL and embedded migrations
|
||||
|
||||
Date: 2026-09-14
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Requirement: PostgreSQL, proper migrations, no ORM auto-mutation, optional pgvector.
|
||||
|
||||
## Decision
|
||||
pgx v5 with pgxpool; repository types per domain with hand-written SQL; numbered `NNNN_name.up.sql` / `.down.sql` files embedded from `migrations/` and applied by a small runner (advisory lock, `schema_migrations` table with checksum). The embeddings migration uses a PL/pgSQL block that creates vector objects only if the extension is available; the runtime detects availability at start.
|
||||
|
||||
## Consequences
|
||||
- No hidden schema changes; migration state visible in the System page and `noemactl migrate status`.
|
||||
- Tests needing the database use a per-run schema and skip when no URL is configured.
|
||||
|
||||
## Alternatives considered
|
||||
- golang-migrate/goose: fine tools, but a 150-line runner avoids a dependency and matches our exact needs.
|
||||
- sqlc: attractive later; deferred to keep the toolchain minimal.
|
||||
Reference in New Issue
Block a user