Kind `bedrock` speaks the Converse API for chat roles and InvokeModel for
Titan and Cohere embeddings. Requests are signed with AWS Signature
Version 4 implemented with crypto/hmac and crypto/sha256 only; the signer
is pinned by tests to signatures botocore produces for the same requests,
with and without a session token, so drift fails the build.
The credential is one secret reference, ACCESS_KEY_ID:SECRET_ACCESS_KEY
[:SESSION_TOKEN], encrypted together like every other provider key. The
region is read from the endpoint host, which must be a Bedrock runtime
host over HTTPS unless private endpoints are explicitly allowed; other
hosts are refused. Provider errors are scrubbed of the secret and token.
Migration 0024 widens the provider kind constraint; existing rows are
untouched and the down migration refuses while bedrock rows exist rather
than dropping configuration. UI kind selects, docs/llm-providers.md,
ADR-0010, ARCHITECTURE and the changelog are updated.
Rows in append-only history are never deleted; their content can now be
made irrecoverable through one audited path. A redaction replaces the
content columns of an event, memory, message, model call, belief or
assertion with a fixed marker, keeps identifiers, links, timestamps,
scores and breakdowns, writes an append-only ledger row with a SHA-256 of
the removed content, appends a redaction event, optionally cascades to
derived rows and to the events that quote them, and scrubs stored snapshot
archives before re-signing them. noema_redact() is the only write the
append-only trigger admits, for UPDATE only, under a transaction-local flag
the application never sets; plain updates and all deletes stay refused.
Replay labels redacted stimuli instead of inventing them.
Cryptographic erasure was considered and deferred with reasons (ADR-0009);
docs/privacy.md defines retained, redacted, irrecoverable and deleted and
states what backups and earlier exports are not covered. Migration 0023.
A bounded walk (six hops, two hundred events) over the links events
actually record: causation ids, parent ids and cited-event evidence,
backwards towards causes and forwards towards consequences, with the edge
kind on every arrow. Nodes carry their cycle for grouping, labelled as
correlation rather than causation, and the artefacts (beliefs, memories,
model calls) they rest on. Cyclic references terminate. Exposed at
GET /api/v1/events/{id}/chain and /events/{id}/timeline, linked from the
event page, and documented.
A mind's configuration may override model routing per role (llm_roles);
the router honours it for that mind only and fails closed when the named
model is missing or disabled, so a comparison between providers can never
silently fall back to the default. Experiments may declare variants: each
becomes its own fork of the same snapshot with its own configuration,
personality or routing, receives identical stimuli, and is measured. The
results carry one entry per variant and a deterministic comparison: first
divergent stimulus, belief and attention overlap, mean score difference on
shared propositions, and the ranges of memory counts, goal counts, cost and
capability requests. Single-subject definitions are unchanged.
Snapshots, replays and experiments now record the runtime they ran under:
build (module version, VCS commit, modified tree, commit time, Go version,
platform), schema version, a digest of the non-secret configuration that
ignores secret and credential changes, personality version, policy digest,
model routing including per-mind overrides, prompt versions, deterministic
subsystem versions and timezone. A replay compares its identity with the
snapshot's and is an exact-candidate, a different-build (differences
listed) or incompatible (schema differs; the replay stops before running).
The first non-exact step is analysed into events only in the recording,
events only in the replay, and the components they belong to, labelled as
inferred; live model calls are named as their own source of difference.
Migration 0022 is additive; unrecorded identities are reported as such.
A source's track record is kept per (evidence family, belief subject) and
updated only by outcomes: predictions derived from a belief resolving, and
operators marking a belief wrong. With three or more outcomes in a domain
the record adjusts the credibility of that source's new evidence in that
domain, and the adjustment is written on the evidence row. Standing in one
domain never moves standing in another, and the trust-level credibility
constants are never rewritten. Migration 0021 is additive; belief pages
show each cited family's record in the belief's domain.
Beliefs gain a temporal scope (persistent, transient or unknown, classified
deterministically from the proposition), an observation time and, for
transient states, a validity interval. When a newer belief conflicts with
an older transient one observed at least a minute earlier, the older belief
is closed (valid_until set, status superseded, a supersedes link, a version
and an event saying 'state changed') with its score untouched: 'the door
was open at 10:00' stays true of 10:00 when it is closed at 10:05.
Simultaneous conflicting reports and persistent propositions contradict as
before. Introspection lists superseded states with their validity.
Migration 0020 is additive.
Every breakdown now carries its meaning: an uncalibrated epistemic score
derived from recorded evidence, not a probability of truth. Belief detail
adds a calibrated estimate read from the mind's resolved predictions when
at least twenty were decided in the score's range, and otherwise says why
none is shown; and names the primary uncertainty deterministically (no live
evidence, conflicting evidence, second-hand only, unclear reading, a single
source family, little evidence). The belief list heading becomes Score.
A statement made to a mind is now three things kept apart: the observation
that it arrived, an append-only assertion (attributed actor, stance,
certainty, modality, negation, reported or quoted speech, hypothetical
status, interpretation confidence and notes), and, only for factual
assertions, one piece of evidence for a proposition belief attributed to
whoever actually stands behind it. Reported speech is attributed to the
person reported and weighed as second-hand; hypotheticals, questions and
imperatives create no belief; expressed doubt counts against; a possibly
sarcastic remark is recorded with low interpretation confidence rather
than guessed at. Interpretation is deterministic code; a model may enrich
the claim list but never sets stance or confidence.
Association no longer treats conversational turns as world events:
hypotheticals, questions, requests and consecutive turns by one speaker
cannot seed causal hypotheses.
Adds internal/epistemics, migration 0018, the assertions API, assertions
on belief pages and in belief detail, a shared end-to-end test harness
(tests/harness), end-to-end tests for eight speech patterns, and updates
the milestone test to the new provenance.
Every evidence row now records its derivation (direct, assertion, operator,
external, prediction, consolidation, inference, reflection, dream,
retrieval), its independent-source family (an actor, an integration, an
operator, a prediction) and its root event or memory ids. Confidence
groups evidence by family: the strongest primary item counts in full, each
further primary item adds a quarter, capped at 1.5x, and derived items
count only when the family has no primary member. Three derived
re-expressions of one utterance therefore weigh exactly what the utterance
weighs, while three independent families still add up.
Producers (conversation, association, consolidation, predictions,
metacognition, dreaming, operator and seed input, contradictions) supply
their ancestry. Existing rows are marked legacy_unknown with family =
source; nothing is invented. The belief page shows per-family
contributions and the replay test checks that provenance is reconstructed
identically in the sandbox. Migration 0017 is additive.
- TOTP second factor (RFC 6238, secret encrypted under the master key),
ten hashed single-use recovery codes, two-step login with a pending token
bound to the client address, account enrolment/disable, admin reset
- tests/security: prompt-injection corpus never acts or creates evidence-free
beliefs; roles and CSRF on form and API paths; cross-mind isolation
(demo scenario 5); stored-XSS escaping and CSP across pages; secrets never
echoed (API, pages, database, archives); hostile archive import; session
lifecycle and security headers
- OpenAPI 3.1 document generated from the route table, served at
/api/v1/openapi.json and kept complete by a test
- gosec clean (justified #nosec annotations, cookie/redirect hardening,
bounded multipart parsing, UUID packing without narrowing conversions);
govulncheck clean with toolchain pinned to go1.26.8
- THREAT_MODEL verification pointers, SECURITY.md testing section, README,
CHANGELOG 0.1.0, docs/RELEASE_NOTES.md, PLAN status
- snapshots: checksummed JSON archive of every mind-scoped state table plus
identity/personality/config and the event position; stored, downloadable
- fork: restore into a new mind with all ids remapped; event references are
kept so history before the fork is shared by reference
- export/import: same archive; import is hostile-input validated (size,
allow-listed tables, row caps, column names, checksum, forced mind_id)
- compare: deterministic counts, belief diffs by proposition, personality
and config diffs, event mix
- replay: Runtime.Step for synchronous drivers; sandbox fork driven one
stimulus at a time under a forbidden capability context; recorded model
responses via a replaying router; steps labelled exact / approximate /
diverged with recorded-vs-replayed digests
- experiments: definition (overrides, stimuli, expectations, seed, duration),
background run in a disabled subject mind, stored and exportable results
- migration 0015, API, UI (Snapshots and Replay tabs, Experiments section,
compare page), noemactl snapshots/compare/replay/experiments, docs
- dreaming.Service: deterministic memory replay sample keyed by dream id,
vocabulary-based association of unlinked memory pairs (similar_to edges +
dream_association events), speculative hypotheses as hypothesis-kind beliefs
with two weak memory citations (starts ~0.30, asserted <= 0.35)
- Run marks the context forbidden for capabilities before anything else;
test proves Invoke refuses inside a dream
- per-mind settings: enabled (default off), hours window (wraps midnight),
interval, model-call / memory / hypothesis caps; global pause switch
- optional model wording via new `dream` prompt (role hypothesise)
- promotion adds operator evidence, rejection retracts the belief; both
audited and recorded as operator actions
- migration 0014, API, Dreams tab and page, noemactl dreams, docs/dreaming.md
Phase 12 (ADR-0005). Capabilities are registered in code with parameter
schemas, risk classes and default effects; sudo_shell and delete_database
are locked to DENY and cannot be relaxed. Policies are scoped globally or
per mind with capability, target glob, action kind and optional hour
window; evaluation is deterministic, most specific wins, and ties go to
the stricter effect. Invoke is the single entry point: it refuses
dreaming and replay contexts, validates parameters (unknown keys, non
http URLs and path traversal rejected), evaluates policy and session
grants, hashes the exact request and records a policy_decision event.
ALLOW executes with timeout and panic isolation and turns textual output
into a tool-trust observation; ASK creates an approval request bound to
the hash with an alert notification; DENY is counted and recorded.
Operators approve once, approve for a time-boxed session grant, deny
with a note, or modify parameters (re-validated, re-hashed, original
kept in the append-only decision log). The self-model's capability list
is recomputed from effective policy on start and on every policy change.
The act process maps fetch requests to http.fetch and the action handler
invokes through the layer; replies report capabilities and outcomes.
Policy matrix and approval centre pages, dashboard count, API, CLI
(approvals, policy, invoke) and docs/capabilities.md.
Phase 11. Entities are mind-scoped nodes with canonical matching,
aliases and deterministic kind guessing (hostnames and hyphenated ids
are systems, capitalised names people, file suffixes documents).
Relationships carry a relation from the shared vocabulary, confidence,
source, reinforcement count and the event that produced them; links tie
entities to memories, beliefs, events, goals and predictions.
The perceive and attend handlers upsert the entities a message or
observation mentions and record co-mention edges cited to the event, so
the graph grows only from what was actually seen. Operators can merge
("these are the same": edges and links move, the old name becomes an
alias, the old id still resolves), record that two entities are
different, correct kinds and add typed edges; every correction is an
operator action and a corrections row.
The explorer renders a bounded subgraph on a canvas with a small
force-directed layout in plain JavaScript, with kind filters, focus on
a neighbourhood and a side panel per entity. API, entity page, CLI and
docs/graph.md.
Phase 10. Metacognition is a cognitive process with deterministic
self-checks (insufficient evidence, single-source reliance, model output
without evidence, repeated failed hypotheses, stale contradictions,
calibration overconfidence); its warnings compete for attention and,
when admitted, add counter-evidence or open an investigation goal.
Reflection reviews the period since the last run (prediction errors,
rejected beliefs, corrections, status changes, completed goals,
contradictions, calibration), turns repeated failures into procedural
lessons written once, and records a reflection event; a model may only
word the prose within the mind's budget.
Consolidation clusters unconsolidated episodes by shared stemmed terms,
creates a belief whose confidence is computed from the episodes as
evidence and a semantic memory linked to each episode by derived_from,
and records jobs with inputs and outputs. Wording may come from the
consolidate prompt within a per-run model budget; the job says which.
Operators can run, pause and reject; originals are never touched.
The narrative hook keeps a five-part first-person narrative built from
state and stored with narrative_updated events. The notification hook
applies explicit thresholded rules, aggregates repeats within a cooldown
and records suppressions with reasons. A scheduled job runs
consolidation and reflection per mind, switching the runtime mode while
it works. API, Background tab, notifications page with unread badge,
overview narrative and warnings, CLI commands, docs/cognition.md.
Phase 9. Goals form a mind-scoped tree with kind, priority, progress and
blockers; their deterministic relevance to a candidate feeds the
goal_relevance attention feature and a goal monitor raises stalled goals
and observations that touch one. Drives are a closed set of eight
scalars with operator-configured ranges and personality-derived
baselines; there is no self-preservation, replication or acquisition
drive and no way to add one at runtime. Affect is ten scalars moved by an
explicit rule table over events and decaying towards baseline; both
modulate attention weights each cycle and the effective weights are
recorded with every cycle and shown beside the configured ones.
Predictions carry a window and machine-checkable criteria (event terms
occurring or staying absent, a belief reaching a confidence, or operator
judgement). A resolver job judges them deterministically, keeps an
append-only outcome history for calibration (accuracy, Brier score,
confidence buckets, overconfidence flag), emits prediction_error events
scaled by how confident the miss was, and feeds the outcome back into the
linked belief as evidence. Causal hypotheses derive predictions
automatically. Curiosity turns misses, contradictions and unexplained
observations into questions that compete for attention and become
investigation goals when admitted.
Brain gains after-cycle hooks and goal/motivation state. API, UI (goals,
prediction dashboard, cognitive state panel, effective weights) and CLI
(goal, prediction, state). Demo scenario 4 verified end to end.
Phase 8, completing milestone M1. A user message is stored and appended
as an observation event carrying trust, actor, channel and reference. The
perceive process analyses it deterministically (intent, introspection
kind, entities, claims, urgency) and, within the per-cycle model
allowance, enriches it through the perceive prompt. Its handler writes a
perception event, an episodic memory with participants, turns claims into
beliefs whose evidence is the message, and builds a response intent from
state: beliefs and memories for the topic, evidence rows for why-questions,
the uncertain band, contested beliefs and their links, recent learning,
status changes, and honest declines for subsystems that do not exist yet
or for requests with no capabilities. The respond prompt phrases the
intent when a model is assigned; otherwise the reply is a direct
rendering of the intent. response_generated cites the message, the
perception, every belief and memory used, the episode and the model call.
API: conversations, messages with optional wait, one-shot ask; chat UI
with live refresh and a per-reply intent view. Sample mind Dani as a
strictly decoded seed document with an import endpoint and noemactl mind
import. tests/m1 drives all fourteen milestone steps including restart.
Phase 7 (ADR-0004). Cognitive code asks the router for a role; the
router maps roles to configured models with ordered fallbacks, applies
global and per-mind daily token budgets and a concurrency limit, records
every call (request, response, usage, latency, cost, seed, request hash)
and can serve recordings back for exact replay. Providers are raw HTTP
clients for OpenAI-compatible, Anthropic and Ollama endpoints, with
embedding support for the embed role wired into memory retrieval.
Secrets are references only: env:, file:, or enc: values encrypted with
AES-256-GCM under NOEMA_MASTER_KEY; keys are resolved at call time and
never rendered, logged or exported. Outbound HTTP refuses private and
loopback addresses at dial time unless a provider explicitly allows it,
bounds redirects and caps bodies. Provider errors are scrubbed.
Prompts are named, versioned rows with system and user templates and an
optional JSON schema; untrusted text enters only through delimited data
blocks. Six defaults are seeded and never overwritten. The UI adds the
LLM page (providers, models, role matrix, usage, errors, budgets), a
provider page with connectivity test, and a prompt manager with editor,
test runner, history, restore, clone and diff. CLI: provider, prompt.
Phase 6. Beliefs are propositions with a prior set by epistemic kind and a
confidence computed from typed evidence: sigmoid(logit(prior) + slope ×
(support × diversity − opposition)), where support from a single source
is discounted so repetition cannot manufacture certainty, and bounds keep
the mind away from absolute certainty. Every evaluation stores its
breakdown and appends a version row protected by the append-only
trigger; the belief page shows the arithmetic, evidence for and against,
alternatives, contradictions and history.
Contradiction detection is deterministic first (negation plus token
overlap): a hit emits contradiction_detected, links both beliefs and
makes each counter-evidence for the other so both trend to contested.
Operators can assert beliefs (their statement is the evidence), add
evidence, mark a belief wrong (pinned rejected, history kept) and forbid
an inference; do-not-infer corrections block matching hypotheses.
Retracting a memory retracts the evidence that cited it and re-evaluates
the beliefs concerned.
The associate process proposes low-confidence causal hypotheses from
co-occurring notable observations, and the hypothesis handler turns
admitted ones into beliefs with automatic alternatives, closing demo
scenarios 1 and 2 without any model call.
Phase 5 (ADR-0003). pkg/concepts fixes the epistemic vocabulary
(observation, fact, memory, claim, hypothesis, belief, prediction,
assumption, inference, opinion), memory kinds and relation names.
internal/memory stores episodic, semantic, procedural and
autobiographical memories in one mind-scoped table with trust-derived
credibility, importance, salience and decaying activation. Retrieval
fuses full-text rank (any-term by default), optional pgvector similarity,
recency, activation, importance and credibility, and returns the
component scores so every recall can be explained. Decay lowers
activation and never deletes; retraction and supersession keep rows for
provenance. Relationships are constrained to one mind at insert time.
Embeddings live in a conditional pgvector table keyed by model so vector
spaces are never mixed; without an embedder the system degrades to text
and graph retrieval.
Attention gates encoding: the attend handler turns admitted observations
into episodic memories (operator changes become autobiographical ones)
with memory_created events; the retrieve process proposes recalled
memories as candidates and admitted recalls are reinforced with
memory_retrieved events carrying their scoring components.
Memory explorer UI (search with component breakdown, timeline, filters,
operator knowledge entry), memory detail with provenance, relationships,
citing events and an inaccurate-memory correction; API; noemactl memory
commands; decay and embed-missing background jobs.
Phase 4 (ADR-0001). internal/attention scores thought candidates as a
normalised weighted sum over ten named features with a per-term
breakdown, ranked by (score, id) so ordering is total and replayable.
internal/workspace holds the capacity-limited workspace (refresh by
subject, decayed competition, explained admit/reject outcomes) and the
decaying working memory.
internal/brain runs one runtime per active mind: debounced wake-ups from
the event bus (own output never re-triggers), a settle window so
concurrently committing events are not skipped, bounded concurrent
Observe calls with per-process isolation and timeouts, deterministic
scoring, a loop detector that suppresses recurring thoughts and raises a
metacognition warning, executive dispatch by thought kind, and one
transaction per cycle whose events are published only after commit.
Every candidate, selection, attention shift and broadcast is an event
with provenance back to the triggering observation. Operators can run a
cycle, pause, resume, reset transient state and change attention
weights live; the manager reconciles runtimes with mind status.
cognition/attend is the first process: deterministic candidates from
external events using an urgency lexicon, novelty against context and
trust-based operator priority.
Phase 3. Events are the append-only record of everything a mind does
(ADR-0002). pkg/events defines the closed type vocabulary, categories,
evidence references and the observation payload with explicit trust
levels. The store validates and appends inside the caller's transaction,
publishes to a bounded fan-out bus only after commit, queries by seq
cursor with type/category/source/salience/confidence/time/search
filters, and walks provenance: cause, parents, children, cited evidence,
derived events and correlation groups. Durable per-consumer cursors never
move backwards.
Server-Sent Events streams (per mind and global) subscribe before
replaying history after Last-Event-ID so nothing is missed, apply the
same filter vocabulary live, and heartbeat. Operators can inject
observations tagged with operator trust for demos and tests.
UI: live thought stream with filters and paging, forensic event page
with causal chain and evidence navigation, dashboard important events.
Personality and status changes now leave operator_action events in the
mind's own history. CLI: event inspect/list/tail, observe.
Phase 2. Minds are rows with a slug, status (active/disabled/archived),
cognitive configuration and a pointer to the current personality version.
Personalities are structured (identity, values, traits, communication,
behaviour, free text) with a deterministic Brief() used by prompts and the
editor preview. Every change appends a personality_versions row protected
by the append-only trigger; restore copies an old version forward.
Templates carry reusable personalities; minds record their overrides so a
template can be re-applied without losing mind-specific edits. The
self-model starts with structural limitations the mind cannot drop
(no policy changes, no influence over shutdown, no cross-mind access).
API, UI (list, overview, editor with sliders/diff/history/import/export,
template editor) and noemactl mind/template commands. All mutations are
audited with field-level diffs.
Phase 1 foundation. Local auth per ADR-0006: Argon2id passwords, opaque
hashed session tokens with idle/absolute expiry, CSRF token plus Origin
check on cookie sessions, per-IP token bucket and per-account exponential
lockout, three roles enforced by middleware, personal API tokens for
noemactl. Audit log writer used by every admin mutation.
API: request ids, panic recovery, strict CSP and security headers, body
limits, single JSON error envelope, health/ready/status, users, sessions,
tokens, audit, token-protected /metrics.
UI: embedded html/template pages (login, first-run setup, dashboard,
system, users, account, audit, interpretation boundary), light/dark
theme, dependency-free app.js with CSRF-aware fetch and SSE helper.
Binaries: noemad with graceful shutdown, migrations at start, bootstrap
admin from environment, supervised background tasks; noemactl with
status, migrate, user create and meaningful exit codes.
Andy Dixon