docs(site): document the review pass on the published site

Two new chapters (epistemic honesty, privacy) and updates across
concepts, beliefs, conversation, cognition, graph, replay, providers,
security, decisions, development, glossary, API, interface tour and the
landing page. Rebuilt into dixon.cx (website commit d541bc9).
This commit is contained in:
Andy Dixon
2026-08-29 20:02:09 +01:00
parent d63404b358
commit 0840a6c00a
17 changed files with 157 additions and 36 deletions
+7 -2
View File
@@ -1,4 +1,4 @@
These four tools share one idea: a mind's state is data that can be captured, branched, re-run and compared. They are what make Noema a place to *study* cognition rather than merely run it.
These four tools share one idea: a mind's state is data that can be captured, branched, re-run and compared. They are what make Noema a place to *study* cognition rather than merely run it. Since 0.2 every capture and every re-run also records the **runtime identity** it happened under, so a comparison says up front whether it could have been exact; and an experiment can fork one snapshot into **variants** that differ in exactly one declared thing, including which model serves each role.
{{include docs/snapshots-replay.md}}
@@ -9,9 +9,14 @@ Each step of a replay shows the stimulus, the events the source mind produced af
- **exact**: the deterministic code did the same thing and any model calls were served from recordings. This is the common case for perception, retrieval, belief updates and workspace admission.
- **approximate**: a model was called live because no recording matched the request hash (a prompt version changed, or the sandbox's state produced a different request). The step is still valid; it is simply not a reproduction.
- **diverged**: no live model call, yet the outcome differed. The usual causes are honest: retrieval scoring depends on wall-clock recency, working memory carried something into the original that the snapshot did not, or the code changed since the recording. Divergence is a finding, not a failure.
- **redacted**: the stimulus was redacted after the recording; it is skipped and labelled rather than reconstructed.
Above the steps, the replay states its **compatibility** with the snapshot: an **exact candidate** ran under the same build, schema, configuration, routing, prompts and policy, so differences can only come from state, timing or a model; a **different build** lists what differs; an **incompatible** replay has a different schema and stops before running rather than produce a misleading comparison. The first non-exact step is analysed into the event digests only the recording produced, those only the replay produced, and the components they belong to, with live model calls named as a component of their own.
## Designing an experiment
An experiment definition is JSON with five parts: `config` (cognitive configuration overrides by field name), `personality` (section-by-section overrides), `stimuli` (a list of observations with text, optional source and salience), `expected` (event types that should or should not appear, optionally containing a phrase) and a `seed` and `duration_seconds` for provenance and bounding. The run forks the baseline snapshot into a disabled subject mind, applies overrides, feeds the stimuli one at a time until the mind settles after each, and records events by type, produced-event digests per stimulus, the subject's beliefs, memory and goal counts, expectation outcomes, model-call counts and the LLM role routing in force. Results are stored, shown in the interface, exportable with an archive of the subject, and the subject stays for comparison with the baseline.
Typical experiments: memory decay on and off, dreaming on and off, two personalities on the same stimuli, different attention weights, and model comparisons made by changing role routing between runs.
An experiment may instead declare **variants**: named override sets, each becoming its own fork of the same snapshot and receiving the same stimuli. A variant can change configuration, personality and, through `llm_roles`, the model used for each role for that fork only; a missing or disabled model fails the variant rather than falling back silently. The results hold one entry per variant and a deterministic comparison: the first stimulus at which any two diverged, belief overlap and mean score difference on shared propositions, attention overlap, and the ranges of memory, goal, cost and capability-request counts.
Typical experiments: memory decay on and off, dreaming on and off, two personalities on the same stimuli, different attention weights, and the same mind under two providers.