andy/noema
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(experiments): cognitive forks per model with per-mind routing and comparison

Browse Source 
A mind's configuration may override model routing per role (llm_roles);
the router honours it for that mind only and fails closed when the named
model is missing or disabled, so a comparison between providers can never
silently fall back to the default. Experiments may declare variants: each
becomes its own fork of the same snapshot with its own configuration,
personality or routing, receives identical stimuli, and is measured. The
results carry one entry per variant and a deterministic comparison: first
divergent stimulus, belief and attention overlap, mean score difference on
shared propositions, and the ranges of memory counts, goal counts, cost and
capability requests. Single-subject definitions are unchanged.
This commit is contained in:
Andy Dixon
2026-08-20 22:26:42 +01:00
parent 522de3b026
commit 844de93c8b
10 changed files with 481 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cmd/noemad/main.go
+14
View File
@@ -33,6 +33,7 @@ import (
"github.com/andydixon/noema/internal/graph"
"github.com/andydixon/noema/internal/identity"
"github.com/andydixon/noema/internal/integrations"
"github.com/andydixon/noema/internal/llm"
"github.com/andydixon/noema/internal/llm/prompts"
"github.com/andydixon/noema/internal/llm/router"
"github.com/andydixon/noema/internal/memory"
@@ -120,6 +121,18 @@ func run() error {
}
deps.LLMStore = router.NewStore(db)
deps.LLM = router.NewRouter(deps.LLMStore, sec, log, metrics, cfg.MaxConcurrentLLM)
mindRoles := func(ctx context.Context, id ids.ID) map[llm.Role]ids.ID {
m, err := deps.Minds.Get(ctx, id)
if err != nil || len(m.Config.LLMRoles) == 0 {
return nil
}
out := map[llm.Role]ids.ID{}
for role, model := range m.Config.LLMRoles {
out[llm.Role(role)] = ids.ID(model)
}
return out
}
deps.LLM.MindRoles = mindRoles
deps.LLM.DailyTokenBudget = cfg.DailyTokenBudget
deps.LLM.MindBudget = func(id ids.ID) int64 {
m, err := deps.Minds.Get(ctx, id)
@@ -172,6 +185,7 @@ func run() error {
// router that serves recorded model responses when it can.
replayLLM := router.NewRouter(deps.LLMStore, sec, log, metrics, cfg.MaxConcurrentLLM)
replayLLM.Replay = true
replayLLM.MindRoles = mindRoles
replayLLM.DailyTokenBudget, replayLLM.MindBudget = deps.LLM.DailyTokenBudget, deps.LLM.MindBudget
sandboxCfg := brainCfg
sandboxCfg.Pool = scheduler.NewPool(2)