feat: add provider-neutral LLM layer with secrets, budgets, recording and prompts
Phase 7 (ADR-0004). Cognitive code asks the router for a role; the router maps roles to configured models with ordered fallbacks, applies global and per-mind daily token budgets and a concurrency limit, records every call (request, response, usage, latency, cost, seed, request hash) and can serve recordings back for exact replay. Providers are raw HTTP clients for OpenAI-compatible, Anthropic and Ollama endpoints, with embedding support for the embed role wired into memory retrieval. Secrets are references only: env:, file:, or enc: values encrypted with AES-256-GCM under NOEMA_MASTER_KEY; keys are resolved at call time and never rendered, logged or exported. Outbound HTTP refuses private and loopback addresses at dial time unless a provider explicitly allows it, bounds redirects and caps bodies. Provider errors are scrubbed. Prompts are named, versioned rows with system and user templates and an optional JSON schema; untrusted text enters only through delimited data blocks. Six defaults are seeded and never overwritten. The UI adds the LLM page (providers, models, role matrix, usage, errors, budgets), a provider page with connectivity test, and a prompt manager with editor, test runner, history, restore, clone and diff. CLI: provider, prompt.
This commit is contained in:
Andy Dixon
@@ -175,3 +175,7 @@ tr.dayrow td { background: var(--surface-2); font-weight: 600; font-size: .78rem
|
||||
:root[data-theme="dark"] .badge.st-tentative { background: #3a2a0a; color: #fcd34d; } :root[data-theme="dark"] .badge.st-likely { background: #172554; color: #93c5fd; } :root[data-theme="dark"] .badge.st-contested { background: #3d2e05; color: #fbbf24; }
|
||||
.conf.big .conf-bar.wide { width: 100%; height: 10px; margin: .3rem 0 .8rem; }
|
||||
ul.ev li { padding: .4rem 0; border-bottom: 1px solid var(--border); } ul.ev li:last-child { border-bottom: 0; } ul.ev li.retracted { opacity: .55; }
|
||||
|
||||
/* Prompts */
|
||||
textarea.mono { font-family: var(--mono); font-size: .85rem; }
|
||||
pre.diff .add { color: var(--ok); } pre.diff .del { color: var(--danger); } pre.diff .ctx { color: var(--muted); }
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
{{define "content"}}
|
||||
<div class="pagehead"><h1>Language models</h1><p class="muted">Providers are cognitive service suppliers. Roles decide which model does which job; minds keep their identity whatever you change here.</p></div>
|
||||
{{$admin := .Principal.User.Role.AtLeast "administrator"}}
|
||||
<section class="grid cols-3-1">
|
||||
<div class="stack">
|
||||
<div class="card">
|
||||
<h2>Providers</h2>
|
||||
{{if .Data.Providers}}
|
||||
<table class="table compact">
|
||||
<thead><tr><th>Name</th><th>Kind</th><th>Endpoint</th><th>Key</th><th>Status</th><th>Last test</th><th></th></tr></thead>
|
||||
<tbody>{{range .Data.Providers}}<tr>
|
||||
<td><a href="/llm/providers/{{.ID}}"><strong>{{.Name}}</strong></a></td><td class="small">{{.Kind}}</td><td class="small mono">{{.Endpoint}}</td>
|
||||
<td class="small">{{.KeyDescribed}}</td>
|
||||
<td>{{if .Enabled}}<span class="badge ok">enabled</span>{{else}}<span class="badge">disabled</span>{{end}}{{if .AllowPrivate}} <span class="badge warn" title="May reach private addresses">private ok</span>{{end}}</td>
|
||||
<td class="small">{{if .LastTestAt}}{{if deref .LastTestOK}}<span class="badge ok">ok</span>{{else}}<span class="badge danger">failed</span>{{end}} <span class="muted">{{trunc .LastTestMsg 60}}</span>{{else}}<span class="muted">never</span>{{end}}</td>
|
||||
<td class="nowrap"><form method="post" action="/llm/providers/{{.ID}}/test" class="inline"><input type="hidden" name="_csrf" value="{{$.CSRF}}"><button class="btn small">Test</button></form></td>
|
||||
</tr>{{end}}</tbody>
|
||||
</table>
|
||||
{{else}}<p class="muted">No providers yet. Add one on the right; local Ollama needs “allow private addresses”.</p>{{end}}
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>Models</h2>
|
||||
{{if .Data.Models}}
|
||||
<table class="table compact">
|
||||
<thead><tr><th>Provider</th><th>Model</th><th>Kind</th><th>Context</th><th>Cost in/out per M</th><th>State</th><th></th></tr></thead>
|
||||
<tbody>{{range .Data.Models}}<tr>
|
||||
<td class="small">{{.Provider}}</td><td><code>{{.Name}}</code></td><td class="small">{{.Kind}}{{if .Dims}} ({{.Dims}}d){{end}}</td><td class="small">{{if .ContextWindow}}{{.ContextWindow}}{{else}}—{{end}}</td>
|
||||
<td class="small">${{.InputCostPerM}} / ${{.OutputCostPerM}}</td>
|
||||
<td>{{if .Enabled}}<span class="badge ok">on</span>{{else}}<span class="badge">off</span>{{end}}</td>
|
||||
<td>{{if $admin}}<form method="post" action="/llm/models/{{.ID}}/delete" class="inline"><input type="hidden" name="_csrf" value="{{$.CSRF}}"><button class="btn small btn-danger" data-confirm="Remove model {{.Name}}?">Remove</button></form>{{end}}</td>
|
||||
</tr>{{end}}</tbody>
|
||||
</table>
|
||||
{{else}}<p class="muted">No models yet.</p>{{end}}
|
||||
{{if and $admin .Data.Providers}}
|
||||
<form method="post" action="/llm/models" class="row">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<select name="provider_id" aria-label="Provider">{{range .Data.Providers}}<option value="{{.ID}}">{{.Name}}</option>{{end}}</select>
|
||||
<input name="name" placeholder="model name, e.g. gpt-4o-mini" required size="22">
|
||||
<select name="kind" aria-label="Kind"><option value="chat">chat</option><option value="embedding">embedding</option></select>
|
||||
<input name="context_window" type="number" placeholder="context" size="7" min="0">
|
||||
<input name="input_cost_per_m" type="number" step="0.01" placeholder="$in/M" size="6" min="0">
|
||||
<input name="output_cost_per_m" type="number" step="0.01" placeholder="$out/M" size="6" min="0">
|
||||
<button class="btn small" type="submit">Add model</button>
|
||||
</form>
|
||||
{{end}}
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>Roles <span class="muted small">which model does which cognitive job</span></h2>
|
||||
<table class="table compact">
|
||||
<thead><tr><th>Role</th><th>Purpose</th><th>Primary</th><th>Fallbacks</th><th>Temp</th><th>Max tokens</th>{{if $admin}}<th></th>{{end}}</tr></thead>
|
||||
<tbody>{{range .Data.Roles}}{{$a := .Assignment}}<tr>
|
||||
<form method="post" action="/llm/roles/{{.Info.Role}}">
|
||||
<td><strong>{{.Info.Role}}</strong></td><td class="small muted">{{.Info.Description}} <em>({{.Info.Suggested}})</em></td>
|
||||
<td><input type="hidden" name="_csrf" value="{{$.CSRF}}"><select name="model_id" {{if not $admin}}disabled{{end}}><option value="">— none —</option>{{range $.Data.Models}}<option value="{{.ID}}" {{if and $a.ModelID (eq (print .ID) (print (deref_id $a.ModelID)))}}selected{{end}}>{{.Provider}} / {{.Name}}</option>{{end}}</select></td>
|
||||
<td><select name="fallback_ids" multiple size="2" {{if not $admin}}disabled{{end}}>{{range $.Data.Models}}<option value="{{.ID}}" {{if has_id $a.FallbackIDs .ID}}selected{{end}}>{{.Provider}} / {{.Name}}</option>{{end}}</select></td>
|
||||
<td><input name="temperature" type="number" step="0.05" min="0" max="2" size="4" value="{{if $a.Temperature}}{{deref $a.Temperature}}{{end}}" {{if not $admin}}disabled{{end}}></td>
|
||||
<td><input name="max_tokens" type="number" min="1" size="6" value="{{if $a.MaxTokens}}{{deref_int $a.MaxTokens}}{{end}}" {{if not $admin}}disabled{{end}}></td>
|
||||
{{if $admin}}<td><button class="btn small" type="submit">Save</button></td>{{end}}
|
||||
</form>
|
||||
</tr>{{end}}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>Usage <span class="muted small">last 24 hours</span></h2>
|
||||
{{if .Data.Usage}}<table class="table compact"><thead><tr><th>Provider</th><th>Model</th><th>Role</th><th class="num">Calls</th><th class="num">Errors</th><th class="num">Tokens in</th><th class="num">Tokens out</th><th class="num">Cost</th><th class="num">Avg ms</th></tr></thead>
|
||||
<tbody>{{range .Data.Usage}}<tr><td>{{.Provider}}</td><td><code>{{.Model}}</code></td><td>{{.Role}}</td><td class="num">{{.Calls}}</td><td class="num {{if .Errors}}danger{{end}}">{{.Errors}}</td><td class="num">{{.PromptTokens}}</td><td class="num">{{.OutputTokens}}</td><td class="num">${{printf "%.4f" .Cost}}</td><td class="num">{{printf "%.0f" .AvgLatencyMS}}</td></tr>{{end}}</tbody></table>
|
||||
{{else}}<p class="muted">No calls yet.</p>{{end}}
|
||||
{{if .Data.Errors}}<h3>Recent errors</h3><ul class="plain small">{{range .Data.Errors}}<li><span class="muted mono">{{.CreatedAt.Local.Format "01-02 15:04:05"}}</span> {{.Provider}}/{{.Model}} ({{.Role}}): <span class="danger">{{trunc .Error 160}}</span></li>{{end}}</ul>{{end}}
|
||||
</div>
|
||||
</div>
|
||||
<div class="stack">
|
||||
{{if $admin}}
|
||||
<div class="card">
|
||||
<h2>Add provider</h2>
|
||||
<form method="post" action="/llm/providers" class="stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Name <input name="name" required maxlength="64" placeholder="openai-main"></label>
|
||||
<label>Kind <select name="kind"><option value="openai">OpenAI-compatible (OpenAI, llama.cpp, vLLM, LM Studio)</option><option value="anthropic">Anthropic</option><option value="ollama">Ollama (native)</option></select></label>
|
||||
<label>Endpoint <input name="endpoint" placeholder="leave blank for the kind's default"></label>
|
||||
<label>API key <input type="password" name="api_key" autocomplete="off" placeholder="paste a key, or env:OPENAI_API_KEY"></label>
|
||||
<p class="muted small">Pasted keys are encrypted with the master key and never shown again. References (env:, file:) are resolved at call time.</p>
|
||||
<label class="check"><input type="checkbox" name="allow_private"> Allow private addresses (local Ollama / llama.cpp)</label>
|
||||
<div><button class="btn btn-primary" type="submit">Add provider</button></div>
|
||||
</form>
|
||||
</div>
|
||||
{{end}}
|
||||
<div class="card small">
|
||||
<h2>Budgets</h2>
|
||||
<dl class="kv"><dt>Global</dt><dd>{{.Data.GlobalBudget}} tokens/day</dd><dt>Used today</dt><dd>{{.Data.UsedToday}}</dd><dt>Concurrency</dt><dd>{{.Data.MaxConcurrent}} calls</dd></dl>
|
||||
<p class="muted">Per-mind budgets are in each mind's configuration. When a budget is exhausted, deterministic cognition continues and model-assisted steps are skipped.</p>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
{{end}}
|
||||
@@ -0,0 +1,36 @@
|
||||
{{define "content"}}
|
||||
{{$p := .Data.Prompt}}{{$admin := .Principal.User.Role.AtLeast "administrator"}}
|
||||
<div class="pagehead mindhead"><div><p class="muted small"><a href="/prompts">Prompts</a> › {{$p.Name}}</p><h1><code>{{$p.Name}}</code> <span class="badge">{{$p.Role}}</span> <span class="muted small">v{{$p.CurrentVersion}}</span></h1><p class="muted">{{$p.Description}}</p></div>
|
||||
{{if $admin}}<form method="post" action="/prompts/{{$p.Name}}/clone" class="row"><input type="hidden" name="_csrf" value="{{.CSRF}}"><input name="name" placeholder="clone as…" size="16" required><button class="btn small">Clone</button></form>{{end}}</div>
|
||||
<section class="grid cols-3-1">
|
||||
<div class="stack">
|
||||
<form method="post" action="/prompts/{{$p.Name}}" class="card stack" id="promptform">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Description <input name="description" value="{{$p.Description}}" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>System <textarea name="system" rows="6" class="mono" {{if not $admin}}readonly{{end}}>{{$p.System}}</textarea></label>
|
||||
<label>Template <span class="muted small">Go text/template; use <code>{{"{{"}}data "label" .Field{{"}}"}}</code> for untrusted text</span> <textarea name="template" rows="14" class="mono" required {{if not $admin}}readonly{{end}}>{{$p.Template}}</textarea></label>
|
||||
<label>JSON schema <span class="muted small">optional; when set the model must answer with matching JSON</span> <textarea name="schema" rows="4" class="mono" {{if not $admin}}readonly{{end}}>{{printf "%s" $p.Schema}}</textarea></label>
|
||||
<div class="row"><input name="note" placeholder="change note" class="grow" maxlength="200">{{if $admin}}<button class="btn btn-primary" type="submit">Save as v{{inc $p.CurrentVersion}}</button>{{end}}</div>
|
||||
</form>
|
||||
<div class="card">
|
||||
<h2>Test</h2>
|
||||
<form method="post" action="/prompts/{{$p.Name}}/test" class="stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Sample data (JSON object) <textarea name="data" rows="5" class="mono">{{.Data.Sample}}</textarea></label>
|
||||
<div class="row"><button class="btn" type="submit" name="run" value="0">Render only</button><button class="btn" type="submit" name="run" value="1" {{if not .Data.RoleAvailable}}disabled title="No model assigned to this role"{{end}}>Render and run model</button></div>
|
||||
</form>
|
||||
{{with .Data.Test}}
|
||||
{{with .Error}}<p class="error">{{.}}</p>{{end}}
|
||||
<h3>Rendered system</h3><pre class="json">{{.Rendered.System}}</pre>
|
||||
<h3>Rendered user message</h3><pre class="json">{{(index .Rendered.Messages 0).Content}}</pre>
|
||||
{{with .Response}}<h3>Model response <span class="muted small">{{.Provider}}/{{.Model}} · {{.Usage.TotalTokens}} tokens · {{.Latency.Milliseconds}} ms</span></h3><pre class="json">{{.Text}}</pre>{{end}}
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>History</h2>
|
||||
<table class="table compact"><thead><tr><th>v</th><th>Note</th><th>By</th><th></th></tr></thead><tbody>{{range .Data.Versions}}<tr><td>{{.Version}}{{if eq .Version $p.CurrentVersion}} <span class="badge accent">current</span>{{end}}</td><td class="small">{{.Note}}<div class="muted">{{ago .CreatedAt}}</div></td><td class="small muted">{{.CreatedBy}}</td>
|
||||
<td class="nowrap">{{if ne .Version $p.CurrentVersion}}<a class="small" href="/prompts/{{$p.Name}}/diff?a={{.Version}}&b={{$p.CurrentVersion}}">diff</a> {{if $admin}}<form method="post" action="/prompts/{{$p.Name}}/versions/{{.Version}}/restore" class="inline"><input type="hidden" name="_csrf" value="{{$.CSRF}}"><button class="btn small">restore</button></form>{{end}}{{end}}</td></tr>{{end}}</tbody></table>
|
||||
</div>
|
||||
</section>
|
||||
{{end}}
|
||||
@@ -0,0 +1,5 @@
|
||||
{{define "content"}}
|
||||
<div class="pagehead"><p class="muted small"><a href="/prompts/{{.Data.Name}}">{{.Data.Name}}</a> › diff</p><h1>v{{.Data.A.Version}} → v{{.Data.B.Version}}</h1></div>
|
||||
<div class="card"><h2>Lines</h2><pre class="json diff">{{range .Data.Lines}}<span class="{{.Class}}">{{.Text}}</span>
|
||||
{{end}}</pre></div>
|
||||
{{end}}
|
||||
@@ -0,0 +1,23 @@
|
||||
{{define "content"}}
|
||||
<div class="pagehead"><h1>Prompts</h1><p class="muted">Named, versioned templates. Structured data is injected; untrusted text always travels inside data blocks.</p></div>
|
||||
<section class="grid cols-3-1">
|
||||
<div class="card">
|
||||
<table class="table compact">
|
||||
<thead><tr><th>Name</th><th>Role</th><th>Description</th><th>Version</th><th>Updated</th></tr></thead>
|
||||
<tbody>{{range .Data.Prompts}}<tr><td><a href="/prompts/{{.Name}}"><code>{{.Name}}</code></a></td><td><span class="badge">{{.Role}}</span></td><td class="small">{{.Description}}</td><td class="small">v{{.CurrentVersion}}{{if .Schema}} · JSON{{end}}</td><td class="small muted">{{ago .UpdatedAt}}</td></tr>{{else}}<tr><td colspan="5" class="muted">No prompts.</td></tr>{{end}}</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{{if .Principal.User.Role.AtLeast "administrator"}}
|
||||
<div class="card">
|
||||
<h2>New prompt</h2>
|
||||
<form method="post" action="/prompts" class="stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Name <input name="name" required pattern="[A-Za-z0-9._-]+" placeholder="respond.terse"></label>
|
||||
<label>Role <select name="role">{{range llmRoles}}<option value="{{.Role}}">{{.Role}}</option>{{end}}</select></label>
|
||||
<label>Description <input name="description"></label>
|
||||
<button class="btn btn-primary" type="submit">Create (then edit)</button>
|
||||
</form>
|
||||
</div>
|
||||
{{end}}
|
||||
</section>
|
||||
{{end}}
|
||||
@@ -0,0 +1,31 @@
|
||||
{{define "content"}}
|
||||
{{$p := .Data.Provider}}
|
||||
<div class="pagehead"><p class="muted small"><a href="/llm">Language models</a> › provider</p><h1>{{$p.Name}} <span class="muted small">{{$p.Kind}}</span></h1></div>
|
||||
{{$admin := .Principal.User.Role.AtLeast "administrator"}}
|
||||
<section class="grid cols-2">
|
||||
<form method="post" action="/llm/providers/{{$p.ID}}" class="card stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Name <input name="name" value="{{$p.Name}}" required maxlength="64" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>Kind <select name="kind" {{if not $admin}}disabled{{end}}><option value="openai" {{if eq $p.Kind "openai"}}selected{{end}}>OpenAI-compatible</option><option value="anthropic" {{if eq $p.Kind "anthropic"}}selected{{end}}>Anthropic</option><option value="ollama" {{if eq $p.Kind "ollama"}}selected{{end}}>Ollama</option></select></label>
|
||||
<label>Endpoint <input name="endpoint" value="{{$p.Endpoint}}" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>API key <span class="muted small">currently: {{$p.KeyDescribed}}</span> <input type="password" name="api_key" autocomplete="off" placeholder="leave blank to keep" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>Timeout (ms) <input type="number" name="timeout_ms" value="{{$p.Timeout.Milliseconds}}" min="1000" {{if not $admin}}disabled{{end}}></label>
|
||||
<label class="check"><input type="checkbox" name="allow_private" {{if $p.AllowPrivate}}checked{{end}} {{if not $admin}}disabled{{end}}> Allow private addresses</label>
|
||||
<label class="check"><input type="checkbox" name="enabled" {{if $p.Enabled}}checked{{end}} {{if not $admin}}disabled{{end}}> Enabled</label>
|
||||
{{if $admin}}<div class="row"><button class="btn btn-primary" type="submit">Save</button>
|
||||
<button class="btn" type="submit" formaction="/llm/providers/{{$p.ID}}/clear-key" data-confirm="Remove the stored key reference?">Clear key</button>
|
||||
<button class="btn btn-danger" type="submit" formaction="/llm/providers/{{$p.ID}}/delete" data-confirm="Delete provider {{$p.Name}} and its models?">Delete</button></div>{{end}}
|
||||
</form>
|
||||
<div class="stack">
|
||||
<div class="card">
|
||||
<h2>Connectivity</h2>
|
||||
<p class="small">{{if $p.LastTestAt}}Last test {{ago $p.LastTestAt}}: {{if deref $p.LastTestOK}}<span class="badge ok">ok</span>{{else}}<span class="badge danger">failed</span>{{end}} <span class="muted">{{$p.LastTestMsg}}</span>{{else}}<span class="muted">Never tested.</span>{{end}}</p>
|
||||
<form method="post" action="/llm/providers/{{$p.ID}}/test" class="row"><input type="hidden" name="_csrf" value="{{.CSRF}}"><input name="model" placeholder="model to test (optional)" size="20"><button class="btn" type="submit">Run test</button></form>
|
||||
</div>
|
||||
<div class="card">
|
||||
<h2>Models on this provider</h2>
|
||||
{{if .Data.Models}}<ul class="plain">{{range .Data.Models}}<li><code>{{.Name}}</code> <span class="badge">{{.Kind}}</span></li>{{end}}</ul>{{else}}<p class="muted small">None. Add models on the <a href="/llm">LLM page</a>.</p>{{end}}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
{{end}}
|
||||
Reference in New Issue
Block a user