Andy Dixon
f165adc5f2
Kind `bedrock` speaks the Converse API for chat roles and InvokeModel for Titan and Cohere embeddings. Requests are signed with AWS Signature Version 4 implemented with crypto/hmac and crypto/sha256 only; the signer is pinned by tests to signatures botocore produces for the same requests, with and without a session token, so drift fails the build. The credential is one secret reference, ACCESS_KEY_ID:SECRET_ACCESS_KEY [:SESSION_TOKEN], encrypted together like every other provider key. The region is read from the endpoint host, which must be a Bedrock runtime host over HTTPS unless private endpoints are explicitly allowed; other hosts are refused. Provider errors are scrubbed of the secret and token. Migration 0024 widens the provider kind constraint; existing rows are untouched and the down migration refuses while bedrock rows exist rather than dropping configuration. UI kind selects, docs/llm-providers.md, ADR-0010, ARCHITECTURE and the changelog are updated.
1.5 KiB
ADR-0004: LLM as a replaceable cognitive service
Date: 2026-09-14
Status: Accepted
Context
Minds must survive provider changes; different tasks want different models; every call must be inspectable and replayable; secrets must not leak.
Decision
internal/llm defines LanguageModel (Complete(ctx, CognitiveRequest) (CognitiveResponse, error)) and Embedder. Providers implement raw HTTP clients (OpenAI-compatible, Anthropic Messages, Ollama; Bedrock added in 0.2 behind the same interface with a standard-library SigV4 signer). Provider rows in the database hold endpoint, model, parameters and a secret reference. Roles map cognitive tasks to a primary model and ordered fallbacks. A Recorder wrapper writes every request/response (prompt name+version, rendered messages, parameters, seed, model, usage, latency, estimated cost) and enforces budgets; a Replayer serves recorded responses during replay. Prompts are versioned rows rendered with structured inputs; untrusted text is enclosed in explicit data blocks.
Consequences
- Cognitive code depends only on roles; swapping vendors is configuration.
- Replay of LLM steps is exact when a recorded response exists and approximate otherwise, and is labelled as such.
- Budget enforcement and cost accounting live in one place.
Alternatives considered
- Vendor SDKs: heavy dependency trees, inconsistent behaviour; raw HTTP is small and controllable.
- LangChain-style frameworks in Go: unnecessary abstraction.