feat(llm): Amazon Bedrock provider over a standard-library SigV4 signer
Kind `bedrock` speaks the Converse API for chat roles and InvokeModel for Titan and Cohere embeddings. Requests are signed with AWS Signature Version 4 implemented with crypto/hmac and crypto/sha256 only; the signer is pinned by tests to signatures botocore produces for the same requests, with and without a session token, so drift fails the build. The credential is one secret reference, ACCESS_KEY_ID:SECRET_ACCESS_KEY [:SESSION_TOKEN], encrypted together like every other provider key. The region is read from the endpoint host, which must be a Bedrock runtime host over HTTPS unless private endpoints are explicitly allowed; other hosts are refused. Provider errors are scrubbed of the secret and token. Migration 0024 widens the provider kind constraint; existing rows are untouched and the down migration refuses while bedrock rows exist rather than dropping configuration. UI kind selects, docs/llm-providers.md, ADR-0010, ARCHITECTURE and the changelog are updated.
This commit is contained in:
Andy Dixon
+1
-1
@@ -95,7 +95,7 @@ A **belief** is a proposition with `status` (hypothesis → tentative → likely
|
||||
|
||||
## 8. Language models and prompts
|
||||
|
||||
`internal/llm` defines `LanguageModel` and `Embedder` interfaces. Providers (OpenAI-compatible, Anthropic, Ollama/llama.cpp; Bedrock behind the same interface later) are configured in the database with secret *references*, not secrets. **Roles** (classify, extract, hypothesise, summarise, narrate, respond, embed…) map to a primary model and fallbacks. Every call is recorded (`llm_request`/`llm_response` events plus `llm_calls` with prompt version, parameters, seed, usage, latency, cost) for inspection and replay.
|
||||
`internal/llm` defines `LanguageModel` and `Embedder` interfaces. Providers (OpenAI-compatible, Anthropic, Ollama/llama.cpp, Amazon Bedrock via a standard-library SigV4 signer) are configured in the database with secret *references*, not secrets. **Roles** (classify, extract, hypothesise, summarise, narrate, respond, embed…) map to a primary model and fallbacks. Every call is recorded (`llm_request`/`llm_response` events plus `llm_calls` with prompt version, parameters, seed, usage, latency, cost) for inspection and replay.
|
||||
|
||||
Prompts are named and versioned rows; templates receive structured data. Untrusted content is passed in clearly delimited data blocks with an explicit instruction that it is data, and model output is parsed into typed JSON and validated. Model output never becomes an action without passing through the capability layer, and never becomes a belief without an evidence link.
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ All notable changes to Noema are recorded here. Format follows Keep a Changelog;
|
||||
- Security suite: SQL-injection effect test, static allow-list scan for assembled SQL, deceived-mind authority test, garbage model output test, self-preservation vocabulary test; threat model rewritten so every verification cites a test that asserts the claim.
|
||||
|
||||
### Added
|
||||
- Amazon Bedrock provider: Converse API for chat roles and Titan/Cohere embeddings, signed with a standard-library SigV4 implementation pinned to botocore reference signatures; credentials as one encrypted reference; non-Bedrock endpoints refused.
|
||||
- Privacy redaction (ADR-0009): the audited path that makes an event, memory, message, model call, belief or assertion irrecoverable while keeping the row, its links, timestamps and scores; a ledger with a hash of the removed content; cascade to derived rows and the events that quote them; stored snapshot archives scrubbed and re-signed; replay labels redacted stimuli. `noema_redact()` is the only write the append-only trigger admits. Migration 0023. `docs/privacy.md` defines retained, redacted, irrecoverable and deleted.
|
||||
- Causal timeline: `GET /api/v1/events/{id}/chain` and `/events/{id}/timeline` walk the links events actually record (causation, parents, cited evidence) backwards and forwards within bounded depth and size, label edge kinds, group by cycle as correlation only, and infer nothing.
|
||||
- Cognitive forks per model: minds may override model routing per role (`llm_roles`, fail-closed); experiments may declare variants that fork one snapshot under different configuration, personality or routing, receive identical stimuli, and are compared deterministically (first divergent stimulus, belief and attention overlap, score differences, memory, goal, cost and capability-request ranges).
|
||||
|
||||
@@ -7,7 +7,7 @@ Status: Accepted
|
||||
Minds must survive provider changes; different tasks want different models; every call must be inspectable and replayable; secrets must not leak.
|
||||
|
||||
## Decision
|
||||
`internal/llm` defines `LanguageModel` (`Complete(ctx, CognitiveRequest) (CognitiveResponse, error)`) and `Embedder`. Providers implement raw HTTP clients (OpenAI-compatible, Anthropic Messages, Ollama; Bedrock later behind the same interface). Provider rows in the database hold endpoint, model, parameters and a *secret reference*. **Roles** map cognitive tasks to a primary model and ordered fallbacks. A `Recorder` wrapper writes every request/response (prompt name+version, rendered messages, parameters, seed, model, usage, latency, estimated cost) and enforces budgets; a `Replayer` serves recorded responses during replay. Prompts are versioned rows rendered with structured inputs; untrusted text is enclosed in explicit data blocks.
|
||||
`internal/llm` defines `LanguageModel` (`Complete(ctx, CognitiveRequest) (CognitiveResponse, error)`) and `Embedder`. Providers implement raw HTTP clients (OpenAI-compatible, Anthropic Messages, Ollama; Bedrock added in 0.2 behind the same interface with a standard-library SigV4 signer). Provider rows in the database hold endpoint, model, parameters and a *secret reference*. **Roles** map cognitive tasks to a primary model and ordered fallbacks. A `Recorder` wrapper writes every request/response (prompt name+version, rendered messages, parameters, seed, model, usage, latency, estimated cost) and enforces budgets; a `Replayer` serves recorded responses during replay. Prompts are versioned rows rendered with structured inputs; untrusted text is enclosed in explicit data blocks.
|
||||
|
||||
## Consequences
|
||||
- Cognitive code depends only on roles; swapping vendors is configuration.
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# ADR-0010: Amazon Bedrock over a standard-library SigV4 signer
|
||||
|
||||
Date: 2026-09-15
|
||||
Status: Accepted
|
||||
|
||||
## Context
|
||||
Operators asked for Amazon Bedrock. ADR-0004 kept the LLM layer replaceable and deferred Bedrock until a request signer was justified. The AWS SDK for Go is large, pulls in many transitive modules, and would be the only non-database dependency in the runtime; the surface Noema needs is one signed POST per call.
|
||||
|
||||
## Decision
|
||||
Implement AWS Signature Version 4 in `internal/llm/providers/sigv4.go` with `crypto/hmac` and `crypto/sha256` only, restricted to what Bedrock uses (POST, JSON body, host, date, content type, optional session token, no query string). The signer is pinned by tests to signatures produced by botocore for the same requests, with and without a session token, so drift from the reference implementation fails the build. Chat uses the Converse API, embeddings use InvokeModel with the Titan and Cohere shapes. The credential is one secret reference, `ACCESS_KEY_ID:SECRET_ACCESS_KEY[:SESSION_TOKEN]`, encrypted together like every other provider key. The region is read from the endpoint host, which must be a `bedrock-runtime.<region>.amazonaws.com` URL over HTTPS; other hosts are refused unless the provider explicitly allows private endpoints (tests and local gateways). Provider errors are scrubbed of the secret and token before they reach logs or the UI.
|
||||
|
||||
## Consequences
|
||||
- No new module dependency; the module graph stays auditable.
|
||||
- Only the signed request shapes Bedrock needs are supported; presigned URLs, S3-style single encoding and chunked signing are out of scope and would need new tests against the reference signer.
|
||||
- Converse has no JSON-mode switch, so JSON roles rely on the instruction plus Noema's own parsing and validation, as with every provider.
|
||||
- Temporary credentials expire; rotation is the operator's job through the ordinary secret-reference update path.
|
||||
@@ -2,6 +2,10 @@
|
||||
|
||||
Language models are **cognitive service providers**. Cognitive code never names a vendor; it asks the router for a *role* and receives a response with a call id it can cite as evidence (ADR-0004).
|
||||
|
||||
## Amazon Bedrock
|
||||
|
||||
Kind `bedrock` speaks the Bedrock Converse API (`/model/{id}/converse`) and, for embeddings, `/model/{id}/invoke` with the Titan and Cohere request shapes. Requests are signed with AWS Signature Version 4 implemented over the standard library (HMAC-SHA256; the canonical URI is encoded twice as the specification requires outside S3) and pinned by tests to signatures produced by botocore. The endpoint is `https://bedrock-runtime.<region>.amazonaws.com`; the region is taken from the host and any other host is refused. The provider's API key reference resolves to one string, `ACCESS_KEY_ID:SECRET_ACCESS_KEY` or `ACCESS_KEY_ID:SECRET_ACCESS_KEY:SESSION_TOKEN`, so the whole credential is encrypted together and never split across columns. Provider errors are scrubbed of the secret and session token. Converse has no JSON-mode switch; when a role requests JSON the instruction is added to the system text and the answer is parsed and validated on Noema's side as for every other provider.
|
||||
|
||||
## Per-mind routing
|
||||
|
||||
A mind's cognitive configuration may carry `llm_roles`, a map from role to model id, which overrides the daemon-wide routing for that mind alone. It exists so that forks of one snapshot can run on different providers in a controlled experiment. An override names exactly one model; if that model is missing or disabled the call fails with "no model" rather than falling back, and the runtime identity records the override next to the daemon routing.
|
||||
@@ -22,7 +26,7 @@ Each role has a primary model, ordered fallbacks, and optional temperature / max
|
||||
|
||||
## Providers and models
|
||||
|
||||
Provider kinds: `openai` (OpenAI and every OpenAI-compatible server: llama.cpp, vLLM, LM Studio, Ollama's `/v1`), `anthropic` (Messages API), `ollama` (native API with seed and JSON mode). AWS Bedrock is not yet implemented; it fits behind the same interface once a SigV4 signer is added.
|
||||
Provider kinds: `openai` (OpenAI and every OpenAI-compatible server: llama.cpp, vLLM, LM Studio, Ollama's `/v1`), `anthropic` (Messages API), `ollama` (native API with seed and JSON mode), `bedrock` (Amazon Bedrock Converse API, see below).
|
||||
|
||||
Endpoints that resolve to private or loopback addresses are refused unless the provider has **allow private addresses** set (SSRF guard, threat T6). Credentials go in the API key field only; `Authorization` and `x-api-key` headers are rejected in the custom headers map.
|
||||
|
||||
|
||||
+6
-1
@@ -116,10 +116,13 @@ const (
|
||||
KindOpenAI ProviderKind = "openai"
|
||||
KindAnthropic ProviderKind = "anthropic"
|
||||
KindOllama ProviderKind = "ollama"
|
||||
KindBedrock ProviderKind = "bedrock" // Amazon Bedrock Converse API, SigV4-signed
|
||||
)
|
||||
|
||||
// Valid reports whether k is supported.
|
||||
func (k ProviderKind) Valid() bool { return k == KindOpenAI || k == KindAnthropic || k == KindOllama }
|
||||
func (k ProviderKind) Valid() bool {
|
||||
return k == KindOpenAI || k == KindAnthropic || k == KindOllama || k == KindBedrock
|
||||
}
|
||||
|
||||
// DefaultEndpoint suggests a base URL per kind.
|
||||
func (k ProviderKind) DefaultEndpoint() string {
|
||||
@@ -130,6 +133,8 @@ func (k ProviderKind) DefaultEndpoint() string {
|
||||
return "https://api.anthropic.com"
|
||||
case KindOllama:
|
||||
return "http://127.0.0.1:11434"
|
||||
case KindBedrock:
|
||||
return "https://bedrock-runtime.eu-west-2.amazonaws.com"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -0,0 +1,234 @@
|
||||
package providers
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/andydixon/noema/internal/llm"
|
||||
)
|
||||
|
||||
// Bedrock speaks the Amazon Bedrock Converse API with SigV4 signing.
|
||||
// The API key reference resolves to "ACCESS_KEY_ID:SECRET_ACCESS_KEY[:SESSION_TOKEN]".
|
||||
// The region comes from the endpoint host (bedrock-runtime.<region>.amazonaws.com).
|
||||
type Bedrock struct {
|
||||
cfg Config
|
||||
region string
|
||||
creds awsCredentials
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
var reBedrockHost = regexp.MustCompile(`^bedrock-runtime(?:-fips)?\.([a-z0-9-]+)\.amazonaws\.com(?:\.cn)?$`)
|
||||
|
||||
// ErrBedrockConfig marks an unusable provider configuration.
|
||||
var ErrBedrockConfig = errors.New("bedrock: configuration")
|
||||
|
||||
// NewBedrock creates a client. It fails closed on a malformed credential or
|
||||
// an endpoint that is not a Bedrock runtime host (unless private endpoints
|
||||
// are explicitly allowed, which is how tests point it at a local server).
|
||||
func NewBedrock(cfg Config) (*Bedrock, error) {
|
||||
creds, ok := parseAWSCredentials(cfg.APIKey)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%w: credential must be ACCESS_KEY_ID:SECRET_ACCESS_KEY[:SESSION_TOKEN]", ErrBedrockConfig)
|
||||
}
|
||||
u, err := url.Parse(cfg.Endpoint)
|
||||
if err != nil || u.Host == "" {
|
||||
return nil, fmt.Errorf("%w: endpoint must be a URL", ErrBedrockConfig)
|
||||
}
|
||||
region := ""
|
||||
if m := reBedrockHost.FindStringSubmatch(u.Hostname()); m != nil {
|
||||
region = m[1]
|
||||
} else if cfg.AllowPrivate {
|
||||
region = firstNonEmpty(cfg.Headers["X-Noema-Region"], "us-east-1")
|
||||
}
|
||||
if region == "" || (u.Scheme != "https" && !cfg.AllowPrivate) {
|
||||
return nil, fmt.Errorf("%w: endpoint must be https://bedrock-runtime.<region>.amazonaws.com", ErrBedrockConfig)
|
||||
}
|
||||
return &Bedrock{cfg: cfg, region: region, creds: creds, now: time.Now}, nil
|
||||
}
|
||||
|
||||
// Complete implements llm.LanguageModel over /model/{id}/converse.
|
||||
func (p *Bedrock) Complete(ctx context.Context, req llm.Request) (llm.Response, error) {
|
||||
type block struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
type msg struct {
|
||||
Role string `json:"role"`
|
||||
Content []block `json:"content"`
|
||||
}
|
||||
body := map[string]any{}
|
||||
var msgs []msg
|
||||
for _, m := range req.Messages {
|
||||
role := m.Role
|
||||
if role != "assistant" {
|
||||
role = "user"
|
||||
}
|
||||
msgs = append(msgs, msg{Role: role, Content: []block{{Text: m.Content}}})
|
||||
}
|
||||
body["messages"] = msgs
|
||||
system := req.System
|
||||
if len(req.JSONSchema) > 0 {
|
||||
// Converse has no JSON mode switch; the instruction is explicit and the
|
||||
// answer is still parsed and validated on our side.
|
||||
system = strings.TrimSpace(system + "\nAnswer with a single JSON object and nothing else.")
|
||||
}
|
||||
if system != "" {
|
||||
body["system"] = []block{{Text: system}}
|
||||
}
|
||||
inf := map[string]any{}
|
||||
if req.MaxTokens > 0 {
|
||||
inf["maxTokens"] = req.MaxTokens
|
||||
}
|
||||
if req.Temperature != nil {
|
||||
inf["temperature"] = *req.Temperature
|
||||
}
|
||||
if req.TopP != nil {
|
||||
inf["topP"] = *req.TopP
|
||||
}
|
||||
if len(req.Stop) > 0 {
|
||||
inf["stopSequences"] = req.Stop
|
||||
}
|
||||
if len(inf) > 0 {
|
||||
body["inferenceConfig"] = inf
|
||||
}
|
||||
var out struct {
|
||||
Output struct {
|
||||
Message struct {
|
||||
Content []block `json:"content"`
|
||||
} `json:"message"`
|
||||
} `json:"output"`
|
||||
StopReason string `json:"stopReason"`
|
||||
Usage struct {
|
||||
InputTokens int `json:"inputTokens"`
|
||||
OutputTokens int `json:"outputTokens"`
|
||||
TotalTokens int `json:"totalTokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
start := time.Now()
|
||||
if err := p.post(ctx, "/model/"+awsPathSegment(p.cfg.Model)+"/converse", body, &out); err != nil {
|
||||
return llm.Response{}, err
|
||||
}
|
||||
resp := llm.Response{Provider: p.cfg.Name, Model: p.cfg.Model, Latency: time.Since(start), FinishReason: out.StopReason,
|
||||
Usage: llm.Usage{PromptTokens: out.Usage.InputTokens, CompletionTokens: out.Usage.OutputTokens, TotalTokens: out.Usage.TotalTokens}}
|
||||
var text strings.Builder
|
||||
for _, b := range out.Output.Message.Content {
|
||||
text.WriteString(b.Text)
|
||||
}
|
||||
resp.Text = text.String()
|
||||
return resp, finish(req, &resp)
|
||||
}
|
||||
|
||||
// post signs and sends a JSON request, decoding JSON and scrubbing errors.
|
||||
func (p *Bedrock) post(ctx context.Context, path string, in any, out any) error {
|
||||
body, err := json.Marshal(in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.cfg.url(path), bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json")
|
||||
req.Header.Set("User-Agent", "noema/1")
|
||||
for k, v := range p.cfg.Headers {
|
||||
if !strings.HasPrefix(k, "X-Noema-") {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
}
|
||||
signV4(req, body, p.creds, p.region, "bedrock", p.now())
|
||||
res, err := p.cfg.client().Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%w: %s: %s", llm.ErrProvider, p.cfg.Name, scrubAWS(err.Error(), p.creds.SecretAccessKey, p.creds.SessionToken))
|
||||
}
|
||||
defer res.Body.Close()
|
||||
raw, _ := io.ReadAll(io.LimitReader(res.Body, 16<<20))
|
||||
if res.StatusCode < 200 || res.StatusCode >= 300 {
|
||||
return fmt.Errorf("%w: %s: HTTP %d: %s", llm.ErrProvider, p.cfg.Name, res.StatusCode, scrubAWS(truncate(string(raw), 400), p.creds.SecretAccessKey, p.creds.SessionToken))
|
||||
}
|
||||
if err := json.Unmarshal(raw, out); err != nil {
|
||||
return fmt.Errorf("%w: %s: bad JSON response", llm.ErrProvider, p.cfg.Name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// scrubAWS removes the credential secrets and anything scrub already covers.
|
||||
func scrubAWS(s string, secrets ...string) string {
|
||||
for _, sec := range secrets {
|
||||
if sec != "" {
|
||||
s = strings.ReplaceAll(s, sec, "[redacted]")
|
||||
}
|
||||
}
|
||||
return scrub(s)
|
||||
}
|
||||
|
||||
// awsPathSegment percent-encodes a model id (which may contain ':' or an
|
||||
// ARN's '/') once for the request path; the signer encodes it again.
|
||||
func awsPathSegment(s string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(s); i++ {
|
||||
c := s[i]
|
||||
switch {
|
||||
case (c >= 'A' && c <= 'Z'), (c >= 'a' && c <= 'z'), (c >= '0' && c <= '9'), c == '-', c == '_', c == '.', c == '~':
|
||||
b.WriteByte(c)
|
||||
default:
|
||||
b.WriteString(fmt.Sprintf("%%%02X", c))
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// BedrockEmbedder calls /model/{id}/invoke for Titan and Cohere embedding models.
|
||||
type BedrockEmbedder struct {
|
||||
p *Bedrock
|
||||
dims int
|
||||
}
|
||||
|
||||
// NewBedrockEmbedder creates an embedder.
|
||||
func NewBedrockEmbedder(cfg Config, dims int) (*BedrockEmbedder, error) {
|
||||
p, err := NewBedrock(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &BedrockEmbedder{p: p, dims: dims}, nil
|
||||
}
|
||||
|
||||
// Model implements embeddings.Embedder.
|
||||
func (e *BedrockEmbedder) Model() string { return "bedrock/" + e.p.cfg.Model }
|
||||
|
||||
// Dims implements embeddings.Embedder.
|
||||
func (e *BedrockEmbedder) Dims() int { return e.dims }
|
||||
|
||||
// Embed implements embeddings.Embedder. Titan takes one text per call;
|
||||
// Cohere takes a batch.
|
||||
func (e *BedrockEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error) {
|
||||
path := "/model/" + awsPathSegment(e.p.cfg.Model) + "/invoke"
|
||||
if strings.HasPrefix(e.p.cfg.Model, "cohere.") {
|
||||
var out struct {
|
||||
Embeddings [][]float32 `json:"embeddings"`
|
||||
}
|
||||
if err := e.p.post(ctx, path, map[string]any{"texts": texts, "input_type": "search_document", "truncate": "END"}, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out.Embeddings, nil
|
||||
}
|
||||
var vectors [][]float32
|
||||
for _, t := range texts {
|
||||
var out struct {
|
||||
Embedding []float32 `json:"embedding"`
|
||||
}
|
||||
if err := e.p.post(ctx, path, map[string]any{"inputText": t}, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
vectors = append(vectors, out.Embedding)
|
||||
}
|
||||
return vectors, nil
|
||||
}
|
||||
@@ -1,13 +1,16 @@
|
||||
package providers
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/andydixon/noema/internal/llm"
|
||||
)
|
||||
@@ -99,3 +102,95 @@ func TestPrivateEndpointRefusedByDefault(t *testing.T) {
|
||||
t.Fatalf("loopback endpoint should need allow_private: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSigV4MatchesReferenceSigner pins the signer to signatures computed by
|
||||
// botocore for the same request, with and without a session token.
|
||||
func TestSigV4MatchesReferenceSigner(t *testing.T) {
|
||||
body := []byte(`{"messages":[{"role":"user","content":[{"text":"hi"}]}]}`)
|
||||
at := time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC)
|
||||
mk := func() *http.Request {
|
||||
req, _ := http.NewRequest(http.MethodPost, "https://bedrock-runtime.eu-west-2.amazonaws.com/model/"+awsPathSegment("anthropic.claude-3-haiku-20240307-v1:0")+"/converse", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
return req
|
||||
}
|
||||
req := mk()
|
||||
signV4(req, body, awsCredentials{AccessKeyID: "AKIDEXAMPLE", SecretAccessKey: "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY"}, "eu-west-2", "bedrock", at)
|
||||
want := "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20260915/eu-west-2/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=91ab2cb0d4ac1c8ff730487209ce46355c4d033b4d693361ca422142f0f66c9f"
|
||||
if got := req.Header.Get("Authorization"); got != want {
|
||||
t.Fatalf("signature\n got %s\nwant %s", got, want)
|
||||
}
|
||||
if req.Header.Get("X-Amz-Date") != "20260915T100000Z" {
|
||||
t.Fatal(req.Header.Get("X-Amz-Date"))
|
||||
}
|
||||
req = mk()
|
||||
signV4(req, body, awsCredentials{AccessKeyID: "AKIDEXAMPLE", SecretAccessKey: "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", SessionToken: "SESSIONTOKENEXAMPLE"}, "eu-west-2", "bedrock", at)
|
||||
want = "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20260915/eu-west-2/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=72345f5e4ff70084eb6c87e22a2b47c7234de67f71fe2faca1e7840a74c66a2b"
|
||||
if got := req.Header.Get("Authorization"); got != want || req.Header.Get("X-Amz-Security-Token") != "SESSIONTOKENEXAMPLE" {
|
||||
t.Fatalf("session signature\n got %s\nwant %s", got, want)
|
||||
}
|
||||
if c, ok := parseAWSCredentials("AKID:secret:tok"); !ok || c.SessionToken != "tok" {
|
||||
t.Fatal("credential parsing")
|
||||
}
|
||||
if _, ok := parseAWSCredentials("just-a-key"); ok {
|
||||
t.Fatal("a bare key must be rejected")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBedrockWireFormat: Converse request shape, signed headers, response
|
||||
// parsing, JSON-mode instruction, and embeddings for Titan and Cohere.
|
||||
func TestBedrockWireFormat(t *testing.T) {
|
||||
var got map[string]any
|
||||
var auth, date, path string
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
auth, date, path = r.Header.Get("Authorization"), r.Header.Get("X-Amz-Date"), r.URL.RawPath
|
||||
if path == "" {
|
||||
path = r.URL.Path
|
||||
}
|
||||
json.NewDecoder(r.Body).Decode(&got) //nolint:errcheck
|
||||
switch {
|
||||
case strings.HasSuffix(r.URL.Path, "/converse"):
|
||||
w.Write([]byte(`{"output":{"message":{"role":"assistant","content":[{"text":"{\"intent\":\"statement\"}"}]}},"stopReason":"end_turn","usage":{"inputTokens":7,"outputTokens":3,"totalTokens":10}}`)) //nolint:errcheck
|
||||
case strings.Contains(r.URL.Path, "cohere"):
|
||||
w.Write([]byte(`{"embeddings":[[0.1,0.2],[0.3,0.4]]}`)) //nolint:errcheck
|
||||
default:
|
||||
w.Write([]byte(`{"embedding":[0.5,0.6]}`)) //nolint:errcheck
|
||||
}
|
||||
}))
|
||||
defer srv.Close()
|
||||
if _, err := NewBedrock(Config{Name: "b", Endpoint: "https://bedrock-runtime.eu-west-2.amazonaws.com", APIKey: "nokey", Model: "m"}); err == nil {
|
||||
t.Fatal("malformed credential accepted")
|
||||
}
|
||||
if _, err := NewBedrock(Config{Name: "b", Endpoint: "https://example.com", APIKey: "a:b", Model: "m"}); err == nil {
|
||||
t.Fatal("non-Bedrock endpoint accepted")
|
||||
}
|
||||
p, err := NewBedrock(Config{Name: "b", Endpoint: srv.URL, APIKey: "AKID:SECRET:TOKEN", Model: "anthropic.claude-3-haiku-20240307-v1:0", AllowPrivate: true, Headers: map[string]string{"X-Noema-Region": "eu-west-2"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p.now = func() time.Time { return time.Date(2026, 9, 15, 10, 0, 0, 0, time.UTC) }
|
||||
resp, err := p.Complete(context.Background(), llm.Request{System: "be brief", Messages: []llm.Message{{Role: "user", Content: "hi"}}, MaxTokens: 50, JSONSchema: json.RawMessage(`{"type":"object"}`)})
|
||||
if err != nil || resp.Text == "" || resp.Usage.TotalTokens != 10 || string(resp.JSON) != `{"intent":"statement"}` || resp.FinishReason != "end_turn" {
|
||||
t.Fatalf("converse: %v %+v", err, resp)
|
||||
}
|
||||
if !strings.HasPrefix(auth, "AWS4-HMAC-SHA256 Credential=AKID/20260915/eu-west-2/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=") || date != "20260915T100000Z" {
|
||||
t.Fatalf("signing headers: %q %q", auth, date)
|
||||
}
|
||||
if path != "/model/anthropic.claude-3-haiku-20240307-v1%3A0/converse" {
|
||||
t.Fatalf("model id must be encoded once in the path: %s", path)
|
||||
}
|
||||
sys := got["system"].([]any)[0].(map[string]any)["text"].(string)
|
||||
if !strings.HasPrefix(sys, "be brief") || !strings.Contains(sys, "single JSON object") || got["inferenceConfig"].(map[string]any)["maxTokens"] != float64(50) {
|
||||
t.Fatalf("request body: %+v", got)
|
||||
}
|
||||
if strings.Contains(fmt.Sprint(got), "SECRET") {
|
||||
t.Fatal("secret leaked into the body")
|
||||
}
|
||||
titan, _ := NewBedrockEmbedder(Config{Name: "b", Endpoint: srv.URL, APIKey: "AKID:SECRET", Model: "amazon.titan-embed-text-v2:0", AllowPrivate: true}, 2)
|
||||
if v, err := titan.Embed(context.Background(), []string{"a", "b"}); err != nil || len(v) != 2 || v[1][0] != 0.5 || titan.Model() != "bedrock/amazon.titan-embed-text-v2:0" {
|
||||
t.Fatalf("titan: %v %v", err, v)
|
||||
}
|
||||
cohere, _ := NewBedrockEmbedder(Config{Name: "b", Endpoint: srv.URL, APIKey: "AKID:SECRET", Model: "cohere.embed-english-v3", AllowPrivate: true}, 2)
|
||||
if v, err := cohere.Embed(context.Background(), []string{"a", "b"}); err != nil || len(v) != 2 || v[1][1] != 0.4 || got["input_type"] != "search_document" {
|
||||
t.Fatalf("cohere: %v %v", err, v)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
package providers
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// AWS Signature Version 4 over the standard library. Only what Bedrock
|
||||
// needs: POST with a JSON body, a handful of headers, no query string. The
|
||||
// canonical URI is the request path URI-encoded a second time, as the
|
||||
// specification requires for every service except S3.
|
||||
|
||||
type awsCredentials struct {
|
||||
AccessKeyID string
|
||||
SecretAccessKey string
|
||||
SessionToken string
|
||||
}
|
||||
|
||||
// parseAWSCredentials reads "ACCESS_KEY_ID:SECRET_ACCESS_KEY[:SESSION_TOKEN]"
|
||||
// from a resolved secret reference. One reference keeps the whole
|
||||
// credential encrypted together; the secret never leaves this package.
|
||||
func parseAWSCredentials(s string) (awsCredentials, bool) {
|
||||
parts := strings.SplitN(strings.TrimSpace(s), ":", 3)
|
||||
if len(parts) < 2 || parts[0] == "" || parts[1] == "" {
|
||||
return awsCredentials{}, false
|
||||
}
|
||||
c := awsCredentials{AccessKeyID: parts[0], SecretAccessKey: parts[1]}
|
||||
if len(parts) == 3 {
|
||||
c.SessionToken = parts[2]
|
||||
}
|
||||
return c, true
|
||||
}
|
||||
|
||||
const (
|
||||
amzDateFormat = "20060102T150405Z"
|
||||
amzShortDate = "20060102"
|
||||
)
|
||||
|
||||
// signV4 adds X-Amz-Date, optionally X-Amz-Security-Token, and Authorization
|
||||
// to req for the given body. The Host header is signed; Content-Type is
|
||||
// signed when present.
|
||||
func signV4(req *http.Request, body []byte, creds awsCredentials, region, service string, now time.Time) {
|
||||
t := now.UTC()
|
||||
amzDate := t.Format(amzDateFormat)
|
||||
req.Header.Set("X-Amz-Date", amzDate)
|
||||
if creds.SessionToken != "" {
|
||||
req.Header.Set("X-Amz-Security-Token", creds.SessionToken)
|
||||
}
|
||||
host := req.Host
|
||||
if host == "" {
|
||||
host = req.URL.Host
|
||||
}
|
||||
headers := map[string]string{"host": host, "x-amz-date": amzDate}
|
||||
if ct := req.Header.Get("Content-Type"); ct != "" {
|
||||
headers["content-type"] = strings.TrimSpace(ct)
|
||||
}
|
||||
if creds.SessionToken != "" {
|
||||
headers["x-amz-security-token"] = creds.SessionToken
|
||||
}
|
||||
names := make([]string, 0, len(headers))
|
||||
for k := range headers {
|
||||
names = append(names, k)
|
||||
}
|
||||
sort.Strings(names)
|
||||
var canonHeaders strings.Builder
|
||||
for _, k := range names {
|
||||
canonHeaders.WriteString(k + ":" + headers[k] + "\n")
|
||||
}
|
||||
signed := strings.Join(names, ";")
|
||||
payload := sha256.Sum256(body)
|
||||
path := req.URL.EscapedPath()
|
||||
if path == "" {
|
||||
path = "/"
|
||||
}
|
||||
canonical := strings.Join([]string{req.Method, canonicalURI(path), canonicalQuery(req.URL.RawQuery), canonHeaders.String(), signed, hex.EncodeToString(payload[:])}, "\n")
|
||||
scope := t.Format(amzShortDate) + "/" + region + "/" + service + "/aws4_request"
|
||||
cr := sha256.Sum256([]byte(canonical))
|
||||
toSign := strings.Join([]string{"AWS4-HMAC-SHA256", amzDate, scope, hex.EncodeToString(cr[:])}, "\n")
|
||||
kDate := hmacSHA256([]byte("AWS4"+creds.SecretAccessKey), t.Format(amzShortDate))
|
||||
kRegion := hmacSHA256(kDate, region)
|
||||
kService := hmacSHA256(kRegion, service)
|
||||
kSigning := hmacSHA256(kService, "aws4_request")
|
||||
sig := hex.EncodeToString(hmacSHA256(kSigning, toSign))
|
||||
req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential="+creds.AccessKeyID+"/"+scope+", SignedHeaders="+signed+", Signature="+sig)
|
||||
}
|
||||
|
||||
func hmacSHA256(key []byte, data string) []byte {
|
||||
h := hmac.New(sha256.New, key)
|
||||
h.Write([]byte(data))
|
||||
return h.Sum(nil)
|
||||
}
|
||||
|
||||
// canonicalURI URI-encodes an already-encoded path once more, keeping '/'
|
||||
// and unreserved characters, which is what SigV4 requires outside S3.
|
||||
func canonicalURI(path string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(path); i++ {
|
||||
c := path[i]
|
||||
switch {
|
||||
case c == '/', (c >= 'A' && c <= 'Z'), (c >= 'a' && c <= 'z'), (c >= '0' && c <= '9'), c == '-', c == '_', c == '.', c == '~':
|
||||
b.WriteByte(c)
|
||||
default:
|
||||
b.WriteString("%" + strings.ToUpper(hex.EncodeToString([]byte{c})))
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// canonicalQuery sorts and encodes query parameters. Bedrock calls send none,
|
||||
// but the function keeps the signer complete.
|
||||
func canonicalQuery(raw string) string {
|
||||
if raw == "" {
|
||||
return ""
|
||||
}
|
||||
parts := strings.Split(raw, "&")
|
||||
sort.Strings(parts)
|
||||
return strings.Join(parts, "&")
|
||||
}
|
||||
@@ -157,6 +157,8 @@ func (r *Router) build(c routing, m Model) (llm.LanguageModel, error) {
|
||||
return providers.NewAnthropic(cfg), nil
|
||||
case llm.KindOllama:
|
||||
return providers.NewOllama(cfg), nil
|
||||
case llm.KindBedrock:
|
||||
return providers.NewBedrock(cfg)
|
||||
}
|
||||
return nil, fmt.Errorf("provider %s: unsupported kind %s", p.Name, p.Kind)
|
||||
}
|
||||
@@ -187,6 +189,11 @@ func (r *Router) Embedder(ctx context.Context) embeddings.Embedder {
|
||||
switch p.Kind {
|
||||
case llm.KindOllama:
|
||||
return providers.NewOllamaEmbedder(cfg, m.Dims)
|
||||
case llm.KindBedrock:
|
||||
if e, err := providers.NewBedrockEmbedder(cfg, m.Dims); err == nil {
|
||||
return e
|
||||
}
|
||||
return embeddings.None{}
|
||||
case llm.KindOpenAI:
|
||||
return providers.NewOpenAIEmbedder(cfg, m.Dims)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
-- Fails if any bedrock provider rows exist; delete them first (deliberate: never silently drop configuration).
|
||||
ALTER TABLE llm_providers DROP CONSTRAINT IF EXISTS llm_providers_kind_check;
|
||||
ALTER TABLE llm_providers ADD CONSTRAINT llm_providers_kind_check CHECK (kind IN ('openai','anthropic','ollama'));
|
||||
@@ -0,0 +1,3 @@
|
||||
-- Allow the Amazon Bedrock provider kind. Existing rows are unaffected.
|
||||
ALTER TABLE llm_providers DROP CONSTRAINT IF EXISTS llm_providers_kind_check;
|
||||
ALTER TABLE llm_providers ADD CONSTRAINT llm_providers_kind_check CHECK (kind IN ('openai','anthropic','ollama','bedrock'));
|
||||
@@ -75,7 +75,7 @@
|
||||
<form method="post" action="/llm/providers" class="stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Name <input name="name" required maxlength="64" placeholder="openai-main"></label>
|
||||
<label>Kind <select name="kind"><option value="openai">OpenAI-compatible (OpenAI, llama.cpp, vLLM, LM Studio)</option><option value="anthropic">Anthropic</option><option value="ollama">Ollama (native)</option></select></label>
|
||||
<label>Kind <select name="kind"><option value="openai">OpenAI-compatible (OpenAI, llama.cpp, vLLM, LM Studio)</option><option value="anthropic">Anthropic</option><option value="ollama">Ollama (native)</option><option value="bedrock">AWS Bedrock (Converse API; key as ACCESS_KEY_ID:SECRET_ACCESS_KEY[:SESSION_TOKEN])</option></select></label>
|
||||
<label>Endpoint <input name="endpoint" placeholder="leave blank for the kind's default"></label>
|
||||
<label>API key <input type="password" name="api_key" autocomplete="off" placeholder="paste a key, or env:OPENAI_API_KEY"></label>
|
||||
<p class="muted small">Pasted keys are encrypted with the master key and never shown again. References (env:, file:) are resolved at call time.</p>
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<form method="post" action="/llm/providers/{{$p.ID}}" class="card stack">
|
||||
<input type="hidden" name="_csrf" value="{{.CSRF}}">
|
||||
<label>Name <input name="name" value="{{$p.Name}}" required maxlength="64" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>Kind <select name="kind" {{if not $admin}}disabled{{end}}><option value="openai" {{if eq $p.Kind "openai"}}selected{{end}}>OpenAI-compatible</option><option value="anthropic" {{if eq $p.Kind "anthropic"}}selected{{end}}>Anthropic</option><option value="ollama" {{if eq $p.Kind "ollama"}}selected{{end}}>Ollama</option></select></label>
|
||||
<label>Kind <select name="kind" {{if not $admin}}disabled{{end}}><option value="openai" {{if eq $p.Kind "openai"}}selected{{end}}>OpenAI-compatible</option><option value="anthropic" {{if eq $p.Kind "anthropic"}}selected{{end}}>Anthropic</option><option value="ollama" {{if eq $p.Kind "ollama"}}selected{{end}}>Ollama</option><option value="bedrock" {{if eq $p.Kind "bedrock"}}selected{{end}}>AWS Bedrock</option></select></label>
|
||||
<label>Endpoint <input name="endpoint" value="{{$p.Endpoint}}" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>API key <span class="muted small">currently: {{$p.KeyDescribed}}</span> <input type="password" name="api_key" autocomplete="off" placeholder="leave blank to keep" {{if not $admin}}disabled{{end}}></label>
|
||||
<label>Timeout (ms) <input type="number" name="timeout_ms" value="{{$p.Timeout.Milliseconds}}" min="1000" {{if not $admin}}disabled{{end}}></label>
|
||||
|
||||
Reference in New Issue
Block a user