relata/docs/SEARCH_AND_SCORING.md

Search and scoring

Relata computes three different numbers and keeps them apart everywhere — in storage, in the API, and on screen. Conflating them would make every explanation meaningless.

Number	Question it answers	Range	Where it lives
Semantic similarity	How alike are these two pieces of text?	~0–1 cosine	VectorMatch.Score, Result.semantic_similarity
Relationship strength	How well evidenced is a link between two items?	0–1	Relationship.score + score_version
Query relevance	How well does this item answer this query?	0–1, relative	Result.query_relevance + rank_version

None of them is a probability. All three are stated as such wherever they are displayed.

---

1. Semantic similarity

Cosine similarity between embedding vectors, computed by Qdrant. It is what it is: a measure of how alike two passages read.

The critical property: similarity is not evidence of a factual relationship. Two incident reports about the same warehouse, written by the same team in the same house style, will be highly similar and describe entirely different events. The demonstration dataset contains exactly that pair.

Relata therefore:

• reports similarity as its own field, never folded into another score;
• rebases it before it becomes a relationship signal (see below);
• uses similar_to as the relationship type when similarity is all there is.

---

2. Relationship strength

Version: relationship-score-v1 (constant relationships.ScoreVersion)

A relationship's strength is the sum of weighted contributions from named signals, capped at 1.00:

score = min(1, Σ (signal.strength × signal.weight × diminishing_factor))

Every signal is stored on the relationship, so the arithmetic is always inspectable:

{
  "score": 0.72,
  "score_version": "relationship-score-v1",
  "signals": [
    {
      "type": "semantic_similarity",
      "value": "0.89 cosine",
      "strength": 0.76,
      "weight": 0.30,
      "detail": "How alike the wording and subject matter are. Similar wording is not evidence that two items describe the same thing."
    },
    {
      "type": "shared_entity",
      "value": "Example Ltd",
      "strength": 1.00,
      "weight": 0.25,
      "detail": "Both items refer to this organisation."
    },
    {
      "type": "claim_contradiction",
      "value": "cost",
      "strength": 1.00,
      "weight": 0.22,
      "detail": "The sources give different figures for cost: 42000 and 51500. Relata does not decide which is correct."
    }
  ]
}

Signal weights

Signal	Weight	What it means
content_hash_match	0.45	Byte-identical content
semantic_similarity	0.30	Rebased cosine similarity
shared_entity	0.25	An exact entity match, discounted by corpus frequency
claim_support	0.22	Two claims agree about the same subject
claim_contradiction	0.22	Two claims disagree about the same subject
model_classification	0.15	A language model's opinion — one signal, never the score
shared_concept	0.12	Overlapping concept vocabulary
alias_entity	0.10	A match through an alias a person confirmed. Unconfirmed candidates contribute nothing.
temporal_proximity	0.10	Created close together in time
shared_metadata	0.08	Matching user-supplied metadata
user_confirmation	1.00	A person confirmed it
user_rejection	−1.00	A person rejected it

Why these weights

The weights encode three deliberate judgements.

Similarity alone cannot look like evidence. Cosine similarity between unrelated documents from one corpus routinely sits at 0.5–0.7, so raw cosine carries almost no information at the low end. The similarity signal rebases against a floor of 0.55:

strength = max(0, (cosine − 0.55) / (1 − 0.55))

A perfect 0.99 similarity therefore yields 0.98 × 0.30 ≈ 0.29 — below the default storage threshold of 0.25 only marginally, and far below anything that reads as strong. A pair of documents that are merely alike will be recorded as similar_to at a weak score, which is the honest description.

A model cannot manufacture a relationship. The model signal's weight is 0.15, so even a maximally confident classification contributes 0.15 — below the 0.25 storage threshold. A model opinion can shape a relationship that other evidence already supports; it cannot create one. There is a test asserting this.

Human decisions dominate. Confirmation adds 1.00, saturating the score. Rejection subtracts 1.00, driving it to zero. Neither deletes anything: the original proposal, its provenance and its signals are all retained.

Diminishing returns

Repeated signals of one type contribute with a factor of 1.0, 0.5, 0.25, then nothing:

shared_entity × 6  →  0.25 + 0.125 + 0.0625  =  0.44

Breadth of evidence beats repetition. A document mentioning one company twelve times should not outrank one that relates on several independent grounds.

Entity frequency discount

An entity appearing in most documents says little about any particular pair:

strength ×= 1 / (1 + ln(corpus_frequency))

Concepts and subjects start at 0.5 rather than 1.0, because they overlap by nature.

Relationship type inference

Deterministic evidence outranks a model opinion:

1. content_hash_match → derived_from
2. claim_contradiction → contradicts
3. claim_support → supports
4. the model's type, except same_entity_as, supersedes and derived_from, which are never established on a model's word alone
5. shared entity plus differing dates → occurred_before / occurred_after
6. shared entity → mentions
7. otherwise → similar_to

Chronology yields only a chronological statement. Relata never infers possibly_explains, updates or supersedes from ordering alone — that would be inferring causation from sequence, which is precisely the mistake the design forbids.

Thresholds

Setting	Default	Effect
RELATA_REL_MIN_SCORE	0.18	Below this, a candidate is discarded before any model call
RELATA_REL_STORE_SCORE	0.25	At or above this, stored as pending review
RELATA_NEAR_DUPLICATE_SIMILARITY	0.94	Flagged as a likely near-duplicate — never merged

Nothing is ever auto-confirmed. Every discovered relationship arrives as a proposal.

---

3. Query relevance

Version: query-rank-v1 (constant ranking.RankVersion)

Retrieval paths

A query runs through several independent paths, each producing its own ordering:

Path	Weight	Source
Lexical (items)	1.00	MongoDB $text index over title and extracted text
Lexical (chunks)	1.00	MongoDB $text index over chunk text
Vector	1.00	Qdrant cosine search, workspace-filtered
Entity	0.60	Query n-grams matched against normalised entity keys
Title	0.50	Substring match on the item title
Relationship expansion	0.35	Items linked to a strong match
Confirmed relationship	0.25	Bonus when a person confirmed the link
Recency	0.15	Exponential decay, 180-day half-life

Fusion by rank, not by score

MongoDB text scores and cosine similarities are not on comparable scales — adding them directly would let whichever happens to have the larger range dominate. Relata fuses by reciprocal rank:

contribution = weight × 1 / (60 + rank)

Rank position is comparable by construction. Paths without a meaningful internal ordering (entity match, title match, expansion) contribute a scaled value pitched at roughly a mid-ranked hit, so they augment rather than swamp the ranked paths.

The final score is normalised against the top result, so the best result in any response is 1.00. This makes relevance comparable within one result set, not across queries.

Relationship expansion

After direct matching, Relata pulls in items connected to the strongest matches:

• only from the top RELATA_SEARCH_MAX_EXPANDED_ITEMS seeds;
• only relationships at or above RELATA_SEARCH_EXPANSION_MIN_SCORE (default 0.45);
• never through a relationship a person has rejected;
• optionally only through confirmed relationships (confirmed_only=true).

An expanded result is labelled in the interface and in the API:

Not a direct match. Reached because this item updates a matching item — strength 0.68 (suggested by analysis, not yet reviewed).

Why each result matched

Every result carries a why_matched list and the raw contributions:

{
  "query_relevance": 0.83,
  "semantic_similarity": 0.89,
  "why_matched": [
    "A section of this item is close in meaning to your query (similarity 0.89).",
    "The item's text contains words from your query.",
    "This item refers to Example Ltd (organisation), which your query names.",
    "Created 12 days ago."
  ]
}

---

Answer generation

"Answer from stored knowledge" retrieves, then generates strictly from what was retrieved.

Guarantees that hold regardless of what the model returns:

1. The model sees only retrieved passages, inside delimited untrusted blocks.
2. Passages are labelled S1, S2… — item IDs are never shown to the model, so a fabricated citation cannot resolve to a real record. Citations naming an unsupplied reference are dropped and counted as invalid output.
3. An answer claiming sufficiency while citing nothing is downgraded to insufficient.
4. Contradictions Relata has recorded between the retrieved items are surfaced independently of the model's own account, so a model that quietly picked a side cannot hide the disagreement.
5. Inferred relationships that influenced retrieval are listed as proposals, not facts.
6. If a passage tried to instruct the model, that is reported rather than hidden.
7. A model that returns nothing usable does not cost you the retrieval. Output rejected as invalid — including an answer that echoes the schema back ({"answer": "sufficient"}) — produces no prose, but the sources and any recorded contradictions between them are still returned, and the rejection reason is stated. Retrieval and contradiction detection are deterministic; neither depends on the model behaving. A transport failure (timeout, unreachable provider) is different and is reported as an error, because that is an operational condition rather than a bad answer.

Every answer has a "Sources used" section, and an "Uncertainty and conflicts" section whenever conflicts, insufficiency, inferred-relationship reliance or an injection attempt applies.

---

Changing the scoring model

Weights live in relationships.Weights; retrieval weights in ranking.Weights.

If you change either:

1. Bump the version constant. relationship-score-v1 → -v2. Every stored relationship records the version that produced its score, so mixed generations remain distinguishable.
2. Existing relationships are not rescored automatically. Reprocess the affected items to recompute them; the dedupe key means this updates rather than duplicates.
3. Update the weights table above and the settings page, which reads the live map.
4. Re-run internal/relationships/scoring_test.go. Several tests encode the design invariants — similarity alone staying weak, a model opinion never reaching the storage threshold, human decisions dominating — and will fail if a change breaks them.