Skip to content

hybrid_search

Hybrid retrieval over the KnowledgeStore: metadata filter + BM25 + vector cosine.

A single search entrypoint that the agentic research loop calls as a tool. Structured WHERE-clause filters (person, time range, sources) narrow the candidate set, then BM25 (FTS5) and dense cosine similarity score the survivors. The two ranks are fused with Reciprocal Rank Fusion, which is robust to the very different score scales the two signals produce (BM25 ~ [0, 20], cosine ~ [0.4, 0.9] for nomic-embed-text).

Each result is enriched with its thread context: when a hit belongs to a thread_id, the surrounding chunks are attached so the synthesis model sees the conversation, not an isolated fragment.

Brute-force vector scan is fine at the current corpus size (~5k chunks × 768 dims fits in ~15 MB and matmuls in <50 ms). Swap in an ANN index when that stops being true.

HybridSearch

Hybrid BM25 + dense-cosine retrieval over a KnowledgeStore.

Parameters

store: The store to query. embedder: Embedding client used to encode the query. When None, search falls back to BM25 only and reports vector_score=0. bm25_weight, vector_weight: Weights on the two RRF terms. Defaults to 0.5 / 0.5; raise either to bias retrieval toward lexical or semantic matches. rrf_k: RRF damping constant. Larger values flatten the contribution of deeper ranks; 60 is the canonical value from the original paper. recall_k: How deep each individual ranker recalls before fusion. Should be at least a few times limit so the fuser has overlap to work with.

search(query, *, person=None, time_range=None, sources=None, limit=20)

Run the hybrid pipeline and return up to limit hits.

See module docstring for ranking semantics. query may be empty when callers want a pure metadata filter (e.g. "all mail from X in May") — in that case only the vector leg runs (and only if an embedder is configured); if neither leg yields anything the structured filter is applied directly. Upcoming calendar timelines are returned nearest-first; other fallbacks return the most recent rows.

SearchHit dataclass

A single hybrid-search result with enough context for citation.