Skip to content

retriever

diapason.connectors.retriever

TwoStageRetriever — BM25 recall from KnowledgeStore + optional reranking.

Composes a fast BM25 first-stage recall (via KnowledgeStore) with an optional second-stage Reranker for semantic reordering. The default ColBERTReranker lazy-loads a ColBERT checkpoint and scores candidates via MaxSim; it degrades gracefully when colbert-ai is not installed.

Typical usage::

store = KnowledgeStore()
retriever = TwoStageRetriever(store, reranker=ColBERTReranker())
results = retriever.retrieve("neural networks", top_k=10)

ColBERTReranker

Bases: Reranker

Semantic reranker backed by ColBERT MaxSim scoring.

Lazy-loads a ColBERT checkpoint on first use. If the colbert-ai package is not installed the reranker falls back to returning the BM25-ordered candidates unchanged (with a warning logged once).

Parameters

checkpoint: Path or HuggingFace model ID for the ColBERT checkpoint. Defaults to "colbert-ir/colbertv2.0". embedding_store: Optional :class:EmbeddingStore for caching per-chunk ColBERT token embeddings on disk. When provided, the reranker looks up pre-computed embeddings before falling back to docFromText(), and stores newly computed embeddings for future reuse.

rerank(query, candidates, *, top_k=10)

Rerank candidates using ColBERT MaxSim scores.

Falls back to BM25 order if colbert-ai is unavailable.

Reranker

Bases: ABC

Abstract base class for second-stage semantic rerankers.

rerank(query, candidates, *, top_k=10) abstractmethod

Rerank candidates for query and return the top top_k results.

Parameters

query: The original search query. candidates: List of RetrievalResult objects from the BM25 recall stage. top_k: Maximum number of results to return.

Returns

List[RetrievalResult] Reranked results, truncated to top_k.

TwoStageRetriever

BM25 recall + optional semantic reranking for Deep Research.

Stage 1 retrieves max(recall_k, top_k * 3) candidates from the KnowledgeStore using FTS5/BM25. Stage 2 optionally passes those candidates through a Reranker to produce a semantically ordered final list of top_k results.

Parameters

store: The KnowledgeStore to query in Stage 1. reranker: An optional Reranker for Stage 2. When None the retriever returns BM25-ordered results directly. recall_k: Number of candidates to fetch in Stage 1. The actual recall size is max(recall_k, top_k * 3) so that the reranker always has a meaningful pool to work with.

retrieve(query, *, top_k=10, source='', doc_type='', author='', since='', until='')

Run the two-stage retrieval pipeline.

Parameters

query: Full-text search query. top_k: Maximum number of results to return. source: Restrict to chunks from this source (e.g. "gmail"). doc_type: Restrict to chunks of this doc type (e.g. "email"). author: Restrict to chunks authored by this person. since: Exclude chunks whose timestamp is earlier than this ISO string. until: Exclude chunks whose timestamp is later than this ISO string.

Returns

List[RetrievalResult] Up to top_k results, reranked when a reranker is configured.