retriever
diapason.connectors.retriever
¶
TwoStageRetriever — BM25 recall from KnowledgeStore + optional reranking.
Composes a fast BM25 first-stage recall (via KnowledgeStore) with an
optional second-stage Reranker for semantic reordering. The default
ColBERTReranker lazy-loads a ColBERT checkpoint and scores candidates via
MaxSim; it degrades gracefully when colbert-ai is not installed.
Typical usage::
store = KnowledgeStore()
retriever = TwoStageRetriever(store, reranker=ColBERTReranker())
results = retriever.retrieve("neural networks", top_k=10)
ColBERTReranker
¶
Bases: Reranker
Semantic reranker backed by ColBERT MaxSim scoring.
Lazy-loads a ColBERT checkpoint on first use. If the colbert-ai
package is not installed the reranker falls back to returning the
BM25-ordered candidates unchanged (with a warning logged once).
Parameters¶
checkpoint:
Path or HuggingFace model ID for the ColBERT checkpoint.
Defaults to "colbert-ir/colbertv2.0".
embedding_store:
Optional :class:EmbeddingStore for caching per-chunk ColBERT
token embeddings on disk. When provided, the reranker looks up
pre-computed embeddings before falling back to docFromText(),
and stores newly computed embeddings for future reuse.
rerank(query, candidates, *, top_k=10)
¶
Rerank candidates using ColBERT MaxSim scores.
Falls back to BM25 order if colbert-ai is unavailable.
Reranker
¶
Bases: ABC
Abstract base class for second-stage semantic rerankers.
rerank(query, candidates, *, top_k=10)
abstractmethod
¶
TwoStageRetriever
¶
BM25 recall + optional semantic reranking for Deep Research.
Stage 1 retrieves max(recall_k, top_k * 3) candidates from the
KnowledgeStore using FTS5/BM25. Stage 2 optionally passes those
candidates through a Reranker to produce a semantically ordered
final list of top_k results.
Parameters¶
store:
The KnowledgeStore to query in Stage 1.
reranker:
An optional Reranker for Stage 2. When None the retriever
returns BM25-ordered results directly.
recall_k:
Number of candidates to fetch in Stage 1. The actual recall
size is max(recall_k, top_k * 3) so that the reranker always
has a meaningful pool to work with.
retrieve(query, *, top_k=10, source='', doc_type='', author='', since='', until='')
¶
Run the two-stage retrieval pipeline.
Parameters¶
query:
Full-text search query.
top_k:
Maximum number of results to return.
source:
Restrict to chunks from this source (e.g. "gmail").
doc_type:
Restrict to chunks of this doc type (e.g. "email").
author:
Restrict to chunks authored by this person.
since:
Exclude chunks whose timestamp is earlier than this ISO string.
until:
Exclude chunks whose timestamp is later than this ISO string.
Returns¶
List[RetrievalResult] Up to top_k results, reranked when a reranker is configured.