store
diapason.connectors.store
¶
KnowledgeStore — source-aware SQLite/FTS5 memory backend for Deep Research.
Extends MemoryBackend with per-document provenance columns so that the
IngestionPipeline and the knowledge_search tool can filter results by
source, doc_type, author, and timestamp ranges.
Pure Python sqlite3 (no Rust extension required).
KnowledgeStore
¶
Bases: MemoryBackend
Source-aware SQLite/FTS5 knowledge store for Deep Research.
Stores document chunks with rich provenance metadata and supports filtered BM25 retrieval by source, doc_type, author, and timestamp.
clear()
¶
Remove all stored chunks.
close()
¶
Close the underlying SQLite connection.
count()
¶
Return the total number of stored chunks.
delete(doc_id)
¶
Delete all chunks with the given doc_id. Returns True if any existed.
distinct_sources()
¶
Return the sorted list of distinct source values currently indexed.
Used by the research agent to populate the system prompt with the sources the user actually has connected — so the model doesn't mention "Notion" or "Apple Notes" when nothing from those sources is in the corpus.
get_document(doc_id)
¶
Le document ENTIER, recousu depuis ses chunks — None s'il n'existe pas.
Le dernier kilomètre (Atlas, 25 août 2026) : l'index n'est pas un index d'extraits — le corps complet y est, découpé en chunks de 2 048 caractères. C'est la reconstruction « doc_id-based » que le commentaire de SearchHit.url prévoyait sans que personne ne l'écrive.
Le chunker prépend ~100 jetons de recouvrement à chaque chunk après le premier : la couture retire ce doublon quand elle le retrouve en tête du chunk suivant, et le laisse sinon — un doublon vaut mieux qu'un trou.
retrieve(query, *, top_k=5, source=None, doc_type=None, author=None, since=None, until=None, **kwargs)
¶
Search using FTS5 BM25 with optional column filters.
Parameters¶
query: Full-text search query. top_k: Maximum number of results. source: Restrict to chunks from this source (e.g. "gmail"). doc_type: Restrict to chunks of this type (e.g. "email"). author: Restrict to chunks authored by this person. since: Exclude chunks whose timestamp is earlier than this value. until: Exclude chunks whose timestamp is later than this value.
store(content, *, source='', doc_type='', doc_id=None, title='', author='', participants=None, timestamp=None, thread_id=None, url=None, metadata=None, chunk_index=0, source_id='', participants_raw=None, channel=None, content_hash='', embedding=None, embedding_model_version='', last_synced=None)
¶
Persist a content chunk and return its unique chunk id.
All source-level fields are merged into the stored metadata so that
retrieve() results carry full provenance.