Skip to content

store

diapason.connectors.store

KnowledgeStore — source-aware SQLite/FTS5 memory backend for Deep Research.

Extends MemoryBackend with per-document provenance columns so that the IngestionPipeline and the knowledge_search tool can filter results by source, doc_type, author, and timestamp ranges.

Pure Python sqlite3 (no Rust extension required).

KnowledgeStore

Bases: MemoryBackend

Source-aware SQLite/FTS5 knowledge store for Deep Research.

Stores document chunks with rich provenance metadata and supports filtered BM25 retrieval by source, doc_type, author, and timestamp.

clear()

Remove all stored chunks.

close()

Close the underlying SQLite connection.

count()

Return the total number of stored chunks.

delete(doc_id)

Delete all chunks with the given doc_id. Returns True if any existed.

distinct_sources()

Return the sorted list of distinct source values currently indexed.

Used by the research agent to populate the system prompt with the sources the user actually has connected — so the model doesn't mention "Notion" or "Apple Notes" when nothing from those sources is in the corpus.

get_document(doc_id)

Le document ENTIER, recousu depuis ses chunks — None s'il n'existe pas.

Le dernier kilomètre (Atlas, 25 août 2026) : l'index n'est pas un index d'extraits — le corps complet y est, découpé en chunks de 2 048 caractères. C'est la reconstruction « doc_id-based » que le commentaire de SearchHit.url prévoyait sans que personne ne l'écrive.

Le chunker prépend ~100 jetons de recouvrement à chaque chunk après le premier : la couture retire ce doublon quand elle le retrouve en tête du chunk suivant, et le laisse sinon — un doublon vaut mieux qu'un trou.

retrieve(query, *, top_k=5, source=None, doc_type=None, author=None, since=None, until=None, **kwargs)

Search using FTS5 BM25 with optional column filters.

Parameters

query: Full-text search query. top_k: Maximum number of results. source: Restrict to chunks from this source (e.g. "gmail"). doc_type: Restrict to chunks of this type (e.g. "email"). author: Restrict to chunks authored by this person. since: Exclude chunks whose timestamp is earlier than this value. until: Exclude chunks whose timestamp is later than this value.

store(content, *, source='', doc_type='', doc_id=None, title='', author='', participants=None, timestamp=None, thread_id=None, url=None, metadata=None, chunk_index=0, source_id='', participants_raw=None, channel=None, content_hash='', embedding=None, embedding_model_version='', last_synced=None)

Persist a content chunk and return its unique chunk id.

All source-level fields are merged into the stored metadata so that retrieve() results carry full provenance.