Skip to content

embeddings

diapason.tools.storage.embeddings

Embeddings abstraction for dense retrieval backends.

Embedder

Bases: ABC

Base class for text embedding models.

Subclasses must implement :meth:embed and :meth:dim.

dim() abstractmethod

Return the dimensionality of the embedding vectors.

embed(texts) abstractmethod

Embed texts and return a numpy array of shape (n, dim).

OllamaEmbedder

Bases: Embedder

Embedder backed by an Ollama server's /api/embed endpoint.

Sends batches in parallel (up to max_parallel) since Ollama serializes items within a single HTTP request but happily serves multiple concurrent connections.

Parameters

model: Ollama model tag; defaults to nomic-embed-text (768-dim). base_url: Ollama server base URL. Defaults to http://localhost:11434. batch_size: Items per HTTP request. Tuned for Ollama — larger batches help throughput but increase memory on the server. max_parallel: How many concurrent HTTP requests to issue. Ollama CPU/GPU saturation is typically hit around 8. timeout_s: Per-request timeout.

dim()

Return the embedding dimensionality (probes the server on first call).

embed(texts)

Return a numpy array of shape (len(texts), dim).

Float32, L2-normalized per row so callers can use dot-product as cosine similarity. Empty input → shape (0, dim).

SentenceTransformerEmbedder

Bases: Embedder

Embedder backed by sentence-transformers.

Parameters

model_name: HuggingFace model identifier. Defaults to the lightweight all-MiniLM-L6-v2 (384-dim, ~22 MB).

dim()

Return the embedding dimensionality.

embed(texts)

Return a numpy array of shape (len(texts), dim).