Skip to content

Index

diapason.engine

Inference Engine primitive — LLM runtime management.

EngineConnectionError

Bases: Exception

Raised when an engine is unreachable.

EngineContextLengthError

Bases: EngineConnectionError

The prompt exceeds the served model's maximum context window.

Subclasses EngineConnectionError so existing except EngineConnectionError handlers keep catching it, while callers that want a distinct, user-facing "conversation too long" message can branch on this type (or the is_context_length_error marker) instead of surfacing a generic engine failure.

InferenceEngine

Bases: ABC

Base class for all inference engine backends.

Subclasses must be registered via @EngineRegistry.register("name") to become discoverable.

can_serve(model)

Return True if this engine can serve model.

Defaults to True: local engines accept any model id (whether a specific model is installed is a separate concern from engine selection). Engines that multiplex provider-specific clients (e.g. the cloud engine) override this so selection can skip an engine whose client for the model's provider isn't configured (see #532).

close()

Release resources (HTTP clients, connections, threads, etc.).

generate(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs) abstractmethod

Synchronous completion — returns a dict with content and usage.

health() abstractmethod

Return True when the engine is reachable and healthy.

list_models() abstractmethod

Return identifiers of models available on this engine.

prepare(model)

Optional warm-up hook called before the first request.

stream(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs) abstractmethod async

Yield token strings as they are generated.

stream_full(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs) async

Yield full StreamChunks including tool_calls and finish_reason.

Default implementation wraps stream() for backward compatibility. Engines with native tool-call streaming should override this.

discover_engines(config)

Probe registered engines and return [(key, instance)] for healthy ones.

Results are sorted with the config default engine first.

discover_models(engines)

Call list_models() on each engine and return a dict.

get_engine(config, engine_key=None, model=None)

Get a specific engine by key, or the default with fallback.

When model is given, an engine is selected only if it can actually serve that model (engine.can_serve(model)). This stops the cloud fallback from being chosen — when the local engine is down — for a model whose provider client is missing, which otherwise surfaces as a confusing "OpenAI client not available" instead of a helpful "start your local engine" message (see #532). When model is None selection stays model-agnostic (unchanged behaviour).

Returns (key, engine_instance) or None if no engine is available.

looks_like_context_length_error(text)

True when text reads like a context-window overflow error.

The single shared heuristic for recognizing vendor context-overflow phrasings — used by the engine layer (typing upstream 400s), agent error classification, and the server stream bridge, so a new vendor phrasing only ever needs to be added here.

messages_to_dicts(messages)

Convert Message objects to OpenAI-format dicts.