Index
diapason.engine
¶
Inference Engine primitive — LLM runtime management.
EngineConnectionError
¶
Bases: Exception
Raised when an engine is unreachable.
EngineContextLengthError
¶
Bases: EngineConnectionError
The prompt exceeds the served model's maximum context window.
Subclasses EngineConnectionError so existing except
EngineConnectionError handlers keep catching it, while callers that want a
distinct, user-facing "conversation too long" message can branch on this type
(or the is_context_length_error marker) instead of surfacing a generic
engine failure.
InferenceEngine
¶
Bases: ABC
Base class for all inference engine backends.
Subclasses must be registered via
@EngineRegistry.register("name") to become discoverable.
can_serve(model)
¶
Return True if this engine can serve model.
Defaults to True: local engines accept any model id (whether a
specific model is installed is a separate concern from engine
selection). Engines that multiplex provider-specific clients (e.g.
the cloud engine) override this so selection can skip an engine whose
client for the model's provider isn't configured (see #532).
close()
¶
Release resources (HTTP clients, connections, threads, etc.).
generate(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs)
abstractmethod
¶
Synchronous completion — returns a dict with content and usage.
health()
abstractmethod
¶
Return True when the engine is reachable and healthy.
list_models()
abstractmethod
¶
Return identifiers of models available on this engine.
prepare(model)
¶
Optional warm-up hook called before the first request.
stream(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs)
abstractmethod
async
¶
Yield token strings as they are generated.
stream_full(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs)
async
¶
Yield full StreamChunks including tool_calls and finish_reason.
Default implementation wraps stream() for backward compatibility.
Engines with native tool-call streaming should override this.
discover_engines(config)
¶
Probe registered engines and return [(key, instance)] for healthy ones.
Results are sorted with the config default engine first.
discover_models(engines)
¶
Call list_models() on each engine and return a dict.
get_engine(config, engine_key=None, model=None)
¶
Get a specific engine by key, or the default with fallback.
When model is given, an engine is selected only if it can actually
serve that model (engine.can_serve(model)). This stops the cloud
fallback from being chosen — when the local engine is down — for a model
whose provider client is missing, which otherwise surfaces as a confusing
"OpenAI client not available" instead of a helpful "start your local
engine" message (see #532). When model is None selection stays
model-agnostic (unchanged behaviour).
Returns (key, engine_instance) or None if no engine is available.
looks_like_context_length_error(text)
¶
True when text reads like a context-window overflow error.
The single shared heuristic for recognizing vendor context-overflow phrasings — used by the engine layer (typing upstream 400s), agent error classification, and the server stream bridge, so a new vendor phrasing only ever needs to be added here.
messages_to_dicts(messages)
¶
Convert Message objects to OpenAI-format dicts.