Skip to content

cloud

diapason.engine.cloud

Cloud inference engine.

OpenAI, Anthropic, Google, MiniMax, and DeepSeek API backends.

CloudEngine

Bases: InferenceEngine

Cloud inference via OpenAI, Anthropic, Google, MiniMax, and DeepSeek SDKs.

can_serve(model)

Return True only if the provider client for model exists.

health() is True whenever any provider client is configured, but a request for, say, a gpt-* model still needs the OpenAI client specifically. Without this check the cloud engine gets picked as a fallback (when the local engine is down) for a model it can't serve, then dies at call time with " client not available" instead of the user getting a helpful "start your local engine" message (see #532).

stream_full(messages, *, model, temperature=0.7, max_tokens=1024, **kwargs) async

Yield StreamChunks with content, tool_calls, and finish_reason.

estimate_cost(model, prompt_tokens, completion_tokens)

Estimate USD cost based on the hardcoded pricing table.

resolve_pricing(model)

The (input, output) per-million rate for model, or None if unknown.

Exact match first, then the longest matching prefix. Longest matters: the table holds both gpt-4o and gpt-4o-mini, and a dated variant like gpt-4o-mini-2024-07-18 is a prefix match for both. Taking the first hit in insertion order billed it at the full gpt-4o rate — 16× the real price — while MiniMax-M2.7-highspeed-v2 fell back to plain MiniMax-M2.7 and was billed at half.