Skip to content

backend

backend

Abstract base class for inference backends.

Classes

InferenceBackend

Bases: ABC

Base class for all inference backends used in evaluation.

Attributes
framework_commit_value property 
framework_commit_value: str

Default: empty (subclasses override). Used by runner.py error paths.

Methods:
generate abstractmethod 
generate(
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> str

Generate a response and return just the text content.

Source code in src/diapason/evals/core/backend.py 
24
25
26
27
28
29
30
31
32
33
34
@abstractmethod
def generate(
    self,
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> str:
    """Generate a response and return just the text content."""
generate_full abstractmethod 
generate_full(
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> Dict[str, Any]

Generate a response and return full details.

Returns dict with keys: content, usage, model, latency_seconds, cost_usd.

Source code in src/diapason/evals/core/backend.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
@abstractmethod
def generate_full(
    self,
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> Dict[str, Any]:
    """Generate a response and return full details.

    Returns dict with keys: content, usage, model, latency_seconds, cost_usd.
    """
close 
close() -> None

Release resources.

Source code in src/diapason/evals/core/backend.py 
51
52
def close(self) -> None:
    """Release resources."""