tb_agent

tb_agent

Custom terminal-bench agent that wraps NaiveAgent with LiteLLM construction.

Classes

DiapasonTerminalBenchAgent

DiapasonTerminalBenchAgent(
    model_name: str = "openai/default",
    api_base: str = "http://localhost:8000/v1",
    temperature: float = 0.2,
    **kwargs,
)

Bases: NaiveAgent

NaiveAgent that constructs its own LiteLLM from string kwargs.

This avoids serialization issues in terminal-bench's lock file by accepting string parameters instead of an LiteLLM object.

Source code in src/diapason/evals/backends/tb_agent.py

def __init__(
    self,
    model_name: str = "openai/default",
    api_base: str = "http://localhost:8000/v1",
    temperature: float = 0.2,
    **kwargs,
):
    llm = LiteLLM(
        model_name=model_name,
        temperature=temperature,
        api_base=api_base,
    )
    super().__init__(llm=llm, **kwargs)