Skip to content

student_runner

diapason.learning.spec_search.student_runner

Real student runner for spec-search experiments.

Replaces the MagicMock() in the experiment runner script with a callable that actually invokes the student model via vLLM (or any OpenAI-compatible engine) and returns structured results.

StudentResult dataclass

Result from running the student model on a task.

VLLMStudentRunner

Invoke the student model via a vLLM OpenAI-compatible endpoint.

Parameters

host : vLLM server URL (e.g. http://localhost:8001). model : Model name as registered in vLLM (e.g. Qwen/Qwen3.5-9B). temperature : Sampling temperature. max_tokens : Max tokens for the student response.

__call__(query, session_id='', **kwargs)

Run the student on query and return a StudentResult.

build_benchmark_samples_from_traces(trace_store, *, limit=50, min_feedback=None)

Build PersonalBenchmarkSample objects from the trace store.

Pulls recent traces (optionally filtered by feedback score) and converts them into benchmark samples the teacher can reference.