student_runner
diapason.learning.spec_search.student_runner
¶
Real student runner for spec-search experiments.
Replaces the MagicMock() in the experiment runner script with a
callable that actually invokes the student model via vLLM (or any
OpenAI-compatible engine) and returns structured results.
StudentResult
dataclass
¶
Result from running the student model on a task.
VLLMStudentRunner
¶
Invoke the student model via a vLLM OpenAI-compatible endpoint.
Parameters¶
host :
vLLM server URL (e.g. http://localhost:8001).
model :
Model name as registered in vLLM (e.g. Qwen/Qwen3.5-9B).
temperature :
Sampling temperature.
max_tokens :
Max tokens for the student response.
__call__(query, session_id='', **kwargs)
¶
Run the student on query and return a StudentResult.
build_benchmark_samples_from_traces(trace_store, *, limit=50, min_feedback=None)
¶
Build PersonalBenchmarkSample objects from the trace store.
Pulls recent traces (optionally filtered by feedback score) and converts them into benchmark samples the teacher can reference.