Skip to content

coding_assistant

diapason.evals.scorers.coding_assistant

coding_assistant scorer — test-based evaluation of bug fixes.

Extracts fixed code from model output, runs the test suite, and computes: - fix_rate: fraction of originally-failing tests now passing - regressions: count of originally-passing tests that broke

CodingAssistantScorer

Bases: Scorer

Score coding assistant bug fixes by test execution.