synthesizer
diapason.learning.optimize.personal.synthesizer
¶
Synthesize personal benchmarks from interaction traces.
PersonalBenchmark
dataclass
¶
A synthesized benchmark from user interaction traces.
PersonalBenchmarkSample
dataclass
¶
A single sample in a personal benchmark.
PersonalBenchmarkSynthesizer
¶
Mines interaction traces into a reusable personal benchmark.
synthesize(workflow_id='default', min_feedback=0.7, max_samples=100)
¶
Build a personal benchmark from high-quality traces.
- Query traces that have feedback >= min_feedback.
- Group by query class (agent + first 50 chars of query).
- For each class, pick the trace with the highest feedback as reference.
- Return a :class:
PersonalBenchmarkcapped at max_samples.