stage_router
stage_router
¶
Adapter for orchestration eval script.
Provides: StageSkillHandbook (load from JSON), parse_skill_analysis, get_routing_strategy. Compatible with JSON produced by to_stage_router.
Classes¶
Skill
dataclass
¶
Skill(
skill_id: str,
name: str,
description: str,
stage: str,
examples: List[str] = list(),
discovered_from_problems: List[Dict[str, str]] = list(),
)
A skill that models can have.
ModelProfile
dataclass
¶
ModelProfile(
model_alias: str,
actual_model: str,
stage: str,
skill_scores: Dict[str, float] = dict(),
skill_attempts: Dict[str, int] = dict(),
skill_successes: Dict[str, int] = dict(),
overall_success_rate: float = 0.5,
total_attempts: int = 0,
total_successes: int = 0,
avg_prompt_tokens: float = 0.0,
avg_completion_tokens: float = 0.0,
avg_cost_usd: float = 0.0,
strengths: List[str] = list(),
weaknesses: List[str] = list(),
)
Performance profile for a model alias.
StageSkillHandbook
¶
Handbook for eval routing. Load from JSON produced by to_stage_router.