Skip to content

security_scanner

diapason.evals.datasets.security_scanner

security_scanner dataset — 30 vulnerable code projects for security auditing.

Each task presents a small Python project with planted vulnerabilities. The agent must identify vulnerabilities, classify severity, and recommend fixes.

Difficulty tiers: - easy (10): single obvious vulnerability (hardcoded secret, SQL injection) - medium (10): multiple vulnerabilities requiring deeper analysis - hard (10): subtle vulnerabilities mixed with safe patterns (false positive traps)

SecurityScannerDataset

Bases: DatasetProvider

30 vulnerable code projects for security auditing evaluation.