Evaluation
Benchmarks for measuring AI-assisted AI improvement loops.
Research
Pensera builds evaluation, interpretability, and control systems for AI systems that improve AI. Our work centers on one question: when models begin improving models, how do we keep the loop measurable and safe?
What we study
Interpretability
Tools for understanding when models generate, select, or optimize improvements.
Control
Protocols for bounding autonomy, logging changes, and preventing unsafe escalation.
RSI-Bench
RSI-Bench is a sandboxed benchmark for evaluating AI systems on controlled model-improvement tasks, including debugging training code, improving small models, designing ablations, and detecting metric gaming.
Each task ships with a verifiable target and a held-out scorer, so an agent’s contribution is measured by the real change it produces — a model that trains, a bug that is actually fixed, an ablation that answers the question it claims to — rather than by its description of the work. Tasks are designed to reward genuine improvement and to surface reward hacking when it occurs.
In development.
RSI-Sandbox
RSI-Sandbox is the isolated environment that RSI-Bench tasks run inside. Every model-improvement run executes in a hermetic container with no outbound network access and capped compute and wall-clock budgets, so an experiment cannot reach beyond the resources it was granted.
The environment keeps a complete, append-only record of everything a system does — each file edited, command executed, and model trained — and enforces declared capability and autonomy limits itself, rather than trusting the agent to respect them. Runs are deterministic and replayable, so any result can be reproduced, inspected, and audited after the fact.
In development.