Great paper from Google on making ML experiments reproducible for coding agents.

(bookmark it)

If you run agents on research code, this one is worth your time.

They find that refined artifacts raise agent reproduction success from 76.4% to 92.6% across four models and cut API spending by 19.7%.

ReproBridge turns one successful reproduction into a package the next agent can reuse. The package holds the cleaned code, the environment setup, scripts that launch each experiment, and a written guide.

A new agent reruns the experiment from that package. A separate auditor then checks that the runs actually happened.

The benchmark covers 1,029 tasks from 36 papers, each with five human-verified reference scores. The gap between the best and worst model drops from 21.6 to 3.9 points. Removing the code or the environment setup causes the largest losses.

If your agents keep failing to rerun a repo, save the setup steps from the first successful run before trying a bigger model.

https://arxiv.org/abs/2611.08452

Chat with Paper: https://academy.dair.ai/papers/reprobridge-building-reproducibility-into-ml-research-artifacts-2611.08452