Preregister tasks
Freeze the commit, acceptance criteria, allowed files, checks, security constraints, and timeout before testing.
Download schema →OPEN DATA SCHEMA · VERSION 1.0 · 2026-07-19
Publish AI coding evidence others can audit.
This open kit connects tasks selected before testing, every run observed during testing, and a bounded case summary. The files are blank by design: we do not present synthetic results as original research.
Reusable research assets
From preregistration to a publishable case.
CC0-licensed schemas make independent replication and comparison easier. Blank means unavailable; zero means measured zero. Cite the repository's versioned citation metadata when publishing results.
Record every attempt
Keep accepted and rejected runs with versions, time, review effort, usage, cost, retries, and failure category.
Download schema →Publish a bounded case
Report the evaluation window, sample size, baseline, decision, limitations, evidence, and publication consent.
Download schema →Minimum evidence standard
A case is evidence only when its boundaries remain visible.
Publish exact versions and sample counts, retain failures, link supporting artifacts, disclose conflicts, and obtain consent before naming an organization.
Every attempt remains in the run file, including setup, model, test, review, and policy failures.
An anonymized pilot can describe measured conditions, but must not imply customer approval.
Results support decisions within the tested repository, model, environment, and time window.
CONTRIBUTE EVIDENCE
Run the protocol and publish a versioned dataset.
Your first task: fix, test, explain
The results below are a locally authored and tested Codex reference exercise, not a MonkeyCode product run or a customer benchmark. No timing, token-cost or agent-success rate is claimed.
Reproduce the reference