Editorial correction: This page no longer repeats the July price-cut percentages, launch prices or comparative cost figures previously taken from secondary reporting. They are not current quotes.
Keep the numerator and denominator honest
For a defined evaluation window, add actual model charges, tool fees and compute charges across all attempts. Divide by the number of changes that pass your acceptance checks. If no change is accepted, report “no accepted result” rather than a zero cost per task. Track human review minutes separately, or disclose the hourly rate when converting them into money.
Record a quote before running
Capture the provider, model, currency, time, input/cache/output categories and any plan-specific terms. Record actual usage after running; an estimate is not a receipt. Refresh the quote when the provider or model alias changes.
Separate comparison from an onboarding exercise
The public first-task pack helps check your workflow. Its published reference solutions make it unsuitable as a blind evaluation. For a product decision, repeat the method on unseen tasks and retain every failure in the attempt template.