The packet belongs to the guide to benchmark freshness in an ordinary workflow, which explains why a recent private fix with a reproducible test counts for more than an old public one. This page is the practical counterpart: the documents, their order, and who in the engineering organization fills in each one.
Contents list, in assembly order
The order follows the questions a reviewer asks: where the code lives, whether it stayed private, what one task looks like, whether it can be checked, how many exist, and who may license them.
| # | Document | What it shows | Prepared by |
|---|---|---|---|
| 1 | Repository inventory | Which repositories, which code host, how far back history runs, any migrations | Head of engineering |
| 2 | Exposure statement | Whether any code was published, mirrored, deposited in escrow or shared with outsiders | CTO with legal counsel |
| 3 | Task sheet | One representative change described field by field | A senior engineer on the team that made it |
| 4 | Test and environment note | How tests run and how an old build is rebuilt | Build or platform engineering lead |
| 5 | Acceptance check | The exact pass condition for a solution | Engineering lead, reviewed by a second engineer |
| 6 | Volume and span sheet | How many similar changes per month, over how many months | Engineering manager |
| 7 | Rights and fixture note | Authorship, contractor terms, customer data in test fixtures | Legal counsel or the CFO |
Each document is short. The whole packet is usually a few pages.
What each document must answer
1. Repository inventory. Name the repositories in scope, the code host, the earliest commit date, and any migration that rewrote or squashed history. A migration date matters because merge dates before it may be load dates, not real dates.
2. Exposure statement. Freshness depends on the code never having left the company and its bound contractors. List the exceptions: modules open-sourced, public mirrors, code shared in a vendor pilot, escrow deposits, or contributions to a shared test pool. "None known" is a valid answer if it is true; "probably none" should be written as such. For one common route, see data shared with an industry benchmark consortium.
3. Task sheet. One change, described in words, as in the filled example below. It shows a reviewer the shape of the class.
4. Test and environment note. Explain how a change from a year ago would be rebuilt: lock files, container definitions, build logs kept from the time. SWE-bench-Live ships a dedicated container image per task so the check runs the same way every time; the note says how close the company's own setup comes to that.
5. Acceptance check. State the pass condition exactly. SWE-bench's version is the model to follow: a task counts as solved only when the tests that failed before the fix now pass and the tests that passed before still pass.
6. Volume and span sheet. Rough monthly counts of tested fixes, months of history, and whether new changes keep arriving. Freshness wears off, so a continuing flow is worth recording; the why data freshness matters page explains why.
7. Rights and fixture note. Who wrote the code (employees, contractors under assignment, an outsourcer), any customer contract that restricts use, and whether test fixtures hold real customer data.
A filled task sheet for one change (Illustrative)
The company and change are fictional: a 450-person freight software company, describing a fix to its private rating engine.
| Field | What it records | Entry |
|---|---|---|
| Task statement | The problem as reported | Weekend delivery fee charged twice on multi-stop loads |
| Change date | Merge date from the code host | September 2026 |
| Exposure | Ever published or shared outside the company | Never; private repository, employees only |
| Fail-to-pass test | New test that fails before and passes after | Multi-stop weekend fee test |
| Pass-to-pass tests | Existing tests that must keep passing | Rating engine regression suite |
| Environment | How to rebuild the code as it was | Lock file and container definition committed with the change |
| Acceptance check | The exact pass condition | All listed tests pass on the fixed code, and the new test fails on the old code |
| Volume and span | How many similar changes, over what period | About 40 tested fixes a month; 30 months since the last repository migration |
| Rights | Ownership and limits | Employee authorship; one contractor team, assignment terms on file |
The September 2026 date is what makes the change useful for testing models trained before then, the same logic LiveCodeBench applies by dating every problem. The entries describe the change; none of them reproduces it.
The fixture problem
The last row of most task sheets hides the most common open question. Test fixtures in business software often contain real customer records: names, addresses, contract rates, shipment details. Developers copy them in because realistic data catches real bugs.
The rights and fixture note should say, at metadata level, whether fixtures contain customer data, roughly how much, and in which repositories. Whether they can be removed or replaced with synthetic values without breaking the tests is decided in SourceX's review. For test material more generally, see test case data for AI training.
Who signs off, and what the packet leaves open
Engineering prepares most of the packet; legal reviews the exposure statement and the rights note; the CEO or CTO signs off as the person who can authorize a licensing discussion. Introducers do not assemble or hold the packet. Their role ends at the metadata conversation, and they never request code, repository access or exports.
The company must meet the basics: 50 or more full-time employees at peak (contractors excluded), documented operating history, rights to license, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
The packet leaves open how many changes pass the acceptance check, how fixtures will be handled, whether contracts restrict use, and whether any buyer wants this kind of task. Some buyers want fresh material only for testing, which suits an evaluation-only license. To check fit before any of this, use the benchmark freshness qualification scorecard; for weak spots, see benchmark freshness evidence gaps.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the head of engineering could fill in a task sheet like the one above for a steady flow of recent changes, register as a partner, check the company against who qualifies, and read the terms before you introduce the CEO or CTO.