Does negative demonstrations data qualify? A four-gate, four-depth scorecard for reversed transactions

Short answer

Score the company in two parts. Four gates must all pass: 50 or more full-time employees at peak, documented operating history, rights to license, and someone who can authorize a licensing discussion (plus a location the program covers, with at least 70% English records outside the U.S., Canada and the EU). Then four depth rows earn 0 to 3 points each: reversal reasons, a stored intended state, retention of logs and tickets, and a recorded safer alternative. Ten or more of 12 with every gate passed is a strong fit.

Visual briefFull-size diagram
A semicircle dial with weak, borderline and strong bands on the left; on the right, four gates pass and four depth rows score 3, 3, 2 and 2, and the needle settles at 10 of 12, a strong fit, for a fictional 340-person building-products wholesaler.
Illustrative score for a fictional company. A strong score sorts an introduction; it does not predict acceptance or a payout.

This scorecard is part of the guide to negative demonstrations in an ordinary workflow, which annotates one integration retry that sent a supplier two purchase orders for the same approved requisition. That guide explains what makes a failed action useful as data. This page turns it into a rating you can fill in during one conversation with a client, with nothing shared but spoken answers.

How the scorecard is built: gates first, depth second

Negative demonstrations are records of actions that went wrong, kept with the reason they went wrong. A company can be large and well run and still have none worth reviewing, because most businesses fix mistakes without writing down why. So the scorecard separates two different questions.

Gates decide whether there is a company to introduce at all. They are pass or fail, and one failure stops the introduction no matter how good the records sound. Depth rows decide whether the reversal records carry enough explanation for someone to label the failing step. They are scored 0 to 3 so that partial answers count for something.

Why reasons and safer alternatives carry so much weight: published guidance already says which retries are wrong. The HTTP standard, RFC 9110, says a client should not automatically retry a request that is not idempotent unless it knows the retry is safe. Stripe's API documentation says a server error leaves the outcome unknown, so the object should be checked before retrying. A company whose tickets record that kind of reason lets a reviewer tie each failure to a stated rule instead of guessing.

The negative demonstrations scorecard

RowTypeFull marksPartialZero or fail
HeadcountGate50+ full-time employees at peak, contractors excludedNot applicableUnder 50, or mostly contractors
Operating historyGateSeveral years of purchasing records in a known ERPNot applicableA recent start or rebuilt books
RightsGateThe company owns its ERP, integration and ticket recordsNot applicableA parent, outsourcer or vendor controls them
AuthorityGateA named CEO can approve a discussionNot applicableNobody can say who decides
Reversal reasonsDepth, 0-33: reason code plus a linked ticket on most reversals1-2: codes on some, free text on others0: reversals carry no explanation
Intended state stored apartDepth, 0-33: approved requisition or order kept in a separate system1-2: only partly recoverable0: only the corrected record survives
RetentionDepth, 0-33: logs, tickets and canceled records kept for years with a known start date1-2: tickets kept, logs roll over0: everything older than a few weeks is gone
Safer alternative recordedDepth, 0-33: change tickets show what was fixed and when1-2: fixes known informally0: nobody recorded a fix

Location is a fifth gate, and outside the U.S., Canada and the EU it includes English share. The rule in full: Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

Reading the total. With every gate passed, 10 to 12 depth points is strong, 6 to 9 is borderline (introduce with the low rows named), and 5 or fewer is weak for now. Any failed gate means no introduction yet, whatever the depth total.

Five questions that fill in the depth rows

The depth rows are the hard part to rate, and five questions settle them. Ask the head of procurement, finance operations manager or integration owner; each question can be answered from memory.

#QuestionAnswer that scores highAnswer that scores zero
1Which wrong actions get reversed often enough to have a reason code?"Duplicate orders and reversed payments, each with a cancellation reason""Mistakes happen, but we just fix them"
2Is the intended state stored apart from the record that went wrong?"The approved requisition sits in the procurement tool, separate from the ERP order""Only the corrected record survives"
3Does anyone write down why the action was wrong?"A ticket is linked to every reversal""Reversals have no explanation"
4How long are integration logs, tickets and canceled records kept?"Years, with a known start date""Logs roll over after weeks and nothing else is kept"
5Who owns these systems, and who can authorize a licensing discussion?"The CFO owns them, the CEO can approve, and the integration vendor contract gives us the logs""Unclear, or an outside vendor holds them"

Questions 1 to 3 map to the reasons and intended-state rows, question 4 to retention, and question 5 to the rights and authority gates. Two zero answers among questions 1 to 3 usually mean the company has operational data but not negative demonstrations in the sense a buyer would test. Write each answer in the person's own words; it is more useful to SourceX than your score.

Strong, borderline and weak fits (Illustrative)

All three companies are fictional, and the scores show the method, not a forecast.

Strong: a 340-person building-products wholesaler. Canceled orders carry one of six reason codes, and the AP team links a ticket to each reversed payment. Approved requisitions live in a procurement tool apart from the ERP. Integration logs go back four years; the change log records when a duplicate-check was added. The CFO owns the systems. All gates pass; depth scores 3, 3, 2 and 2 for 10 of 12.

Borderline: a 140-person online home-goods retailer. Order cancellations have reasons, but refunds are explained only in a shared inbox. The intended state survives in the storefront platform. Integration logs roll over after 30 days, though tickets go back three years. Gates pass; depth scores 2, 2, 1 and 1 for 6 of 12. Worth introducing, with the log retention stated plainly.

Weak: a 75-person staffing agency. Payroll corrections are made by an outsourced provider that holds the records, and nobody can say who would approve a licensing discussion. The rights and authority gates fail, so the depth rows do not matter yet.

Answers that sound strong but score low

Some replies sound like full marks and turn out to be partial on a second question.

  • "Our ERP has a full audit trail." An audit trail shows that a field changed, not why. Ask whether a person writes the reason anywhere.
  • "We have thousands of error logs." Errors without a linked correction and reason show that something broke, not which choice broke it. The guide to failure data for AI training explains the difference.
  • "Duplicates are rare here." Good for the business, but the rows are about whether each case is explained, not how few there are. A handful of well-documented reversals can still score well.
  • "IT keeps everything." Ask for the start date. Integration logs are often the first records a company deletes, as the page on integration error records notes.

Owners, authority and what the score cannot tell you

Procurement or finance operations usually owns the reversal records, IT or an integration team owns the logs, and the CEO authorizes a licensing discussion. The company needs 50 or more full-time employees at peak (contractors excluded), documented operating history, rights to license, and an authorized contact.

The score cannot show how many reversals pass a reviewer's acceptance check, how supplier and employee details would be de-identified, whether integration vendor contracts limit use, or whether any buyer wants this failure type. Vendors such as Appen list failure-mode annotation among their agent data services, which shows the category exists; it does not show that any given company's reversal archive is wanted. Discovery stays at metadata level, so partners never request exports, screenshots or sample orders.

Before introducing, read how to document client consent for a referral and point the owner to questions to ask before licensing company data. Low depth rows can be described using negative demonstrations evidence gaps, and after an introduction the company prepares the negative demonstrations review packet. For the wider picture of how introductions work, see the introducer guide to operational data licensing.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If every gate passes and the depth rows reach at least 6 of 12, register as a partner, confirm the basics on who qualifies, and read the terms before you introduce the CFO or COO.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Who should answer the depth questions?

Whoever runs purchasing operations or the ERP integration day to day: a head of procurement, a finance operations manager or an integration owner. The CFO or COO can usually settle the headcount and rights gates; the authority gate needs the CEO, who in an owner-run firm is usually the owner.

What if the company scores well on depth but fails a gate?

Wait. A failed gate on size, rights, authority or English share stops the introduction however rich the reversal records are. Note the depth score so the conversation can restart if the gate changes.

Can I ask for one reversed order to check the reason codes myself?

No. Partners stay at metadata level: which systems, how many reversals a month, how long records are kept, who owns them. SourceX looks at actual records only under its own agreement with the company.

Sources

  1. rfc-editor.org/rfc/rfc9110.html
  2. docs.stripe.com/api/idempotent_requests
  3. appen.com/agentic-ai
  4. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment