# Synthetic-to-real validation for introducers | SourceX

Canonical: https://refer.sourcex.si/guides/concept-synthetic-to-real-validation
Last updated: 2026-11-04

Synthetic-to-real validation means checking a simulated task, such as a generated customer return, against the real thing: the company's policy in force at the time, what actually happened in comparable cases, and the judgment of an experienced operator. For an introducer, the useful question is whether the company holds the versioned policies, the case history with outcomes and the people who can say where a simulation is wrong, and whether someone can authorize a licensing discussion.

## What is synthetic-to-real validation, in plain terms?

AI developers often train and test agents on generated scenarios because they are cheap and plentiful. The open question is whether an agent that handles the simulated version will handle the real one. Validation is the step that answers it: take simulated cases, compare them field by field with real policy and real records, and have someone who did the job mark every divergence.

Two research findings explain why the step matters. A 2024 paper in Nature showed that indiscriminate training on model-generated content causes defects in which the tails of the original distribution disappear, and that generated data needs careful filtering. In business terms, the tails are the rare branches: the damaged parcel, the fraud check, the holiday exception. A COLM 2024 paper on best practices for synthetic data stresses factuality, fidelity and unbiasedness. Fidelity to a specific company's rules is exactly what an outside generator cannot know. The live comparison of [synthetic data and real data for AI agents](/compare/synthetic-data-vs-real-data-ai-agents) covers when each wins; this page is about the check that connects them.

### One simulated return, checked by a returns lead (Illustrative)

A 450-person online retailer of consumer electronics is asked to review a simulated return generated for an agent test. The returns lead, with eight years on the team, compares it with the policy in force and with how comparable real cases were handled. All details are fictional.

| Field | Simulated case | Operator's verdict | What settles it in the real records |
|---|---|---|---|
| Return window | 45 days from delivery | Diverges: 30 days, extended to January 31 for holiday orders | Policy document, version in force on the order date |
| Restocking fee | 15% on every opened item | Partly matches: only on opened large appliances, waived for defects | Fee rules plus refund lines on past cases |
| Serial number check | Not performed | Diverges: required on items above a price threshold before refund | Warehouse receiving scans |
| Carrier damage | Refund issued at once | Diverges: carrier claim filed first, refund after inspection | Claim records linked to the order |
| Refund method | Original payment method | Matches | Payment processor refund records |
| Disposition | Restock | Diverges: graded A, B or C for resale, refurbishment or recycling | Grading log in the warehouse system |

One field matches, one partly matches and four diverge. Three of the four divergences sit on rare or conditional branches (holiday extension, high-value serial checks, damage claims), which is what the Nature finding would lead you to expect from generated data. Without the company's records, nobody outside could have marked them.

## What should an introducer ask about synthetic-to-real validation?

Ask five questions of the operations or customer service lead and score each answer 0, 1 or 2. This is a rough sorting aid for you, not a SourceX acceptance test.

| Question | 2 points | 1 point | 0 points |
|---|---|---|---|
| Are past versions of the policy kept, with dates? | Every version, dated | Current plus some old | Current only |
| Can a case be traced from request to final outcome? | One case ID across systems | Possible with effort | Outcome not recorded |
| Are exception cases tagged (damage, fraud, warranty)? | Tagged reason codes | Free-text notes | Not distinguishable |
| Are experienced operators still with the company? | Several with five-plus years | One or two | Team turned over |
| Who could authorize a licensing discussion? | Named executive with authority | Needs parent or board input | Unknown |

A score of 7 or more out of 10 suggests the records exist to validate simulations in this workflow. Lower scores point to what the owner should check first; the guide to [questions to ask before licensing company data](/resources/questions-to-ask-before-licensing-company-data) helps with that.

## Who owns the records, and who can say yes?

Returns policies usually sit with customer service or legal; case history with e-commerce or customer service; inspection and grading with the warehouse; refunds with finance. Authority to discuss a license normally rests with the CEO, COO, CFO or owner. Customer personal data will need de-identification, which SourceX settles with the company before any delivery. Escalation-heavy workflows, like those in the [support escalation process](/data-types/support-escalation-process) overview, often sit right next to returns.

The company must have 50+ full-time employees at peak (contractors excluded), a documented operating history, rights to license, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. See [who qualifies](/who-qualifies) for detail.

What stays unknown until SourceX reviews it: SourceX's current data needs for these records, the rights review result, the de-identification required, the company's offer and any reward. More record families are listed under [data types](/data-types).

## Go deeper on synthetic-to-real validation

- [Record gaps that make synthetic-to-real validation hard to assess](/questions/synthetic-to-real-validation-evidence-gaps): a guide to the missing policy versions, outcomes and tags that weaken a validation.
- [Synthetic-to-real validation review packet](/resources/synthetic-to-real-validation-review-packet): a checklist of what belongs in an evidence packet for SourceX review.

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If a client scores 7 or more on the five questions, [register as a partner](/register), confirm the basics on [who qualifies](/who-qualifies), and read the [program terms](/terms) before introducing the executive who can authorize the discussion.

## Sources

- https://refer.sourcex.si/terms
- https://www.nature.com/articles/s41586-024-07566-y
- https://openreview.net/forum?id=OJaWBhh61C
