# SQL query logs as AI training data for text-to-SQL

Canonical: https://refer.sourcex.si/data-types/sql-query-logs-ai-training-data
Last updated: 2026-10-20

SQL query logs are scarce, valuable AI training data because they pair real business questions with real queries against production schemas. BI and data consultants who see warehouse history and glossaries at every client can introduce companies with 50+ full-time employees at peak to SourceX without copying any query.

## Why is SQL query history valuable training data for text-to-SQL?

A company's warehouse query history pairs real business questions with the SQL analysts actually wrote against real, messy schemas. That pairing is scarce because query logs sit inside companies and are rarely shared, while public text-to-SQL benchmarks tend to use cleaner schemas than a production warehouse with years of renamed columns.

For a BI or data consultant, this is a record set you see at every client: the query history table in the warehouse, the saved questions in the BI tool, and the glossary a data team keeps to define "active customer". Licensed query history with the question and metric definitions behind it is the useful version.

## What makes a query log more than a pile of SQL?

Raw SQL alone teaches syntax. Value comes from the context around it.

| Layer | What it holds | Why it helps |
|---|---|---|
| Query text | Joins, filters, window functions, CTEs | Real patterns against the real schema |
| Business question | Ticket, Slack request or dashboard title | The natural-language side of the pair |
| Metric definitions | Glossary, semantic layer, dbt docs | What "revenue" or "churn" means here |
| Corrections | Edited queries, reviewer comments | Where first attempts were wrong |
| Usage signals | Reruns, saved dashboards, scheduled reports | Which queries the business trusted |

Evaluation sets matter as much as training sets here. A set of questions with known-correct SQL lets a buyer test whether an agent can answer from a specific warehouse, which is the check an enterprise customer cares about.

## Where do the records live?

- Warehouse query history views and audit logs (Snowflake, BigQuery, Redshift, Databricks and similar)
- BI tool saved questions, dashboards and their descriptions
- dbt or semantic-layer repositories, with model and metric documentation
- Data team ticket queues and Slack channels where analysts were asked questions
- Wiki pages that act as a business glossary

Retention differs by platform and plan, and logs often roll off after a set period. Ask the client's data lead what their own retention is rather than assuming.

## What are the rights and privacy pitfalls?

Query text can embed literals: customer names, emails, account numbers pasted into a filter. Results are not the issue, because the license concerns queries and context, but literals still need scrubbing. Schemas may reveal internal names that the company treats as confidential, and some tables hold personal data governed by the company's own privacy commitments.

None of this blocks a deal. It means the company agrees redaction rules and exclusions before any work begins, and delivery happens only after an executed agreement and the company's authorization. This is general information, not legal, tax or financial advice. Confirm specifics with the company's own counsel.

## The query-log depth screen

- [ ] Query history goes back well beyond the platform default, or is archived elsewhere.
- [ ] A named person owns the glossary or semantic layer.
- [ ] Questions can be linked to queries (tickets, titles or comments).
- [ ] More than one warehouse or BI generation exists, with the older one archived.
- [ ] The client has 50+ full-time employees at peak (contractors excluded) and years of documented operations.
- [ ] Management would consider an exclusive license for AI training.

A client with a thick glossary and short logs can still qualify because the surrounding systems carry the rest. The company, not you, decides what goes in the inventory.

## When do you raise it in an engagement?

| Moment | What you see | Opening question |
|---|---|---|
| Warehouse migration | Old platform being shut down | "Who is keeping the old query history?" |
| Semantic layer project | Metric definitions being written | "Is the old glossary being archived?" |
| BI tool rationalization | Hundreds of dashboards retired | "Do the saved questions get exported first?" |
| Data team handover | Analysts leaving, documents scattered | "Where does the knowledge live now?" |

## How does the introduction work?

1. Mention the idea to the executive who owns data, usually the CFO, COO or head of data.
2. Share your referral link or submit basic fit details through the referral form.
3. SourceX qualifies size, history, data breadth and rights.
4. The company lists its systems in a data inventory; the [data inventory builder](/tools/data-inventory-builder) is one way to start. Metadata only.
5. Price and terms are agreed, SourceX review, and nothing binds until signature.
6. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

You never copy queries or schemas from a client environment. Check your own engagement agreement for confidentiality limits.

> What to say: "Your warehouse has years of real questions and the SQL that answered them. AI developers look for that because public benchmarks do not look like a production schema. I cannot see your logs. If you are curious, I can introduce you to a team that explains the licensing process."

## How do rewards work?

A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details. The reward rate is 25% of Actual Supplier Payout. The cumulative cap is $100,000 per referred company. The partner reward is never deducted from what the company receives. No reward is guaranteed.

## When is it not worth raising?

- The warehouse is under two years old and nothing is archived.
- Queries are generated entirely by a vendor tool the company cannot export.
- The company analyses data it holds for clients, so the rights are theirs, not the company's.
- Management will not consider an exclusive license.

## Related reading

[Management consultants](/for/management-consultants) often meet the same records through operating reviews. The sibling pages on [GL coding histories](/data-types/gl-coding-history-ai-training-data), [sales handoff notes](/data-types/sales-to-customer-success-handoff), [capex reviews](/data-types/capex-request-post-implementation-review) and [vendor bank-detail change verification](/data-types/vendor-bank-account-change-verification-process) cover other record sets, and [what AI training data is](/proof/what-is-ai-training-data) gives the basics. The [exception handling guide](/guides/exception-handling-ai-agents-data) explains why corrections add value.

## Next step

Read the [who qualifies](/who-qualifies) baseline, then [register as a partner](/register) and send your first introduction.