Computer-use task data in an ordinary company: a worked desktop task card for referral partners

Short answer

Computer-use task data is the trail a desktop task leaves: what the person saw, what they did in which application, and an end state someone can check without watching. In an ordinary company that trail sits in an inbox, a business system's status and field history, and a sent message. The worked task card below shows one accounts payable task written the way researchers write benchmark tasks, and marks which steps leave records and which do not.

Visual briefFull-size diagram
A task card for an illustrative invoice-hold task: a starting state panel, six numbered observation and action steps across an inbox, an ERP and email, and three completion checks that turn green, with the two steps that leave no record marked in orange.
Illustrative accounts payable task at a fictional distributor. Completion checks read the end state from records, not from a screen recording.

This page is part of the computer-use agent explainer, which defines the term and walks a freight example. Here the job is different: one complete task card a partner can use as a pattern when a client describes their own desktop work.

How researchers write a computer-use task

The OSWorld benchmark, presented at NeurIPS 2024, sets 369 tasks in real desktop and web applications, including workflows that span several applications. Every task has three parts:

  1. A starting state, set up by a configuration so the machine looks the same each time.
  2. An instruction, the goal in plain language.
  3. An evaluation script that inspects the final state and decides pass or fail, without grading the path the agent took.

At publication, humans completed 72.36% of tasks and the best model 12.24%. Scores have moved since; the structure is what matters to a partner, because ordinary company records often hold the first and third parts already. The AI agent benchmarks guide compares OSWorld with other benchmarks.

Worked task card: releasing an invoice held on a price variance

Illustrative and fictional. A 240-person industrial parts distributor. An accounts payable clerk handles a vendor invoice that the ERP has put on hold because the unit price is higher than the purchase order.

Starting state. Invoice on hold with status "Price variance". PO line: 400 bearings at the agreed unit price. Vendor invoice billed at a higher unit price. Vendor's sales rep known from earlier correspondence.

Instruction. "Resolve the price variance on this invoice so the vendor is paid the PO price, and tell the vendor."

StepTypeWhat happensApplicationRecord left behind
1ObserveOpens the hold queue and sees the invoice flaggedERPHold status with reason and date
2ObserveOpens the invoice PDF and the PO side by sideERP and PDF viewerNone: viewing is rarely logged
3DecideConfirms the PO price is the agreed oneHead and screenNone: the comparison is not recorded
4ActEmails the vendor asking for a credit memo for the differenceEmailSent message, timestamp, thread
5ActEnters the vendor's credit memo when it arrives and matches it to the invoiceERPCredit memo document linked to the invoice
6ActReleases the holdERPStatus change with user and time

Completion checks. Written as conditions a script, or a reviewer, could test against records:

  • Check A: the hold status changed from "Price variance" to released, after the credit memo date.
  • Check B: invoice amount minus credit memo equals PO quantity times PO price.
  • Check C: the payment run paid that net amount once, with no second payment for the same invoice.

Each check reads only what the systems kept. Steps 2 and 3 leave nothing behind, and that is normal; researchers and buyers decide how to handle gaps in the middle. The start (step 1) and the checkable end (steps 5 and 6, plus the payment) are what make this a usable task.

What turns an ordinary process into a usable task

Not every process makes a good task. Four properties separate the useful ones, and a partner can ask about each in a sentence.

PropertyWhy it mattersQuestion to ask the operations lead
A recorded triggerFixes the starting stateIs the request or exception that starts the work kept with the record?
Recorded actionsShows what was done, in which systemDo status changes and edits keep old value, new value, user and time?
A checkable endLets someone decide pass or failIs there a field, total or status that proves the job was done right?
Enough spanGives many examples, including rare casesHow many years of that history do you keep, including retired systems?

Span is where records most often fall short. Field history is not kept forever by default in many systems; Salesforce, for example, retains field history for up to 18 months, or 24 months through the API, unless the company adds Field Audit Trail. Ask how long, rather than assuming.

The which jobs are AI agents learning guide lists the kinds of roles whose desktop work tends to fit this pattern; browser-only work is covered in browser agent training data, and the longer explainer on workflow data is computer-use agents and workflow training data.

Qualifying a company from the task card

A partner never needs the records to qualify the company. Stay at metadata level:

  • Which records: hold queues, status histories, credit memos, sent messages.
  • How much: rough tasks per month and years kept.
  • Which systems: ERP, email, any retired platform that held earlier years.
  • Who owns them: usually the CFO for finance workflows, the COO for operations.
  • Who can authorize a licensing discussion: the CEO, or the owner in an owner-run company.

The company also needs at least 50 full-time employees at peak (contractors excluded), a documented operating history, rights to license and an authorized contact; the company fit checker runs through these. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

What stays unknown until SourceX reviews it: whether vendor names and amounts must be removed, which years and systems are in scope, whether any buyer wants this task type and on what terms. The company is paid under its own licensing agreement with SourceX; any partner reward is funded separately by SourceX and never deducted from what the company receives.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If a client's operations lead can describe a task with a recorded trigger and a checkable end, register as a partner, confirm fit on who qualifies, and read the program terms before you introduce the owner.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Why not just record the clerk's screen?

Most companies do not, and recordings capture far more personal and customer detail than a task needs. The records a business system already keeps, such as status changes, field histories and sent messages, usually show the end state well enough to check.

Does every step need a record for the task to be useful?

No. In OSWorld the evaluation checks the final state rather than every click. Gaps in the middle, such as searches and comparisons people do on screen, are normal; what matters is that the start and the checkable end are recorded.

Should a partner ask for an example task from the client's system?

No. Describe the pattern, ask the four metadata questions, and stop there. Any example data is handled by SourceX under an agreement, never by the partner.

Sources

  1. arxiv.org/abs/2404.07972
  2. neurips.cc/virtual/2024/poster/97468
  3. developer.salesforce.com/docs/atlas.en-us.securityImplGuide.meta/secu…
  4. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment