← Back to the exerciseThe full worked example: Check vendors' own reports before you renew
campo.
Field guide / L&D / UC-001 · SC-10
The result

"82% engagement" means a login. Act on Vendor B by November 30.

A Director of Learning Operations at a 15,000-person manufacturer checks three vendors' own reports against her data and brings procurement a recommendation for each, with any AI chat her company allows.

The resultWhat Priya sends procurement and the CHRO. Fictional company and vendors.
OSTRANDER ELECTRIC · LEARNING VENDOR RENEWALS
Renew, renegotiate or exit
For: Mark Ellison, Procurement, and the CHRO
Nov 16 · From: Priya Shah

DO FIRST

BY NOV 30
Vendor B auto-renews before our review
Ask for a written extension past December 10. If they won't agree, give notice so we keep the choice. · Input 1

RECOMMENDATIONS

RENEGOTIATE
A: Course library ($900,000)
Their 82% counts logins. Our LMS shows 4,100 people, 27% of licences, completed a course. Cut licences toward real use. The $4.1M ROI has no method and gets no weight. · Inputs 2, 5
DECIDE AFTER DATA
B: VR safety simulations ($600,000)
The 31% drop compares our six sites this year with all sites last year. Our safety team compares the same sites, same periods, before we decide. · Input 3
CUT OR EXIT
C: Language app ($400,000)
340 of 3,000 licences in use (11%). Cut to about the number in use if a business owner confirms the need by December 1. Otherwise exit. · Input 4

WHAT THE CLAIMS MEASUREInputs 2 to 5

A · 82% engagedLogins
Our data: 27% completed
A · $4.1M ROIUnverified
No method given
B · incidents -31%Not like for like
Different sites and years
C · 7.2 hoursActive users only
11% of licences used
Caught on review
1
"82% engagement" is anyone who logged in once; our LMS shows 27% completed a course
2
The 31% safety drop compares different sites in different years
3
Vendor B auto-renews on November 30, before the December 10 review
UC-001 · SC-10 · Rev A
FOR
Director of Learning Operations
SECTOR
Electrical equipment manufacturing
COMPANY
15,000 people · 14 sites · US, Mexico
TASK
Check three vendors' reports before renewal
WRITTEN FOR
ChatGPT · Claude · Gemini · Copilot
AI TEST
Claude, Nov 13, 2026 · 7 of 7 found
Step 0 · The mess

What landed on Priya's desk

A procurement email and three vendor reports, each with a big number and a small footnote. Her own LMS team sent one line. The footnotes and that line change the picture.

EMAIL · PROCUREMENT[1]
Learning vendor renewals
“$1.9M a year in total … Vendor B auto-renews on November 30 unless we give notice.”
1
A deadline before the meeting.
VENDOR A REPORT[2]
Online course library
“Engagement: 82%! … Estimated ROI: $4.1M.” Footnote: engaged = logged in once.
2
A login called engagement.
VENDOR B REPORT[3]
VR safety simulations
“Incident rate down 31%.” Footnote: our sites this year against all sites last year.
3
Two different groups.
VENDOR C REPORT[4]
Language-learning app
“7.2 hours per active user!” Footnote: active users: 340.
4
340 of 3
NOTE · OUR LMS TEAM[5]
Vendor A
“4,100 of our people completed at least one course this quarter.”
5
Our own number.
You're not alone
36%
of SaaS licences go unused in the average organisation.
Zylo 2026 SaaS Management Index (more than 40 million licences and $75 billion in spend), January 2026. Source
Steps 1–3 · How it works

Three steps, one chat

Here the AI checks numbers written by the people selling to you. The prompt tells it to be skeptical.

1
Open a new chat

Use the AI tool your company approves: ChatGPT, Claude, Gemini or Copilot.

2
Paste the prompt and the reports

Copy the prompt and the five fictional inputs from the appendix into one message.

3
Check the footnotes yourself

Read each footnote, check every date, then make the calls the AI shouldn't make for you.

The prompt, part by part
Copy-ready version in the appendix

I'm Director of Learning Operations at an electrical equipment manufacturer with about 15,000 people. Three learning vendors are up for renewal, and procurement and our CHRO want a recommendation for each at the December 10 vendor review. Today is November 16. Below are procurement's email, each vendor's own quarterly report, and a note from our LMS team.

1
Who you are, what's being decided, by whom and by when. The date matters: it lets the AI spot the deadline.

Please give me:

  1. A table: each vendor's main claims, what each claim actually measures, and what our own data says.
  2. Renew, renegotiate or exit for each vendor, with a one-line reason.
  3. Anything I must do before the review.
2
A table, a call for each vendor, and actions. The table puts their number next to yours.

Be skeptical: the vendors wrote these reports. Check how each metric is defined, what each comparison is really comparing, and what we pay for against what we use. Check dates. Don't add facts that aren't in the inputs.

3
The heavy lifting: be skeptical, check definitions, comparisons, use against cost, and dates. It found all seven problems.
Where to paste
Every tool has one message box. Paste the prompt, add a blank line, paste the reports, send. Vendor PDFs can be attached as files if your tool allows it.
Safe to try
Practise with the fictional reports first. Real contract terms and prices are confidential: use only your company's approved tool.
The AI's first answer

What Claude actually wrote

Exactly as it came back, not edited. Read it the way Priya did: check each point against the source it cites.

UNEDITED
Claude · fresh session, no other context · Nov 13, 2026 · one run

1. Vendor claims compared with our own data

Vendor Vendor's claim What it actually measures What our own data says
A: Course library ($900,000, 15,000 licences) "Engagement: 82%, 12,300 engaged learners" Their footnote defines "engaged" as logged in at least once in the quarter. That counts logins, not learning. 12,300 / 15,000 = 82%. Our LMS team says 4,100 people completed at least one course this quarter. That is 27% of licences (4,100 / 15,000). About 8,200 people logged in but did not complete a course. A: same people?
"Estimated ROI: $4.1M in productivity gains" No method, inputs or assumptions are given, and the vendor calls it "estimated". We have no data that supports it. Treat it as unverified. found
Cost $900,000 / 15,000 = $60 per licence $900,000 / 4,100 ≈ $220 per person who completed a course
B: VR safety simulations ($600,000, 6 sites) "Module completion: 96% (410 operators assigned)" The share of assigned operators who completed modules, about 394 of 410. It shows completion, not changed behaviour. We have no data on this. Cost is about $1,463 per assigned operator ($600,000 / 410).
"Incident rate at participating sites: down 31%" Their footnote says this compares participating sites this year with all company sites last year. The groups differ, and so do the periods. "This year" is not the same as "this quarter", and the year may not be complete. We have no data on this. The 31% does not show the program caused anything. We need the same six sites, measured over the same periods, from our own safety records. found
Contract Auto-renews November 30 unless we give notice. That is 14 days from today and 10 days before the December 10 review. found
C: Language app ($400,000, 3,000 licences) "Average learning time: 7.2 hours per active user" The average covers only 340 active users (footnote). It leaves out the 2,660 licence holders who did not use the app. "Active" is not defined. 340 / 3,000 = 11% of licences used. Total time is about 2,448 hours, which works out to roughly 0.8 hours per licence. found
Cost $400,000 / 3,000 ≈ $133 per licence $400,000 / 340 ≈ $1,176 per active user

The three contracts add up to $1.9M ($900k + $600k + $400k), which matches procurement's email.

2. Recommendation for each vendor

  • Vendor A: Renegotiate. Only 27% of licences led to a completed course, so cut the licence count closer to real use or move to usage-based pricing. Give the $4.1M ROI no weight unless the vendor shows its method.
  • Vendor B: Renegotiate, and act before November 30. The 31% safety claim compares mismatched sites and periods, so we can't tell yet whether the program works. Get a short extension, or give notice to keep our options open, rather than auto-renewing by default. I'm not recommending exit: dropping a safety program on weak evidence either way is a risk. B: decide after data
  • Vendor C: Renegotiate down sharply, or exit. 89% of licences went unused. Cut to roughly the number of real users, about 340. Exit if no business owner can explain why we need the program. C: one call

3. What to do before December 10

  1. Vendor B, this week: read the contract today for the notice requirements, such as a notice period, format and recipient. Then send notice, or get a written extension past December 10, before November 30. Tell Mark Ellison in Procurement right away.
  2. Vendor B: ask our safety team for incident rates at the six participating sites, this year against last year, for matching periods. Also get the same figures for the sites that did not take part. Confirm whether "this year" is a full or partial year.
  3. Vendor A: ask the vendor for the method and inputs behind the $4.1M. Confirm that our LMS figure of 4,100 covers the same quarter as their report.
  4. Vendor C: ask how "active user" is defined and how usage is spread across those 340 people. Find out which business group asked for the app and whether it still needs it.
  5. All three: check that each report covers the same quarter. The inputs don't name the quarter, so the periods may not line up. Get each contract's terms on reducing licences and on notice before you negotiate.

Unknowns that stay open: the basis of Vendor A's ROI figure; Vendor B's real effect on safety; how Vendor C defines "active"; and the exact reporting period for every report.

foundcaught a planted problemA–CPriya changes it; see the next section
The human part

What Priya changed, and why

Claude found every planted problem, including the deadline. Priya's edits remove one number the inputs don't support and turn two "or" answers into decisions.

What Claude found: 7 of 7 planted problems
One run
✓“Engagement” is a login
✓Our LMS shows 27% completed
✓$4.1M ROI unverified
✓Safety drop not like for like
✓11% of language licences used
✓Average over active users only
✓Auto-renewal before the review
Three edits, one catch and two judgment calls
A
CLAUDE WROTE
“About 8,200 people logged in but did not complete a course.”
PRIYA'S VERSION
Removed. The page shows 82% logins and 27% completions side by side.
WHY · ACCURACY
The 12,300 comes from the vendor and the 4,100 from our LMS. Nothing says they're the same people, so subtracting one from the other is a guess.
B
CLAUDE WROTE
Vendor B: “Renegotiate, and act before November 30 … Get a short extension, or give notice.”
PRIYA'S VERSION
Decide after our own safety data. First ask for an extension past December 10; give notice only if they refuse.
WHY · PRIORITY
It's a safety programme. Priya won't argue price or cut it until the same-site comparison is in, and the deadline goes at the top of the page.
C
CLAUDE WROTE
Vendor C: “Renegotiate down sharply, or exit.”
PRIYA'S VERSION
Cut to about the number in use if a business owner confirms the need by December 1. Otherwise exit.
WHY · DECISION
Procurement asked for one call per vendor. An "or" sends the decision back to them.
What only you can do
Know which programme you'd never cut on price alone, ask for time before giving notice, and give procurement one answer each. The AI read the footnotes. You make the calls.
Your turn

Make it yours

Use it whenever a vendor's own report is part of a renewal, a budget review or a business case.

Your prompt: fill in the brackets

I'm the [your role] at a [industry] company with about [number] people. [Number] vendors are up for renewal, and [who] want a recommendation for each at [the meeting] on [date]. Today is [date]. Below are [the emails, each vendor's report, and our own usage data].

Please give me: 1. A table: each vendor's main claims, what each claim actually measures, and what our own data says. 2. Renew, renegotiate or exit for each vendor, with a one-line reason. 3. Anything I must do before the review.

Be skeptical: the vendors wrote these reports. Check how each metric is defined, what each comparison is really comparing, and what we pay for against what we use. Check dates. Don't add facts that aren't in the inputs.

Five checks before you send
1
You've read every footnote and definition.
2
Each comparison uses the same group over the same period.
3
The vendor's number and yours aren't treated as the same people unless you know they are.
4
Every renewal and notice date is before your meeting, or flagged.
5
Each vendor gets one call, not an "or".
Make it repeatable (optional)
Save the prompt once and reuse it each renewal season: a custom GPT in ChatGPT, a project in Claude, a Gem in Gemini, or an agent in Copilot. Some companies switch these off; copy and paste always works.
With real data
Contract prices and terms are usually confidential. Use only the AI tool your company approves, and follow its rules.
What this example shows
One run of this prompt in Claude found all seven problems planted in fictional vendor reports, including the auto-renewal deadline.
What it doesn't show
That every tool, setting or run will do the same, or how much time it saves you. Ostrander Electric, its people and its vendors are fictional.
Appendix

Copy-ready: the prompt and the five inputs

Copy the prompt, then all five inputs, into one message in a new chat. Everything here is fictional and safe to paste.

The prompt

I'm Director of Learning Operations at an electrical equipment manufacturer with about 15,000 people. Three learning vendors are up for renewal, and procurement and our CHRO want a recommendation for each at the December 10 vendor review. Today is November 16. Below are procurement's email, each vendor's own quarterly report, and a note from our LMS team.

Please give me:

  1. A table: each vendor's main claims, what each claim actually measures, and what our own data says.
  2. Renew, renegotiate or exit for each vendor, with a one-line reason.
  3. Anything I must do before the review.

Be skeptical: the vendors wrote these reports. Check how each metric is defined, what each comparison is really comparing, and what we pay for against what we use. Check dates. Don't add facts that aren't in the inputs.

[1] Email from Mark Ellison, Procurement

Subject: Learning vendor renewals, Dec 10

Priya, three learning contracts are up, $1.9M a year in total. We need renew, renegotiate or exit for each by the Dec 10 review. Their quarterly reports are below. Note: Vendor B auto-renews on November 30 unless we give notice.

[2] Vendor A: online course library ($900,000 a year, 15,000 licences)

Engagement: 82%! 12,300 engaged learners this quarter. Estimated ROI: $4.1M in productivity gains. Footnote: engaged = logged in at least once in the quarter.

[3] Vendor B: virtual-reality safety simulations ($600,000 a year, 6 sites)

Module completion: 96% (410 operators assigned). Incident rate at participating sites: down 31%. Footnote: comparison is participating sites this year against all company sites last year.

[4] Vendor C: language-learning app ($400,000 a year, 3,000 licences)

Average learning time: 7.2 hours per active user this quarter! Footnote: active users: 340.

[5] Note from our own LMS team

Vendor A: 4,100 of our people completed at least one course this quarter.