A Director of Leadership Development at a 40,000-person energy equipment maker checks her own programme's claims before the CHRO takes them to the talent review, with any AI chat her company allows.
A CHRO who wants to say the programme works, a proud summary from her own team, and the HR data underneath. The data includes the comparison the summary didn't use.
Here the AI checks whether a programme deserves the credit it claims. The prompt asks who is in each group.
Use the AI tool your company approves: ChatGPT, Claude, Gemini or Copilot.
Copy the prompt and the four fictional inputs from the appendix into one message.
Check the key comparisons yourself, then choose one fair test you'll run before next year's review.
I'm Director of Leadership Development at an energy equipment maker with about 40,000 people in 15 countries. Our CHRO wants one page on whether our Emerging Leaders Program works, for the annual talent review on December 8. Today is November 23. Below are his email, the programme team's summary, HR's cohort data and a 360 feedback summary.
Please give me:
Check who is in each group being compared, who is missing from the numbers, whether each time window is complete, and how many people each figure rests on. Don't add facts that aren't in the inputs.
Exactly as it came back, not edited. Read it the way Amara did: check each point against the source it cites.
1. Does the evidence show the programme works?
No. The evidence doesn't show that it works or that it doesn't. Measured against the right peer group, graduates were promoted at 41% (19 of 46) and other high-potential managers at 38% (34 of 90). That gap is small, it leaves out the 12 people who didn't graduate, and it includes a cohort whose 18 months aren't over yet.
2. Each claim in the programme team's summary
| Claim | What it compares | Does it hold? |
|---|---|---|
| "Graduates are promoted at 3x the rate of their peers" | 41% for graduates against 14% for "all managers" (41 ÷ 14 ≈ 2.9) | No. "All managers" aren't the graduates' peers. Graduates were picked because they were rated high potential. The inputs give the right comparison: high-potential managers not in the programme, 34 of 90 promoted (38%). Against that group graduates are at about 1.1x, not 3x. The 14% figure also comes with no headcount, no source and no time window. found |
| "41% of graduates were promoted within 18 months" | 19 promoted out of 46 graduates (11 + 8) | The arithmetic is right (41.3%), but the figure overstates. (a) It counts graduates only. 58 were selected, and the 12 who didn't graduate (7 left the company, 5 withdrew) are left out. Counting everyone selected, at least 19 of 58 (33%) were promoted, which is below the 38% for non-participants. Whether any of the 5 who withdrew were promoted is unknown. (b) The 2024 cohort graduated in June 2025, so its 18-month window doesn't close until December 2026. On November 23 it has about 17 of 18 months, assuming the current year is 2026. Its 8 of 22 is therefore an incomplete count. The 2023 cohort's window is complete: 11 of 24 (46%). (c) For the 34 of 90 comparison, the inputs don't say whether every member has had a full 18 months, or whether it includes people who left. A: like for like |
| "Results in Asia were exceptional: 83% promoted" | 5 of 6 Asia graduates, both cohorts | No. It rests on 6 people, so one promotion more or less moves it by about 17 points. It has the same problems as the overall figure: graduates only, partly incomplete window. And there's no Asia comparison group. found |
| 360 scores rose from 3.6 to 4.1 (from the 360 summary, not the team's summary, but likely to be cited) | The average of all 46 graduates before, against 12 graduates after | No. These are different groups. Only 12 of 46 (26%) completed the "after" 360, and we don't know who they were or what they scored before. If the people who chose to complete it were the ones doing well, the rise could come from that alone. found |
3. What we can honestly tell the talent review
4. What evidence we'd need to know whether it works
Claude found every planted problem. Amara removed one comparison that mixes two differently counted groups, and turned "we don't know" into a plan to find out.
Use it whenever a programme takes credit for an outcome: promotions, retention, performance, safety.
I'm the [your role] at a [industry] company with about [number] people. [Who] wants [a page or claim] on whether [the programme] works, for [the meeting] on [date]. Today is [date]. Below are [the request, the programme's own claims, the cohort data and any survey data].
Please give me: 1. Does the evidence show the programme works? Two sentences. 2. Each claim in [the summary]: what it compares, and whether it holds. 3. What we can honestly tell [the audience]. 4. What evidence we'd need to know whether it works.
Check who is in each group being compared, who is missing from the numbers, whether each time window is complete, and how many people each figure rests on. Don't add facts that aren't in the inputs.
Copy the prompt, then all four inputs, into one message in a new chat. Everything here is fictional and safe to paste.
I'm Director of Leadership Development at an energy equipment maker with about 40,000 people in 15 countries. Our CHRO wants one page on whether our Emerging Leaders Program works, for the annual talent review on December 8. Today is November 23. Below are his email, the programme team's summary, HR's cohort data and a 360 feedback summary. Please give me: 1. Does the evidence show the programme works? Two sentences. 2. Each claim in the programme team's summary: what it compares, and whether it holds. 3. What we can honestly tell the talent review. 4. What evidence we'd need to know whether it works. Check who is in each group being compared, who is missing from the numbers, whether each time window is complete, and how many people each figure rests on. Don't add facts that aren't in the inputs. [1] Email from Henrik Dahl, CHRO Subject: Emerging Leaders, for the talent review Amara, I'd like to tell the talent review that Emerging Leaders works. Can you give me one page on the evidence by December 1? [2] Summary from the programme team Graduates are promoted at 3x the rate of their peers! 41% of graduates were promoted within 18 months, against 14% of all managers. Results in Asia were exceptional: 83% promoted. [3] Cohort data from HR | Cohort | Selected | Graduated | Did not graduate | Graduates promoted within 18 months | |---|---|---|---|---| | 2023 (graduated June 2024) | 30 | 24 | 6 (4 left the company, 2 withdrew) | 11 | | 2024 (graduated June 2025) | 28 | 22 | 6 (3 left the company, 3 withdrew) | 8 | Who is selected: managers rated high potential in the annual talent review. High-potential managers not in the programme, same period: 34 of 90 promoted within 18 months. Asia, both cohorts: 5 of 6 graduates promoted. [4] 360 feedback summary Average 360 score: 3.6 before the programme, 4.1 after (out of 5). Before: all 46 graduates. After: 12 graduates completed it.I'm Director of Leadership Development at an energy equipment maker with about 40,000 people in 15 countries. Our CHRO wants one page on whether our Emerging Leaders Program works, for the annual talent review on December 8. Today is November 23. Below are his email, the programme team's summary, HR's cohort data and a 360 feedback summary.
Please give me:
Check who is in each group being compared, who is missing from the numbers, whether each time window is complete, and how many people each figure rests on. Don't add facts that aren't in the inputs.
Subject: Emerging Leaders, for the talent review
Amara, I'd like to tell the talent review that Emerging Leaders works. Can you give me one page on the evidence by December 1?
Graduates are promoted at 3x the rate of their peers! 41% of graduates were promoted within 18 months, against 14% of all managers. Results in Asia were exceptional: 83% promoted.
| Cohort | Selected | Graduated | Did not graduate | Graduates promoted within 18 months |
|---|---|---|---|---|
| 2023 (graduated June 2024) | 30 | 24 | 6 (4 left the company, 2 withdrew) | 11 |
| 2024 (graduated June 2025) | 28 | 22 | 6 (3 left the company, 3 withdrew) | 8 |
Who is selected: managers rated high potential in the annual talent review. High-potential managers not in the programme, same period: 34 of 90 promoted within 18 months. Asia, both cohorts: 5 of 6 graduates promoted.
Average 360 score: 3.6 before the programme, 4.1 after (out of 5). Before: all 46 graduates. After: 12 graduates completed it.