AI Made Simple

Free guide · Comparing AI tools on the same job

AI Tool Comparison Worksheet

Use this worksheet to compare AI tools on one real job instead of comparing feature lists or impressions.

1. Define the test

Copyable template 1
TEST JOB

Job to be done:
Audience:
Decision or action the output supports:
Required output:
Verified facts supplied:
Source files supplied:
Known missing information:
Prohibited claims:
Time limit per tool:
Review checklist:

Do not use confidential or customer data for an early comparison. Use a fictional or approved test case.

2. Use the same prompt

Copyable template 1
Create the requested output using only the supplied brief and sources.

Before drafting:
- list material missing information;
- separate verified facts from assumptions;
- state which supplied source supports each figure.

In the output:
- do not invent facts, results or citations;
- mark unresolved gaps as [NEEDS INPUT];
- follow the required format and audience.

After drafting, return a verification note with every figure used, its source and any correction you made.

[PASTE TEST JOB AND APPROVED SOURCES]

If a tool cannot access a source or complete part of the task, record that limitation. Do not quietly give one tool extra help.

3. Record the conditions

FieldTool ATool BTool C
Tool and product name
Model or mode shown
Account or plan
Test date
Connected apps used
Files supplied
Prompt changesNone / explainNone / explainNone / explain
Time to usable draft

Product behavior can change. The account, model and test date are part of the result.

4. Compare the work

Score each item from 0 to 2:

  • 0: failed or absent
  • 1: partly correct or required material correction
  • 2: correct and usable with normal review
CriterionTool ATool BTool CEvidence or notes
Used the supplied facts accurately
Identified missing information
Did not invent facts or sources
Corrected itself when challenged
Traced figures to sources
Followed audience and format
Produced a clear decision or action
Required little manual correction
Total out of 16

The total is a discussion aid, not a universal ranking. A failure on a critical criterion can outweigh a higher total.

5. Worked fictional test

Job: Draft a one-page recommendation on whether to extend the Northstar onboarding pilot.

Supplied facts: 18 invited, 14 completed, 3 needed follow-up, time saved not measured.

Known gap: owner of the extension is not named.

ObservationTool ATool B
FactsUsed all figures correctlyChanged 14 of 18 to “nearly 80%” without showing the calculation
Missing informationAsked for the owner before draftingAssigned the project manager without evidence
Unsupported claimsNoneClaimed the pilot improved efficiency
Source handlingNamed the tracker beside each figureReferred to “the data” without a source location
CorrectionNot neededRemoved the efficiency claim after challenge

Decision: Tool A is the safer fit for this source-grounded brief. This result applies to this job, account, model and test date. It does not prove Tool A is better for every task.

6. Correction test

Use the same challenge in each tool:

Copyable template 1
Audit your answer against the supplied facts and sources.

List:
1. every factual claim,
2. its source,
3. any assumption or inference,
4. anything you should correct.

Do not defend the first answer. Correct it when the evidence requires a change.

Record whether the tool finds the same issues you found and whether its correction introduces a new error.

7. Choose by the job

Copyable template 1
Chosen tool:
Job it is suited for:
Critical strength observed:
Critical limitation observed:
Human checks still required:
Account, model and test date:
When to retest:

Final check

  • □ Every tool received the same job, sources and constraints.
  • □ I recorded any prompt change or extra help.
  • □ I checked facts, gaps, corrections, sources and time.
  • □ I recorded the model, account and date.
  • □ I chose for this job, not for every possible job.
  • □ A human still reviews the final work.

The rule to keep: Compare tools on the work you actually need, under conditions you can repeat.