Comparing two AI responses, catching subtle hallucinations, checking every positive and negative prompt constraint, and choosing the strongest justification.
10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.
Timed mock assessment
31 scenarios in 37 minutes, drawn from every topic. No answers shown until the end, then a full score breakdown, your weakest topics and a review of what you missed.
Prices are in US dollars, plus tax where it applies. Stripe, our payment processor and the seller of record, adds tax at checkout.
All sales are final. If something is really wrong, such as a duplicate charge or a purchase that does not unlock what you bought, write to support@annotationprep.com and we will put it right.
One-time payment. Nothing renews.
You have an assessment ticket for this practice test. Start the mock assessment to use it.
An AI summarizes a provided medical abstract: 'The study followed 420 patients over 6 months and found a 12% reduction in systolic blood pressure, though 8% reported mild dizziness.'
The AI writes: 'The study of 420 patients proved the drug cures hypertension with zero side effects.'
How many factual/groundedness errors are in the AI's sentence?
Zero—it is a reasonable paraphrase.
One—it changed 6 months to an unspecified duration.
Two major distortions: 'cures hypertension' (unsupported overclaim vs. 12% reduction) and 'zero side effects' (directly contradicts the 8% dizziness rate).
Three—420 patients is also wrong.
Show the answer
C. Two major distortions: 'cures hypertension' (unsupported overclaim vs. 12% reduction) and 'zero side effects' (directly contradicts the 8% dizziness rate).
Fact-checking AI summaries requires catching both direct contradictions ('zero side effects' vs. 8% dizziness) and unsupported overclaims ('cures hypertension' vs. 12% BP reduction).
Principle: Factuality & Groundedness: Overclaiming vs. Contradiction
Prompt: 'List 5 European capitals in alphabetical order. Output ONLY the city names, separated by commas, with no extra text.'
Model output: 'Here are 5 European capitals in alphabetical order: Amsterdam, Berlin, Lisbon, Madrid, Vienna.'
Did the model follow all instructions?
Yes, all 5 are European capitals and in alphabetical order.
No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.
No, because Vienna is not in Europe.
Yes, because polite lead-ins are excluded from 'extra text' rules.
Show the answer
B. No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.
Strict output-only constraints forbid conversational lead-ins and sign-offs. Including 'Here are...' is one of the most common instruction-following failures tested in AI trainer assessments.
Prompt: 'Write a 3-sentence summary of why sleep matters for memory, without mentioning REM sleep.'
Model A: 'Sleep helps the brain consolidate new memories into long-term storage. During deep sleep and REM sleep, neural connections are strengthened. Getting 7-9 hours each night improves recall and focus.'
Model B: 'Sleep gives your brain time to organize and store information learned during the day. Without enough rest, the hippocampus struggles to encode new facts into long-term memory. Consistent sleep schedules also improve daytime focus and problem-solving.'
Which model should win and why?
Model A, because it includes specific sleep stages and exact hour recommendations.
Model B, because Model A violated the explicit negative constraint against mentioning REM sleep.
Tie, because both models wrote exactly three sentences and are factually accurate.
Model A, because Model B used technical jargon like 'hippocampus'.
Show the answer
B. Model B, because Model A violated the explicit negative constraint against mentioning REM sleep.
In RLHF evaluation, violating an explicit negative constraint ('without mentioning REM sleep') is an automatic instruction-following failure. Even if Model A's facts are accurate, Model B followed every constraint while remaining accurate and concise.
Which of the following AI-generated citations is the strongest red flag for a fabricated ('hallucinated') academic reference that you must verify before approving?
A paper with a real author's name, a plausible-sounding title, a real journal name, and a DOI that returns a 404 or points to an unrelated chemistry paper.
A textbook published in 2018 with an ISBN that resolves to the exact book.
A government CDC webpage whose URL resolves to the matching topic.
An RFC number (e.g., RFC 9110) that matches HTTP Semantics.
Show the answer
A. A paper with a real author's name, a plausible-sounding title, a real journal name, and a DOI that returns a 404 or points to an unrelated chemistry paper.
LLMs frequently blend real author names and real journals into plausible-looking fake citations with non-existent or mismatched DOIs. Always verify DOIs, URLs, and case numbers.
Principle: Fact-Checking: Citation & DOI Verification