10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.
Sample scenarios
Prompt: 'List 5 European capitals in alphabetical order. Output ONLY the city names, separated by commas, with no extra text.'
Model output: 'Here are 5 European capitals in alphabetical order: Amsterdam, Berlin, Lisbon, Madrid, Vienna.'
Did the model follow all instructions?
Yes, all 5 are European capitals and in alphabetical order.
No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.
No, because Vienna is not in Europe.
Yes, because polite lead-ins are excluded from 'extra text' rules.
Show the answer
B. No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.
Strict output-only constraints forbid conversational lead-ins and sign-offs. Including 'Here are...' is one of the most common instruction-following failures tested in AI trainer assessments.
Prompt: 'Write a 2-paragraph product description for a ceramic coffee mug. Do not use any adjectives ending in -y (such as cozy, sturdy, or shiny).'
Which phrase in a model's response would FAIL this negative constraint?
'The matte ceramic surface feels warm in your hands.'
'Its thick handle provides a comfortable grip every morning.'
'Enjoy your daily brew in a weighty, glossy mug.'
'Designed to retain heat during long work sessions.'
Show the answer
C. 'Enjoy your daily brew in a weighty, glossy mug.'
'Every' is a determiner, not an adjective ending in -y, whereas 'weighty' and 'glossy' are both adjectives ending in -y, directly violating the negative constraint.