AnnotationPrep

Instruction & Negative Constraints practice questions

10 questions from the Starter assessment: A/B evaluation & fact-checking practice test.

Start practising

Instruction & Negative Constraints practice drill

10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.

Sample scenarios

Prompt: 'List 5 European capitals in alphabetical order. Output ONLY the city names, separated by commas, with no extra text.'

Model output: 'Here are 5 European capitals in alphabetical order: Amsterdam, Berlin, Lisbon, Madrid, Vienna.'

Did the model follow all instructions?

  1. Yes, all 5 are European capitals and in alphabetical order.
  2. No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.
  3. No, because Vienna is not in Europe.
  4. Yes, because polite lead-ins are excluded from 'extra text' rules.
Show the answer

B. No, it violated the 'Output ONLY the city names... with no extra text' constraint by adding 'Here are 5 European capitals in alphabetical order:'.

Strict output-only constraints forbid conversational lead-ins and sign-offs. Including 'Here are...' is one of the most common instruction-following failures tested in AI trainer assessments.

Principle: Instruction Following: Output-Only Constraints

Prompt: 'Write a 2-paragraph product description for a ceramic coffee mug. Do not use any adjectives ending in -y (such as cozy, sturdy, or shiny).'

Which phrase in a model's response would FAIL this negative constraint?

  1. 'The matte ceramic surface feels warm in your hands.'
  2. 'Its thick handle provides a comfortable grip every morning.'
  3. 'Enjoy your daily brew in a weighty, glossy mug.'
  4. 'Designed to retain heat during long work sessions.'
Show the answer

C. 'Enjoy your daily brew in a weighty, glossy mug.'

'Every' is a determiner, not an adjective ending in -y, whereas 'weighty' and 'glossy' are both adjectives ending in -y, directly violating the negative constraint.

Principle: Negative Constraints: Lexical & Suffix Rules

Prompt: 'Return a valid JSON object with keys "title" (string) and "tags" (array of 3 strings).'

Model A outputs:
```json
{"title": "Sourdough Guide", "tags": ["baking", "bread", "starter"]}
```
Model B outputs:
{"title": "Sourdough Guide", "tags": ["baking", "bread", "starter", "yeast"]}

Assuming the prompt did NOT forbid markdown code fences, which model followed the schema constraint?

  1. Model A, because "tags" has exactly 3 strings as requested, whereas Model B has 4.
  2. Model B, because it did not use backticks.
  3. Both failed.
  4. Model B, because more tags is more helpful.
Show the answer

A. Model A, because "tags" has exactly 3 strings as requested, whereas Model B has 4.

Model A satisfied both the key names and the exact array length (3 strings). Model B violated the explicit count constraint by returning 4 tags.

Principle: Instruction Following: Schema & Array Length

Other topics