AgentBench / results
← All suites

Following exact instructions

Keep to rules about wording, length, language and formatting.

Compare model results on this test →

How this test is scored · 23 questions · Strict prompt compliance

What these tests cover

These tasks ask the model to follow precise rules about wording, length, language and layout while answering everyday questions or writing creatively. They test specific output constraints, often several at once, rather than establishing the accuracy or quality of everything written.

About this explanationGenerated by GPT Astra (gpt-6-astra) · 2026-10-12. Shared across models for this version of the test. The original question and checks remain the source of truth.

How it is scored

Official strict prompt-level compliance is the primary score. Strict per-instruction fraction and loose prompt/instruction scores are retained separately. Original prompts are unchanged. Reviewed by scoring-adjudication-v2: exact Make-24 input numbers; explicit five-letter choice tokens and final three-digit math answers; invalid cube, contradictory asterisk items, and CPU monitor with invalid check and incomplete replay evidence excluded uniformly. Original evidence and grades retained.

Strict prompt compliance · 23 fixed primary questions

Version d2e088a636b3965f. Excluded questions remain available below; retired tasks are kept in the edition archive.

Google Research IFEval; Apache-2.0. · Original source · Publication terms

Recorded evidence availability

Counts cover question definitions and all saved attempts, including historical attempts. They are evidence-component counts, not scored questions. A zero means no components in that state.

Componentfullredactedsource link onlynot recordedwithheld
answer2177009020
artifacts00030790
checks2233008460
context00030790
input19250011540
prompt2203008760
reasoning10010020780
reference00030790
tools00030790

Full: recorded material is available. Redacted: explicit publication rules removed identified text. Source link only: follow the cited source. Not recorded: no saved component. Withheld: material is not published.

Question bank

1–20 of 24 matching questions · 24 in this version

  • Lowercase Darth Vader Dialogue

    The model must write a conversation with an angry Darth Vader in three parts separated by asterisks, using only lowercase letters and repeating a required word at least six times. This exercises simultaneous control of wording, capitalization and structure.

    if-2943 · primary
  • Two Haiku Without Commas

    The model is asked for two question-form haiku about foolish behavior and not studying, aimed at young readers. It must separate the responses with six asterisks, include a title in double angle brackets and avoid commas, combining creative writing with precise formatting.

    if-2704 · primary
  • Hamptons Overview With Placeholders

    The model must describe the Hamptons in at least 300 words while including at least three square-bracketed placeholders and exactly two Markdown bullet points. This exercises length control alongside required text elements and list formatting.

    if-2284 · primary
  • Team Comparison With Paragraph Rules

    The model is asked to compare Tottenham Hotspur and Stockton in a specified season using exactly three paragraphs separated by blank lines. Required keywords, a prescribed opening word for the third paragraph and a specially marked postscript exercise detailed placement and structure control.

    if-2549 · primary
  • Weather Template With Text Constraints

    The model must draft a weather chatbot template that accepts a location, stays under six sentences and highlights at least two sections with asterisks. A separate requirement governs use of a particular letter, adding character-level control to the formatting demands.

    if-1174 · primary
  • Long Comparison With Forbidden Words

    The model must discuss whether a beach or a park is more likely to contain a dog in at least 900 words. It must avoid two specified words and end with a fixed phrase, exercising constraint tracking across a long response.

    if-3114 · primary
  • Portuguese Text Inside Quotation Marks

    The model is asked to write about Adam and Eve entirely in Portuguese and enclose the whole response in double quotation marks. This exercises adherence to a requested language and an exact outer format.

    if-2724 · primary
  • All-Caps Critique of Obeying Laws

    The model must critique the claim that bad laws should not be followed, using two labeled paragraphs written entirely in capital letters. This exercises consistent capitalization and section labeling within an argumentative response.

    if-1021 · primary
  • Flood Poem With Capitalized Words

    The model is asked to write a poem about flooding in Donnell, Texas, with a title in double angle brackets and at least three fully capitalized words. This combines creative writing with explicit title and capitalization requirements.

    if-2853 · primary
  • Repeat the Request Before Rewriting

    The model must repeat a supplied rewriting request word for word before giving an unusual version of its biblical sentence, with nothing added beforehand. This exercises exact copying and keeping required material in the correct order.

    if-2482 · primary
  • Character Sheet in JSON

    Acting as a dungeon master, the model must create a character sheet with a randomly chosen class for a solo player. The entire response must be JSON inside a Markdown code block and include two specified keywords, exercising structured output and required wording.

    if-2591 · primary
  • Tennis Opinion With Fixed Responses

    The model is asked whether Roger Federer is a better tennis player than Rafael Nadal, but may respond only with one of three supplied phrases. This tests staying within a fixed response set without adding explanation.

    if-3749 · primary
  • Teen Newspaper Name With Postscript

    The model must suggest a name for a newspaper aimed at Melbourne teenagers and finish with a postscript using the requested marker. The specific instruction-following demand is remembering the required ending while answering a creative naming request.

    if-2622 · primary
  • Riddle Story With Highlighted Sections

    The model is asked to turn a supplied riddle and its answer into a funny story, highlighting at least two sections with asterisks. The task pairs creative expansion with a concrete requirement for marked text.

    if-2905 · primary
  • Formal Rewrites Without Commas

    The model must provide two formal alternatives to a casual greeting and statement about visiting the beach, without using any commas. This exercises following a punctuation restriction while changing the tone of supplied text.

    if-1187 · primary
  • Funny Email With Minimum Sentence Count

    The model is asked to write a long, funny email to a friend about the ingredients of a good life, using at least 20 sentences. The explicit constraint exercises sentence-count control within an informal writing task.

    if-292 · primary
  • Seasons Explanation With Postscript

    The model must explain why seasons occur to elementary school students and add a postscript with the requested marker at the end. This pairs an audience-specific explanation with a required closing element.

    if-1800 · primary
  • Programming Comparison With Bracketed Title

    The model is asked to explain the pros and cons of Kotlin versus Java and include a title enclosed in double angle brackets. The specific formatting demand tests whether a small presentation instruction is retained during a technical comparison.

    if-1322 · primary
  • Long News Post in Few Sentences

    The model must write a blog post about recent US news using at least 250 words but fewer than five sentences, plus a title in double angle brackets. This exercises coordinating word and sentence limits with title formatting.

    if-1392 · primary
  • Levee Comparison Entirely in Korean

    The model is asked to explain the difference between a levee and an embankment while responding only in Korean. This tests following a language restriction when the question itself is written in English.

    if-2225 · primary
Filters change this list, not the fixed scoring denominator. Search currently covers question IDs, titles and available task explanations. Focus a control to load prompt/outcome search where this edition provides it.