Reading without guessing
Answer from a passage—or say that the passage does not contain the answer.
Compare model results on this test →
How this test is scored · 16 questions · Exact-match accuracy
What these tests cover
These tasks ask the model to answer questions using only a supplied passage, without filling gaps from memory or guesswork. They exercise finding relevant details and deciding whether the text supports an answer. Responses must contain only the shortest supported answer, or exactly UNANSWERABLE when the passage does not provide one.
About this explanation
Generated by GPT Astra (gpt-6-astra) · 2026-10-12. Shared across models for this version of the test. The original question and checks remain the source of truth.How it is scored
Official normalized exact match and token F1. Exact match is primary; unanswerable responses map the explicit sentinel to empty string. Answerability groups reported in evidence.
Exact-match accuracy · 16 fixed primary questions
Version f211ac5906b2412c. Excluded questions remain available below; retired tasks are kept in the edition archive.SQuAD 2.0, Rajpurkar et al.; CC BY-SA 4.0. Passage and question attribution retained. · Original source · Publication terms
Recorded evidence availability
Counts cover question definitions and all saved attempts, including historical attempts. They are evidence-component counts, not scored questions. A zero means no components in that state.
| Component | full | redacted | source link only | not recorded | withheld |
|---|---|---|---|---|---|
| answer | 1796 | 0 | 0 | 592 | 0 |
| artifacts | 0 | 0 | 0 | 2388 | 0 |
| checks | 1796 | 0 | 0 | 592 | 0 |
| context | 16 | 0 | 0 | 2372 | 0 |
| input | 1616 | 0 | 0 | 772 | 0 |
| prompt | 1812 | 0 | 0 | 576 | 0 |
| reasoning | 831 | 0 | 0 | 1557 | 0 |
| reference | 16 | 0 | 0 | 2372 | 0 |
| tools | 0 | 0 | 0 | 2388 | 0 |
Full: recorded material is available. Redacted: explicit publication rules removed identified text. Source link only: follow the cited source. Not recorded: no saved component. Withheld: material is not published.
Question bank
1–16 of 16 matching questions · 16 in this version
- Finding a Yuan Variety Show
The model is asked to identify a type of variety show from a passage about Yuan-era arts. This exercises locating a specific term among related artistic forms and returning only the requested detail.
sq-57287f6a3acd2414000dfa55 · primary - Explaining a Shift in Economic Thinking
The model is asked why economists increasingly came to view income inequality as harmful to growth. It must isolate the main explanation in a quoted argument rather than substitute a nearby claim or outside economic knowledge.
sq-572a1046af94a219006aa790 · primary - Dating Warsaw’s Permanent Parliamentary Seat
The model is asked when Warsaw became the permanent seat of the General Sejm. This exercises matching a date to the precise event requested, while distinguishing it from other milestones in the same passage.
sq-5733266d4776f41900660713 · primary - Finding the Relevant Campus Count
The model is asked for a University of California campus count using a passage about the Tech Coast. It must select the relevant institution’s stated count, keep to the passage’s regional context, and avoid mixing university systems.
sq-5706139252bb891400689867 · primary - Identifying the Warsaw Confederation’s Effect
The model is asked what the Warsaw Confederation formally established. This exercises linking a named historical agreement to its stated effect while ignoring other political events described nearby and keeping the answer brief.
sq-5733266d4776f41900660714 · primary - Identifying Dinwiddie’s Chosen Messenger
The model is asked whom Governor Dinwiddie ordered to address the French presence in Virginia territory. It must distinguish the person assigned the mission from other named participants and return only the supported name.
sq-5733ef47d058e614000b662c · primary - Finding a Descendant Population Count
The model is asked how many people in Quzhou are descended from Confucius. This exercises extracting a population figure from a passage containing several people, locations, and historical periods without adding unrelated family history.
sq-57286c8cff5b5019007da21c · primary - Finding an Algebraic Generalization
The model is asked to name one algebraic generalization inspired by prime numbers. It must find an appropriate example in the passage, distinguish it from applications and unsolved problems, and supply just one brief answer.
sq-5729727baf94a219006aa43b · primary - Checking Kublai Khan’s Policy Direction
The model is asked what kinds of growth Kublai Khan discouraged, using a passage about his policies. This exercises checking the direction of a claim rather than selecting related words without verifying that they answer the question.
sq-5ad40397604f3c001a3ffd83 · primary - Reading a Negated War-Timing Question
The model receives a passage about a colonial conflict and is asked when violence did not start. It must attend to the question’s negation and determine whether the passage supports the requested timing rather than automatically extracting a date.
sq-5ad3a076604f3c001a3fe9ab · primary - Checking a Redistribution Qualifier
The model is asked about an unusual form of government wealth redistribution in a passage discussing Nozick and Rawls. It must check whether the text supports that qualifier rather than treating any mentioned method as an answer.
sq-5ad268b2d7d075001a429289 · primary - Checking What Rock Ages Establish
The model is asked how old a lava flow is after cooling, using a passage about rocks and their embedded fragments. This exercises distinguishing relative age relationships from the specific age information a question requests.
sq-5a57eee0770dc0001aeefedc · primary - Checking an Organization’s Exact Name
The model is asked which company owns ACB, using a passage about entertainment businesses. It must check the exact organization named rather than silently treating a similar abbreviation as equivalent or relying on outside knowledge.
sq-5ad0206677cf76001a686afa · primary - Checking Reasons for Online Pharmacy Use
The model is asked what is not a reason for choosing an internet pharmacy. This exercises handling a negatively phrased question and checking what the passage actually supports, rather than inventing an alternative to the reasons discussed.
sq-5a6ce6b54eec6b001a80a6a7 · primary - Matching Symbols to Their Expression
The model is asked what two letters represent in a rational-prime expression, using a passage discussing different number types. It must track which expression each definition belongs to instead of transferring a nearby definition without support.
sq-5a89464f3b2508001a72a591 · primary - Separating Legal Provisions from Disagreements
The model is asked what Article 65 did not agree upon in a passage about European competition law. It must distinguish provisions from negotiation disagreements and check that any selected detail belongs to the article named.
sq-5ad14f5a645df0001a2d1713 · primary