Logic and number puzzles
Solve newly generated puzzles using arithmetic, deduction and planning.
Compare model results on this test →
How this test is scored · 34 questions · Mean puzzle credit
What these tests cover
These puzzles ask the model to follow explicit rules, combine clues, calculate exact probabilities, and plan routes. The tasks cover truth-teller logic, house arrangements, course prerequisites, arithmetic expressions, and grid navigation, showing how the model handles these specific kinds of structured reasoning.
About this explanation
Generated by GPT Astra (gpt-6-astra) · 2026-10-12. Shared across models for this version of the test. The original question and checks remain the source of truth.How it is scored
Pinned Reasoning Gym 0.1.25 score_answer, including its partial credit. Fixed seeds/configurations and generated answers saved before any model runs; answer tags extracted uniformly. A score of 1 is a full pass. Reviewed by scoring-adjudication-v2: exact Make-24 input numbers; explicit five-letter choice tokens and final three-digit math answers; invalid cube, contradictory asterisk items, and CPU monitor with invalid check and incomplete replay evidence excluded uniformly. Original evidence and grades retained.
Mean puzzle credit · 34 fixed primary questions
Version 0bafa27052b08d41. Excluded questions remain available below; retired tasks are kept in the edition archive.Reasoning Gym contributors; Apache-2.0; seeded generated subset. · Original source · Publication terms
Recorded evidence availability
Counts cover question definitions and all saved attempts, including historical attempts. They are evidence-component counts, not scored questions. A zero means no components in that state.
| Component | full | redacted | source link only | not recorded | withheld |
|---|---|---|---|---|---|
| answer | 4495 | 0 | 0 | 1242 | 0 |
| artifacts | 0 | 0 | 0 | 5737 | 0 |
| checks | 4531 | 0 | 0 | 1206 | 0 |
| context | 0 | 0 | 0 | 5737 | 0 |
| input | 4105 | 0 | 0 | 1632 | 0 |
| prompt | 4534 | 0 | 0 | 1203 | 0 |
| reasoning | 2041 | 0 | 0 | 3696 | 0 |
| reference | 36 | 0 | 0 | 5701 | 0 |
| tools | 0 | 0 | 0 | 5737 | 0 |
Full: recorded material is available. Redacted: explicit publication rules removed identified text. Source link only: follow the cited source. Not recorded: no saved component. Withheld: material is not published.
Question bank
1–20 of 36 matching questions · 36 in this version
- Three Heroes and Villains
Identify which of three island inhabitants are truth-telling heroes and which are lying villains from their statements about one another. This exercises consistency checking across claims joined by “and” and “or,” with a label required for each person.
rg-knights_knaves-0-0 · primary - Three Saints and Sinners
Classify three inhabitants as truth-telling saints or lying sinners using their linked statements. This exercises reasoning about “if” and “if and only if,” while keeping each speaker’s assigned role consistent with what they say.
rg-knights_knaves-0-1 · primary - Four Heroes with Linked Claims
Determine the roles of four inhabitants whose statements combine direct claims, alternatives, and conditional relationships. The model must find a consistent hero-or-villain assignment and report a role for every person.
rg-knights_knaves-1-0 · primary - Four Saints with Nested Claims
Assign saint or sinner roles to four people using a mix of direct accusations and nested logical statements. This tests whether the model can connect several claims without losing the rule that saints tell the truth and sinners lie.
rg-knights_knaves-1-1 · primary - Five Heroes with Nested Logic
Classify five inhabitants as heroes or villains from statements containing nested “if and only if,” negation, and combined claims. This exercises tracking several interdependent truth conditions while producing a complete list of roles.
rg-knights_knaves-2-0 · primary - Five Sages and Fools
Determine which of five inhabitants are truthful sages and which are lying fools. Their statements mix accusations, conditions, and negation, requiring the model to check that all assigned roles and statements fit together.
rg-knights_knaves-2-1 · primary - Three Houses with Drink Clues
Use clues about three residents, their drinks, and their smoothies to identify who occupies the first house. This exercises combining neighbor relationships with fixed positions and unique attributes, returning only the requested name.
rg-zebra_puzzles-0-0 · primary - Three Houses with Pet Clues
Identify the first house’s resident from clues linking three people to house positions, pets, and nationalities. This exercises elimination using unique assignments and direct facts, with only a single name requested.
rg-zebra_puzzles-0-1 · primary - Four Houses with Lunch Clues
Work out who lives in the first of four houses using clues about colors, lunches, and pets. This exercises combining attribute matches with neighbor and immediately-left relationships while keeping every assignment unique.
rg-zebra_puzzles-1-0 · primary - Four Houses with Flower Clues
Identify the first resident in a four-house row using flower, drink, and smoothie preferences. The model must combine linked preferences, fixed positions, and neighboring-house clues, then return only the resident’s name.
rg-zebra_puzzles-1-1 · primary - Five Houses with Phone Clues
Find the first house’s resident among five people using clues about phones, pets, colors, and nationalities. This exercises keeping several sets of unique attributes aligned while applying fixed-position and immediately-left constraints.
rg-zebra_puzzles-2-0 · primary - Five Houses with Family Clues
Identify who occupies the first of five houses from clues linking residents, nationalities, children, cigars, and pets. This exercises multi-step deduction across several attribute categories, including distinguishing residents’ names from children’s names.
rg-zebra_puzzles-2-1 · primary - Check Prerequisites for 11 Courses
Decide whether all 11 courses can be completed under the listed prerequisites, returning True or False. This exercises following dependency chains and checking whether any circular requirement prevents a valid course order.
rg-course_schedule-0-0 · primary - Check Prerequisites for Eight Courses
Determine whether eight courses can all be taken while respecting the supplied prerequisite pairs. The task exercises interpreting which course must come first and checking for circular dependencies, rather than producing a full schedule.
rg-course_schedule-0-1 · primary - Check Prerequisites for 17 Courses
Check whether the prerequisite relationships among 17 courses permit completing them all. This exercises tracing indirect dependencies and detecting circular requirements, with a True-or-False answer rather than a proposed course sequence.
rg-course_schedule-1-0 · primary - Check Prerequisites for 18 Courses
Decide whether all 18 courses can be completed given a list of prerequisite pairs. This exercises keeping dependency directions straight and checking whether linked requirements form a loop that blocks a valid ordering.
rg-course_schedule-1-1 · primary - Check Prerequisites for 26 Courses
Evaluate whether the prerequisite list for 26 courses allows every course to be completed. This exercises tracking many interconnected requirements and checking for circular dependencies, then summarizing the conclusion as True or False.
rg-course_schedule-2-0 · primary - Check Prerequisites for 25 Courses
Determine whether 25 courses can be arranged in an order that respects all supplied prerequisites. This exercises reasoning over direct and indirect dependencies; the requested output is only whether such an order is possible.
rg-course_schedule-2-1 · primary - Make 24 with a Repeated Pair
Build an expression that makes 24 from four supplied numbers, including a repeated value, using each occurrence once. This exercises arithmetic search and number tracking, with only addition, subtraction, multiplication, and division allowed and an expression-only answer required.
rg-puzzle24-0-0 · primary - Make 24 with Three Repeats
Make 24 from four supplied numbers, three of which share the same value, using each occurrence once. This exercises choosing arithmetic operations and grouping while respecting the available numbers and returning an expression rather than an equation.
rg-puzzle24-0-1 · primary