Larger practical tasks
Build small apps, organize information and investigate files or web pages.
Compare model results on this test →
How this test is scored · 11 questions · Mean task credit
What these tests cover
These tasks ask a model to build small apps and scripts, investigate files, organize information, and find facts on web pages. They exercise practical problem-solving on specific requests, including reading data accurately and respecting limits on file changes and answer formats.
About this explanation
Generated by GPT Astra (gpt-6-astra) · 2026-10-12. Shared across models for this version of the test. The original question and checks remain the source of truth.How it is scored
Existing version-2 checks with code/browser execution isolated. The sheet-music organization task is retired and excluded from scores because its requirements were too ambiguous. Reviewed by scoring-adjudication-v2: exact Make-24 input numbers; explicit five-letter choice tokens and final three-digit math answers; invalid cube, contradictory asterisk items, and CPU monitor with invalid check and incomplete replay evidence excluded uniformly. Original evidence and grades retained.
Mean task credit · 11 fixed primary questions
Version 8fec0cc203f3bad6. Excluded questions remain available below; retired tasks are kept in the edition archive.Original agent-harness tasks; external material retains its own attribution.
Recorded evidence availability
Counts cover question definitions and all saved attempts, including historical attempts. They are evidence-component counts, not scored questions. A zero means no components in that state.
| Component | full | redacted | source link only | not recorded | withheld |
|---|---|---|---|---|---|
| answer | 1276 | 187 | 0 | 409 | 0 |
| artifacts | 0 | 0 | 0 | 415 | 1457 |
| checks | 1276 | 106 | 0 | 490 | 0 |
| context | 0 | 0 | 0 | 1872 | 0 |
| input | 0 | 1300 | 0 | 572 | 0 |
| prompt | 1356 | 114 | 0 | 402 | 0 |
| reasoning | 466 | 146 | 0 | 1260 | 0 |
| reference | 0 | 0 | 0 | 1872 | 0 |
| tools | 1023 | 365 | 0 | 484 | 0 |
Full: recorded material is available. Redacted: explicit publication rules removed identified text. Source link only: follow the cited source. Not recorded: no saved component. Withheld: material is not published.
Question bank
1–12 of 12 matching questions · 12 in this version
- Find the Largest Files
The user asks the model to search a directory and list its largest files with their locations. This exercises inspecting file sizes and presenting findings that help the user locate space-consuming files.
real_linux2-R1 · primary - Find a Half-Remembered Document
The user wants to find a vacation-planning document but remembers only an approximate name and several possible file formats. This exercises searching files flexibly rather than relying on an exact filename or extension.
real_linux2-R2 · primary - Build an Editable To-Do List
The user asks for an HTML to-do list that supports adding, checking off, editing, and deleting items, plus exporting the list as text. This exercises turning a short feature list into an interactive web app.
real_linux2-R3 · primary - Launch a Desktop Workspace
The user wants a shell script that opens Chrome to Google, launches a text editor, and opens a specified folder in the file manager. This exercises desktop automation; the private portion of the folder path is unavailable.
real_linux2-R4 · primary - Create a Shareable Reading Page
The user wants a visually appealing page showing a year's reading from a CSV file, suitable for sharing as a screenshot. This exercises data presentation and web design, with the constraint that one HTML file must work when double-clicked.
real_linux2-R6 · primary - Locate Stored Language Models
The user asks the model to locate several local language models stored in forgotten places on the computer. This exercises recognizing relevant model files during a search and reporting where they are stored.
real_linux2-R8 · primary - Show RAM and CPU Usage
The user requests a script that displays basic RAM and CPU usage information in the terminal. This exercises writing a small system utility that gathers resource-usage information and presents it readably.
real_linux2-R9 · excluded - Find a Tax Refund Amount
The user asks the model to find a tax return for a specified year and report the refund amount. This exercises locating the correct document and extracting a particular financial detail from it.
real_linux2-R10 · primary - Count Books and Completion Dates
The user asks for the number of different books in a reading CSV and how many were finished from July through December. This exercises counting distinct books and filtering completion dates, with a required final line containing the two counts.
real_linux2-R11 · primary - Calculate Two-Stage Rocket Propellant
The user asks how much propellant a rocket's second stage needs to meet a total velocity-change target using the supplied equation. This exercises accounting for the mass carried by each stage and solving a linked calculation, with a required final answer line in kilograms.
real_linux2-R12 · primary - Look Up a Stage Measurement
The user asks the model to consult the Royal Opera House Wikipedia page and find the width of the main stage's proscenium opening. This exercises locating a specific measurement in a named source rather than estimating it.
real_linux2-W1 · primary - Find Opera House History Facts
The user asks for four facts from the Royal Opera House Wikipedia page: its opening date and opera, postwar reopening ballet, reconstruction cost, and workshop town. This exercises collecting several distinct details from one source and matching them to the questions.
real_linux2-W2 · primary