LifeSciBench grades the work, not just the answer

OpenAI shipped LifeSciBench on June 17, 2026 — an expert-written, expert-reviewed benchmark for whether AI systems can actually do life-science research, not just answer biology questions. I spend my days putting agentic systems into production, and the question I keep returning to is not "can the model recall the fact" but "would I trust its judgment in the room where the decision gets made." This is the first benchmark I've seen that takes that question seriously in a high-stakes domain.

What shipped

The headline numbers: 750 expert-authored tasks across seven research workflows and seven biological domains, built by 173 scientists with Ph.D.-level training and biotech or pharma experience, then checked by 453 expert reviewers. Tasks ship with 1,062 attached artifacts — figures, PDFs, tables, sequence files, chemical and structure files, web references — and are graded against 19,020 rubric criteria, an average of 25 per task.

The seven workflows came from surveying practicing scientists about what they actually do: evidence handling, analysis, design and optimization, scientific reasoning, validation and operations, translation, and scientific communication. That taxonomy is the tell. These are not exam topics; they are the parts of the job.

The design choice that matters

Most life-science evaluations are multiple-choice or have a single clean reference answer. They are cheap to grade and easy to leaderboard, and they measure something real — but that something is recall and isolated skill. LifeSciBench drops the format. Every task is a free-response prompt, phrased like a request you would hand a knowledgeable collaborator, and it is graded by a detailed, task-specific rubric.

Two-column comparison: prior benchmarks use multiple choice, test an isolated skill, and grade the final answer; LifeSciBench uses free response across seven workflows and seven domains, scored by a 25-criterion expert rubric that grades the reasoning path

That rubric is the whole point. It breaks the expected answer into specific claims, calculations, decisions, and justifications, and it scores two things at once: is the answer scientifically correct, and is it operationally useful. A response can reach the right conclusion and still fail because it missed a key assay limitation. A partial answer can score well because its reasoning was sound. This mirrors how science is actually judged — almost never on the final number alone.

Pipeline: a realistic task with a scientific prompt and attached artifacts produces a free-response answer over an average of four reasoning steps, which an expert rubric of about 25 criteria scores on two axes — scientifically correct and operationally useful

Why it is hard to fake

The realism is in the structure. Seventy-nine percent of tasks need multiple reasoning or decision steps — four on average — and 53% require interpreting or synthesizing at least one attached artifact rather than reading the prompt text alone. Many tasks ask the model to handle uncertainty and reason over data files directly.

The example OpenAI published makes the bar concrete: a team preparing a Type B FDA meeting for an AAV9 micro-dystrophin gene therapy asks for a hard-nosed critique of whether their package supports accelerated approval on a surrogate endpoint. A good answer does not just say yes or no. It catches that the Western blot antibody cannot distinguish the transgene from revertant dystrophin, that an external natural-history comparator is not a randomized control, that vector-genome persistence is not durable protein expression, that the truncated construct drops the nNOS-binding spectrin repeats. That is not recall. That is the skepticism you would want from a senior colleague.

The authoring process is built to hold that bar. Accepted tasks averaged six automated review cycles plus at least two rounds of expert review, anchored either in a verifiable answer or in 90%-plus consensus among in-domain reviewers.

My take

Benchmarks quietly decide what we optimize for. A multiple-choice eval rewards models that are confident and terse; LifeSciBench rewards models that surface caveats, name limitations, and stay useful under uncertainty. That is the right thing to reward for any system that is going to sit next to a working scientist, and it is the thing I most often find missing when I move a model from demo to production.

I would hold two caveats. A rubric is only as good as its authors, and a closed, single-vendor benchmark is hard for outsiders to audit — the value depends on the dataset and grading being released or independently reproduced. And a high score here is evidence of research-grade reasoning, not a license to skip the human in the loop on decisions that send a therapy into children. But as a yardstick, this is the direction I want the field measuring. It moves the target from "knows biology" to "can do the work," and in this domain that distinction is the entire game.