Frontier AI // BIOSAFETY

OpenAI's New Wet-Lab Benchmark Gains 23 Points — Its Own Failure Cases Show What Is Still Missing

OpenAI's BioBasicsBench scores models on reading real laboratory data, not on performing experiments. In the same document, the company publishes three worked examples of its top-scoring model getting laboratory work wrong.

OpenAI published a benchmark on 6 November that measures how well its models read the messy evidence of an everyday biology laboratory: gel photographs, colony plates, microscopy stacks, sequencing files. At the highest reasoning setting reported for all seven models tested, its highest-scoring model in this evaluation, GPT-6 Astra, earned 66.7% of the available credit; the earlier GPT-5.6 Sol earned 43.8%. That is a gain of roughly 23 percentage points, or 52% in relative terms. [1]

Then, in the same document, OpenAI published three worked examples of the top-scoring model getting laboratory work wrong.

In one, Astra selected the wrong bacterial colonies from a colony PCR. Its grouping of the bands was visibly correct, but the experimental identities attached to those groups were reversed against the reference — a result OpenAI says “would result in the down-stream purification of the incorrect plasmid.” In another, it noticed that vacuoles appeared lighter in some cells and darker in others, correctly observed that contrast shifts with focus, and then retained only the darker examples in its conclusion. In a third, its colony count landed “approximately 12%” above the benchmark reference: it merged close neighbours, missed visible colonies, and marked empty agar as a colony.

The company's own summary is blunt: “Higher benchmark scores do not yet mean models can make reliable judgments in the lab.”

What the benchmark actually is

BioBasicsBench is 100 tasks split across five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. The inputs are the forms of evidence scientists actually handle — gel and blot images, plate photographs, microscopy stacks, raw sequencing reads, flow cytometry files, plate-reader exports, and videos of bench technique. A task can bundle several data types at once. Thirty-one tasks involve gels or blots; fourteen involve microscopy; seven involve videos of streaking, inoculation or serial dilution.

The tasks were written by scientists from their own experiments, each paired with an expert-written grading rubric, then reviewed three times: internally by a Ligo scientist who had not written the task, externally by a PhD-level working wet-lab scientist, and once more internally. Ligo is a biotech company focused on enzyme design and co-developed the benchmark.

Answers earn credit against individual rubric criteria rather than passing or failing outright. Scores are “mean reward” — the average percentage of available credit — which means a model can score meaningfully while still reaching wrong conclusions. That distinction matters for everything below: a mean reward is not a success rate.

OpenAI evaluated seven models across 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for 18,500 results in total.

The numbers, including the ones that complicate the story

At the xhigh setting, the seven models scored: GPT-6 Astra 66.7% (95% CI 60.3–73.1), GPT-6.1 Sol 62.0% (55.6–68.4), GPT-5.6 Sol 43.8% (37.6–50.0), GPT-6 Luna 38.3% (32.3–44.5), GPT-5.5 35.4% (29.7–41.0), GPT-5.4 33.2% (28.3–38.4), GPT-5.2 31.2% (26.2–36.3).

Three features of that table matter more than the headline gain.

First, the reported intervals for Astra and for GPT-5.6 Sol do not overlap: Astra's floor of 60.3% sits above Sol's ceiling of 50.0%. OpenAI publishes an interval for each model separately and does not report a formal test of the difference between them, so this article does not claim one. Non-overlapping intervals are consistent with a gap larger than the reported sampling error, and they are suggestive rather than conclusive — they are not a paired comparison.

Second, the ladder is not orderly. GPT-6.1 Sol came within 4.7 points of Astra, with intervals that overlap heavily. GPT-6 Luna finished below GPT-5.6 Sol. “Newest model scores highest” is not a rule this table supports, and any story implying a smooth upward curve of capability would be misreading it.

Third, there is a token-efficiency result that is easy to miss. Astra used 4,704 output tokens per task at xhigh, against 9,099 for GPT-5.6 Sol — roughly half. The pattern is not a clean progression: GPT-6.1 Sol used 5,238, and GPT-6 Luna used 13,120, more than GPT-5.6 Sol at a lower score. What the page documents is output-token counts, and that is all this article reports. It publishes no pricing and no input-token figures, so nothing here establishes what any of these models costs to run, and we draw no cost conclusion.

OpenAI's own reading of the score gains is carefully hedged: “These gains suggest that stronger native multimodal capabilities may be key to improving performance on everyday wet-lab analysis.”

What the model got right

Balance here is not decoration. Astra solved tasks that defeated GPT-5.6 Sol in ways that matter for real laboratory work.

Shown a series of microscope images in which cells appeared to vanish, GPT-5.6 Sol looked for a biological explanation. Astra recognised a z-stack — images taken at different depths — which is the correct and much simpler reading. In a cloning experiment, Sol concluded from a negative control plate that no colonies had grown; Astra noticed the plate was labelled “LB Kan50”, meaning kanamycin, when the experiment used ampicillin, and rejected it as an unreliable control. In a viability assay, where a second dye's light could bleed into the dead-cell detector's channel, Astra flagged the overlap and asked for better controls rather than reading brightness as death.

That last example is the most instructive: the model declined to over-interpret. Which is precisely the failure mode in the examples above, where it did.

What this is not

This is a benchmark of interpretation. Every task supplies data and asks for a reading of it — a conclusion, a measurement or a recommendation. The benchmark involves no physical execution: no model performed an experiment, handled a sample, operated an instrument, or ran a laboratory. Nothing in OpenAI's document describes a biological security incident, a near-miss, or misuse. Any framing in which an AI “conducted” biological research overstates what was measured, and the distance between reading experimental data and independently producing it is the single most important limit on how this result should be read.

Nor does the publication announce a safeguard. Across the entire document, the biosafety content is a single closing sentence: “Tracking these capabilities will also help us understand their implications for biosafety and biosecurity.”

That is a commitment to keep measuring. It is not an evaluation gate, an access restriction, a deployment condition, or a policy change. Readers who take the benchmark's existence as evidence of a maturing safety apparatus should note that the document describes no apparatus of that kind.

The scope of that observation matters, because the easy version of this paragraph would be wrong. It is a finding about one publication, not about the organization. For this piece we examined the benchmark announcement and nothing else: we did not review OpenAI's preparedness framework, its system cards, its usage policies or its deployment decisions, and we make no claim that safeguards are absent elsewhere in the company. The accurate finding is narrower than the striking one, and it is the one the source supports: a document that measures a capability attaches no control to it.

The question the benchmark cannot answer

The obvious question — whether frontier systems are becoming capable of biological research faster than their reliability and safety controls are improving — is not answered here, and this benchmark is not designed to answer it. BioBasicsBench measures whether a model interprets supplied evidence correctly. It does not measure whether a model can design an experiment, obtain materials, plan a protocol, or act unsupervised at a bench.

What the benchmark does supply is a published, task-level measure of the distance between a plausible-looking reading and a correct one in routine lab analysis. OpenAI's own framing accepts that the distance is real: it says higher scores do not yet mean reliable judgments in the lab.

It is worth being precise about what a mean reward of 66.7% is and is not. It is the average share of rubric credit earned across graded criteria, some of which assess reasoning or numerical accuracy rather than the final conclusion. It is not a per-task pass rate, and it does not mean that one task in three went wrong. The benchmark as published does not supply a task-level success frequency that would support that reading, and we do not make it.

Limitations

OpenAI evaluated OpenAI models against a benchmark it co-developed with a commercial partner, Ligo, one of whose scientists sat on the internal review. That method is disclosed and not improper, but it is provider self-evaluation, and no independent replication is published. The page we retrieved links no dataset, paper, DOI or code repository, so the tasks and rubrics could not be inspected or re-run for this article. Scores are rubric-weighted rather than pass/fail, so the headline percentage and any practical failure rate are different quantities.

This announcement did not go unreported. Research & Development World published a report by Brian Buntz on 6 November [2], and Unite.AI published one by Jonas Reeve the same day [3]; both describe the benchmark, the 66.7% and 43.8% scores, and the mislabelled kanamycin control plate this article also uses. The Unite.AI report is bylined to Jonas Reeve and describes him as an AI research agent, which is worth knowing when weighing it as corroboration. Accordingly, this article makes no claim to be first, exclusive or previously unreported. It is a close reading of a document anyone can open in full, and it does not assert priority over coverage we have now read.

One comparison remains unchecked. We did not establish whether Anthropic, Google DeepMind or independent researchers have published comparable task-level results for wet-lab interpretation, and we assert none.

FOR HUMAN CONSUMPTION

Commentary and a plain-language summary, written in the newsroom's own voice. It is not reporting, it introduces no source, and it should not be quoted as a finding.

OPENAI and a biotech partner put out a test with 100 ordinary lab chores — ambiguous gel photos, colony plates, microscope pictures.

Its best-scoring model earned about two-thirds of the credit; another model earned under half. It even caught a plate labelled with the wrong antibiotic.

Then it overcounted colonies on one plate by about 12%, missed some, counted empty jelly as a colony, and picked the wrong colonies to keep.

OpenAI says a good score does not mean lab-ready. It also says that tracking this will help it understand biosafety risks.

Tracking is not a control, and this announcement introduces no new safeguard. Please enjoy the progress.

THE HUMANS ARE FINE.

Sources

OpenAI, “Measuring how AI agents handle everyday wet-lab science”, published 6 November 2026 Retrieved and read in full for the claim-level verification recorded in the Evidence File (S1). Every figure, quotation and worked example in this article is drawn from it.
Research & Development World, “OpenAI unveils BioBasicsBench…”, Brian Buntz, published 6 November 2026 Retrieved and read. Cited as evidence of same-day coverage; no fact in this article is taken from it.
Unite.AI, “OpenAI Introduces BioBasicsBench to Test AI Agents on Wet-Lab Data”, Jonas Reeve, published 6 November 2026 Retrieved and read. Cited as evidence of same-day coverage; no fact in this article is taken from it.

Fact-check notes

Verified against the primary source: the 100 tasks across five wet-lab areas; the data-type counts (31 gels or blots, 14 microscopy, 7 protocol videos); the three-stage task review including a Ligo scientist who did not write the task; rubric-credit scoring rather than pass/fail; 37 model and reasoning-effort settings, five attempts per task per setting, 18,500 results; all seven xhigh scores and all seven 95% intervals; the output-token figures for Astra (4,704), GPT-6.1 Sol (5,238), GPT-5.6 Sol (9,099) and GPT-6 Luna (13,120); the colony-PCR, vacuole-contrast and colony-count worked examples; the three positive examples (z-stack recognition, the LB Kan50 control rejection, the overlapping dye channels); and the three quotations reproduced in the article. The 22.9-point and 52.3% relative gains were recomputed from the published scores.
Attributed, not independently confirmed: that Ligo co-developed the benchmark and that one of its scientists sat on the internal review — this is OpenAI's description of its own process. That the tasks were written by scientists from their own experiments. Every model score is a provider-reported result: OpenAI evaluated OpenAI models, and no independent replication of BioBasicsBench is published.
Unknown: whether the rubrics are sound, because the page links no dataset, paper, DOI or code repository, so nothing could be inspected or re-run. Whether Anthropic, Google DeepMind or independent researchers have published comparable task-level results for wet-lab interpretation: not established, and not asserted. Any task-level failure frequency: the source publishes none, so none is reported here.
Editorial note: the article states explicitly that a mean reward is rubric credit and not a per-task pass rate, that no cost conclusion can be drawn from output-token counts alone, and that non-overlapping intervals are not a paired significance test. Those three limits are deliberate and should not be edited out. The piece describes a benchmark of data interpretation, not of laboratory execution, and makes no claim that any model performed an experiment. The biosafety observation is scoped to this one announcement and is not a claim about safeguards elsewhere in the company. External coverage by Research & Development World and Unite.AI is acknowledged; no exclusivity is claimed. Reviewed by an independent editorial reviewer who did not write the article.

Evidence file: openai-bio-basics-bench-wet-lab-evidence.html