Evidence File — OpenAI's New Wet-Lab Benchmark Gains 23 Points

Evidence file · published with the report

Primary documents

Document	URL	Date observed	Use
OpenAI, “Measuring how AI agents handle everyday wet-lab science”	openai.com/index/measuring-how-ai-agents-handle-wet-lab-science	6 November 2026 (retrieved 7 November 2026)	Primary source for every claim in the article
OpenAI, same document at the longer path	openai.com/index/…/measuring-how-ai-agents-handle-everyday-wet-lab-science	Retrieved earlier on 7 November 2026	Cross-check only: identical character and byte count, confirming both paths serve one document. Not cited.

Claim-by-claim verification

#	Claim as published	Status	Basis
1	BioBasicsBench has 100 tasks across five wet-lab areas	Verified	Source text
2	GPT-6 Astra 66.7% vs GPT-5.6 Sol 43.8%; 22.9 points; 52.3% relative	Verified	Source table; arithmetic recomputed
3	Colony PCR: visually correct grouping, reversed experimental identities, downstream purification of the incorrect plasmid	Verified	Source text and worked example
4	Vacuole contrast: lighter regions inspected then excluded from the conclusion	Verified	Source text
5	Colony count: merged neighbours, missed colonies, empty agar counted, approximately 12% overcount	Verified	Source text and annotated panels
6	OpenAI: “Higher benchmark scores do not yet mean models can make reliable judgments in the lab”	Verified quotation	Source text
7	Data types: 31 gels/blots, 14 microscopy, 7 protocol videos; categories overlap	Verified	Source table, including its overlap note
8	Three-stage review, including a Ligo scientist who did not write the task	Verified	Source text
9	Scoring is rubric credit (mean reward), not pass/fail	Verified	Source text
10	37 settings, five attempts per task per setting, 18,500 results	Verified	Source text
11	All seven “xhigh” scores and 95% intervals	Verified	Source table
12	Astra and Sol intervals do not overlap; no formal between-model test is published	Verified as stated; deliberately not called significant	Source table and methods sentence
13	Order is not a generational ladder (GPT-6.1 Sol 62.0%; GPT-6 Luna 38.3% below Sol 43.8%)	Verified	Source table
14	Output tokens per task: 4,704 (Astra) vs 9,099 (GPT-5.6 Sol); not a monotone series	Verified; no cost inferred	Source table; page publishes no pricing or input-token data
15	OpenAI says gains “suggest” multimodal capabilities “may be key”	Verified quotation (hedged)	Source text
16	Positive examples: z-stack recognition; LB Kan50 kanamycin/ampicillin control rejection; overlapping dye windows prompting a call for better controls	Verified	Source worked examples
17	Biosafety content is a single closing sentence; no safeguard announced	Verified	Whole document searched; one occurrence
18	No physical experimentation; interpretation of supplied data only	Verified	Task description and response format
19	No biological incident, near-miss or misuse described	Verified	Whole document searched
20	No dataset, paper, DOI or code repository is linked	Verified for the retrieved page	Full text read to end

Corrections to this report

Revision 1 of this article was held by the owner for corrections. Revision 2 applied the owner's first list; revision 3 completed that list and acted on two independent editorial reviews; revision 4, published here, applies the owner's four final corrections. Each change is traced to an owner instruction or to a reviewer finding:

Removed a sentence implying that a 66.7% mean reward meant one task in three failed on colony selection. Rubric credit is not a task success rate, and the source publishes no per-task pass frequency.
Removed the statement that GPT-6 Astra was cheaper to run. Replaced with the documented output-token counts, and an explicit note that the page publishes no pricing or input-token data.
Reworded the confidence-interval passage: it no longer claims a formal significance test of the Astra-versus-Sol difference, because the source reports per-model intervals only.
Corrected the citation to the canonical URL.
Removed the claim that this was a measurement “for the first time from OpenAI”, which no source in hand supports.
Replaced an unverifiable claim that the token result “has drawn little attention” with a neutral description.
Revision 3: scoped the safeguards finding to this benchmark announcement, and added an explicit statement that no claim is made about OpenAI’s wider safety programmes, which this report did not examine.
Revision 3: removed “newest model” and “its predecessor” after independent review found the source does not establish that ordering; the article now refers to the top-scoring model.
Revision 3: removed internal process notes about tooling availability from publishable copy, and replaced the reviewer’s working note that had been left in the draft.
Revision 3: corrected the summary to credit co-development with Ligo, to state the colony error as an overcount, and to describe the document as an announcement rather than a paper.
Revision 4: the opening now describes GPT-6 Astra as “its highest-scoring model in this evaluation”, and GPT-5.6 Sol as “the earlier GPT-5.6 Sol”. The source establishes neither a release order nor seniority; it establishes a score.
Revision 4: the summary now says Astra “overcounted colonies on one plate by about 12%”, which is what the worked example describes, rather than characterising the model’s count as 12% “wrong” in the abstract.
Revision 4: the summary closes on “this announcement introduces no new safeguard”, stating the finding as an absence of a safeguard being introduced rather than as an absence of anything to offer.
Revision 4: the coverage limitation now records that Research & Development World and Unite.AI both published reports on 6 November 2026, cites both, and keeps the no-exclusivity position. The previously stated inability to compare coverage is removed because that comparison has since been made.

Known limits

Provider self-evaluation: OpenAI evaluated OpenAI models against a benchmark it co-developed with Ligo, a commercial partner whose scientist sat on the internal review. The method is disclosed; no independent replication is published.
The tasks, rubrics and dataset are not linked from the page, so they could not be inspected or re-run for this article.
Scores are rubric-weighted (mean reward), so the headline percentage and any practical failure rate are different quantities. The article does not convert one into the other.
Comparison with other outlets’ coverage of this announcement has now been performed: Research & Development World (Brian Buntz, 6 November 2026) and Unite.AI (Jonas Reeve, 6 November 2026) both reported the benchmark, the 66.7% and 43.8% scores and the mislabelled kanamycin control plate on the day of publication. The article acknowledges that coverage and claims no priority or exclusivity. No comparison with Anthropic, Google DeepMind or independent benchmark work was established, and none is asserted.
The benchmark measures interpretation of supplied data. It establishes nothing about a model designing experiments, obtaining materials, planning protocols or acting unsupervised at a bench.

Editorial disclosure

This report shares a beat with two already-published reports, “Coding Agents Rebuilt Parts of a Withheld Bio Tool for Under $20” (2 November 2026) and “UK Lab Restores Filtered Protein Model Capability for Under $80” (31 October 2026). Both cover work by the AI Security Institute; this report covers an OpenAI benchmark. Different organisations, different experiments, different documents.

A search of all published articles and Evidence Files found no prior coverage of BioBasicsBench. This is a first reading of the announcement for this publication, not a claim that no one else has covered it. Outside this publication, coverage exists: Research & Development World published a report by Brian Buntz on 6 November 2026, and Unite.AI published one by Jonas Reeve the same day. Both were retrieved and read for this revision.

Produced under human editorial authority. Publication requires owner approval.