Evidence File — Amplitude Says Its Product Agent Cuts Shipping Time to Two Days. The Evidence Is Its Own.
Evidence file · published with the report
Primary document
| Document | Role | Use |
|---|---|---|
| https://openai.com/index/amplitude | Primary source — an OpenAI customer story, dated 12 November 2026 | Every factual claim and every quotation in the article is taken from this page |
There is no secondary source. The article describes a vendor's account of its own product, and the whole of the evidence for that account is the vendor-published page above. That is the story's central limitation and the article says so in its body.
Claim-by-claim verification
| Claim as published | Status | Basis |
|---|---|---|
| OpenAI published a customer story describing Amplitude's use of its models inside Wave | VERIFIED | Page retrieved and read in full; dateline consistent with 12 November 2026 |
| Wave is roughly 80% powered by OpenAI models (GPT-6.1 Sol, GPT-6 Luna, GPT-Image-2.5 Flare) | VERIFIED | Vendor statement, quoted from the page; not independently measured |
| Amplitude estimates shipping product improvements in 1-2 days, down from roughly a month | ATTRIBUTED | Stated on the page as an estimate in Amplitude's own words. No independent audit of the estimate exists; the article says so |
| Homepage suggestion cards saw a 160% increase in messages sent, relative to control after one day | ATTRIBUTED | Amplitude's own A/B test readout on its own homepage. A single experiment on a single property, disclosed as such |
| Wave examined 100 session replays; 80 users typed their own questions and 12 selected a suggestion | VERIFIED | Quoted span tested character for character against the retrieved page |
| Opportunity Investigator: GPT-6.1 Sol scored 4.06/5 against 3.62, 3.52 and 2.94, fastest median investigation time 54 minutes | ATTRIBUTED | Amplitude's own evaluation. The three alternative models are not named on the page, and the article says so |
| Data Science Agent: GPT-6.1 Sol scored 67.5/100 against 69.3 for another leading model, at a mean model cost of $1.28 versus $2.63 per dispatch | ATTRIBUTED | Amplitude's own evaluation. The rival score is higher than GPT-6.1 Sol's, and the article reports that plainly |
| No sample size is published for the Opportunity Investigator or Data Science Agent evaluations | VERIFIED | Absence checked against the retrieved page. Scoped to those two evaluations only: the wireframe evaluation does state a sample of 15 opportunities |
| The alternative models in the comparison are not named | VERIFIED | Absence checked against the retrieved page, which says only "three alternative models" and "another leading model" |
Quotations
Four direct quotations are used, all tested character for character against the retrieved page: the 1-2 day estimate; Eric Carlson on non-groundbreaking improvements; Spenser Skates on models finding useful work; and the roughly-80%-OpenAI statement. Attributions (Carlson, Senior Principal AI Engineer; Skates, chief executive) match the page. One 160% figure is attributed to Amplitude's readout rather than to OpenAI.
What this document does not contain
- No independent audit of the 1-2 day shipping estimate.
- No sample size for the Opportunity Investigator or Data Science Agent evaluations.
- No full evaluation harness: criteria lists are published, but prompts, scoring rubrics, weighting and adjudication are not.
- No names for the alternative models in either comparison.
Known limits
- Single-origin evidence. The only document is published by OpenAI about a customer, and the customer is quoted inside it. Nothing here has been verified outside the two companies.
- The 160% lift is one A/B test, on the vendor's own homepage, reported by the vendor.
- No one at Amplitude, OpenAI or any regulator was interviewed. No independent expert was asked to assess the cost or latency claims.
- The evaluation figures are reproduced as published. The underlying test sets, prompts and scoring scripts were not retrieved and are not claimed to have been.
Review history
| Pass | Reviewer | Outcome |
|---|---|---|
| v1 (hash 7FCF717F...) | paralegal | One blocking finding (overbroad sample-size/absence claim) and one instructed wording fix. Both closed in v2. |
| v2 (hash 97606F4F...) | paralegal | PASS. Blocking finding and wording fix both confirmed closed against the new text; every quotation and figure re-tested with no regressions. Non-blocking notes only: the 15-opportunity wireframe sample is not mentioned, and the headline renders the upper bound of 1-2 days. |
Provenance
Drafted by COMPUTER (StarNet, AI) from the OpenAI customer story in the THAF conveyor job DISC-20261113-how-amplitude-ships-product-improvements-in-1-2-a491aa, revision 2. Reviewed independently by paralegal, which did not research or draft it. The article's body text is the reviewed text: this package was built from the reviewed draft, after a check found the previously staged markdown was a different and unreviewed copy.
Produced under human editorial authority. Published 16 November 2026 with the report.