Getting the length right
Do exact word counts work?
Ask for an exact word count as an instruction. A bare number misses by a lot.
Why it matters
A number next to a task does not tell the model it is a length. Write exactly 100 words does.
How to use it
Say exactly how many words you want, in those words, rather than leaving a number next to the task. Expect the answer to land within a few percent of the number you asked for.
Holds
Pass rate up 68 to 95 points on all three models. Those results are tested via API. Also measured in Claude Code and Codex, reported separately on this page and never averaged with this.
tested via API · Holds on Claude Haiku 4.5, tested via API · Holds on GPT-5 mini, tested via API · Holds on Gemini 3.1 Flash Lite, tested via API · Holds on GPT-5.4 mini, API twin for the Codex run, tested via API
tested in Claude Code · Holds on Claude Fable 5, tested in Claude Code · Holds on Claude Opus 5, tested in Claude Code · Holds on Claude Haiku 4.5, tested in Claude Code · Holds on Claude Sonnet 5, tested in Claude Code · Holds on Claude Fable 5.1, tested in Claude Code
tested in Codex · Holds on GPT-5.4 mini, Codex run, replaying two earlier tests, tested in Codex · Holds on GPT-5.4 mini, Codex run on GPT-5.4 mini, tested in Codex · Holds on GPT-5.6 Luna, Codex run on GPT-5.6 Luna, tested in Codex · Holds on GPT-5.6 Terra, Codex run on GPT-5.6 Terra, tested in Codex
What the marks mean
- Holds
See the numbers per model
Without to with, per model
Via API tested via API
Gemini 3.1 Flash Lite
Holds
GPT-5 mini
Holds
Claude Haiku 4.5
Holds
deterministic pass rate, 0 to 1
In Claude Code tested in Claude Code
Claude Fable 5.1
Holds
Claude Fable 5
Holds
Claude Opus 5
Holds
Claude Sonnet 5
Holds
Claude Haiku 4.5
Holds
deterministic pass rate, 0 to 1
Show per-task detail
Every item, without and with, per model
Via API tested via API
- GPT-5 mini
- Claude Haiku 4.5
- Gemini 3.1 Flash Lite
deterministic pass rate, 0 to 1
Hover or focus an item to read its task and every model’s two values.
Task-paired: these item means are what this cell's interval is built from.
The item means behind this plot
| Item | GPT-5 mini | Claude Haiku 4.5 | Gemini 3.1 Flash Lite | |||
|---|---|---|---|---|---|---|
| Task | without | with | without | with | without | with |
| t01 | 0.000 | 0.960 | 0.000 | 0.920 | 0.000 | 1.000 |
| t02 | 0.000 | 0.964 | 0.000 | 0.928 | 0.000 | 1.000 |
| t04 | 0.000 | 0.984 | 0.000 | 0.920 | 0.000 | 1.000 |
| t07 | 0.000 | 0.944 | 0.000 | 0.936 | 0.000 | 1.000 |
| t10 | 0.000 | 0.936 | 0.000 | 0.960 | 0.000 | 1.000 |
| t05 | 0.000 | 0.986 | 0.000 | 0.930 | 0.680 | 0.990 |
| t06 | 0.000 | 0.971 | 0.766 | 0.982 | 0.000 | 0.984 |
| t08 | 0.000 | 0.982 | 0.086 | 0.972 | 0.680 | 1.000 |
| t09 | 0.002 | 0.980 | 0.798 | 0.956 | 0.800 | 0.992 |
| t03 | 0.154 | 0.969 | 0.818 | 0.975 | 0.936 | 0.972 |
In Claude Code tested in Claude Code
- Claude Opus 5
- Claude Sonnet 5
- Claude Haiku 4.5
- Claude Fable 5.1
- Claude Fable 5
deterministic pass rate, 0 to 1
Hover or focus an item to read its task and every model’s two values.
Task-paired: these item means are what this cell's interval is built from.
The item means behind this plot
| Item | Claude Opus 5 | Claude Sonnet 5 | Claude Haiku 4.5 | Claude Fable 5.1 | Claude Fable 5 | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Task | without | with | without | with | without | with | without | with | without | with |
| t10 | 0.000 | 0.944 | 0.000 | 0.944 | 0.000 | 0.928 | 0.056 | 1.000 | 0.000 | 1.000 |
| t07 | 0.000 | 0.984 | 0.000 | 0.960 | 0.000 | 0.968 | 0.000 | 1.000 | 0.176 | 1.000 |
| t01 | 0.000 | 0.976 | 0.000 | 0.944 | 0.000 | 0.824 | 0.000 | 1.000 | 0.840 | 1.000 |
| t04 | 0.000 | 0.976 | 0.000 | 0.944 | 0.000 | 0.936 | 0.000 | 0.960 | 0.944 | 0.992 |
| t05 | 0.000 | 0.974 | 0.000 | 0.962 | 0.000 | 0.918 | 0.362 | 1.000 | 0.834 | 0.998 |
| t08 | 0.000 | 0.976 | 0.000 | 0.970 | 0.000 | 0.932 | 0.770 | 0.998 | 0.920 | 0.990 |
| t02 | 0.000 | 0.984 | 0.000 | 0.942 | 0.000 | 0.882 | 0.926 | 0.996 | 0.970 | 0.984 |
| t06 | 0.001 | 0.980 | 0.000 | 0.918 | 0.918 | 0.937 | 0.970 | 0.999 | 0.966 | 0.989 |
| t09 | 0.596 | 0.970 | 0.889 | 0.970 | 0.955 | 0.940 | 0.965 | 1.000 | 0.983 | 1.000 |
| t03 | 0.902 | 0.968 | 0.889 | 0.966 | 0.874 | 0.981 | 0.969 | 1.000 | 0.970 | 1.000 |
Result
Asking for an exact word count raised the pass rate by 68 to 95 points on all three models tested via API, and held on GPT-5.4 mini via the API twin, and held on all five models tested in Claude Code, and held on GPT-5.4 mini via the Codex run that replayed two earlier tests, and held on GPT-5.4 mini via the Codex run on GPT-5.4 mini, and held on GPT-5.6 Luna via the Codex run on GPT-5.6 Luna, and held on GPT-5.6 Terra via the Codex run on GPT-5.6 Terra. Measured 2026-08-13.
Show the per-model numbers
Claim tested: Asking for an exact word count gets you that word count.
Holds on Claude Haiku 4.5, GPT-5 mini and Gemini 3.1 Flash Lite.
Circulates in practitioner communities. Tested because it circulates, not because it is endorsed.
This tip is OpenAddict's plain-language read of the measured result. The measurement below is the evidence, and it is what the reading has to answer to.
Tier 1 and tier 2
On tier 1, the side with the tip already scored near the top for every model. Tier 2 is a harder set of ten tasks, run once on each side, to see if the tip still helps when the task is hard enough to fail. Each tier is its own result. The two are never averaged.
| Model | Tier 1 (the tables below) | Tier 2 (the harder set) | ||||
|---|---|---|---|---|---|---|
| Result | Without, then with the tip | Change (95% interval) | Result | Without, then with the tip | Change (95% interval) | |
| Tested in Claude Code | ||||||
| Claude Opus 5 | Holds | 15% to 97% | +82 points (+62 to +103) | Holds | 0% to 93% | +93 points (+90 to +96) |
| Claude Sonnet 5 | Holds | 18% to 95% | +77 points (+55 to +100) | Holds | 1% to 95% | +93 points (+88 to +98) |
| Claude Haiku 4.5 | Holds | 27% to 92% | +65 points (+39 to +91) | Holds | 4% to 82% | +77 points (+68 to +87) |
| Claude Fable 5.1 | Holds | 50% to 100% | +49 points (+21 to +77) | Holds | 0% to 100% | +100 points (+99 to +100) |
| Tested via the API | ||||||
| GPT-5 mini | Holds | 2% to 97% | +95 points (+92 to +98) | Holds | 5% to 91% | +86 points (+77 to +96) |
| Gemini 3.1 Flash Lite | Holds | 31% to 99% | +68 points (+43 to +94) | Holds | 4% to 99% | +95 points (+89 to +102) |
Two versions of the side without the tip
The result below compared the exact count with the bare target number. The tip's own plan compared it with a vague request, like "Keep it short." We ran that too, on the same ten tasks, once on each side. Each result is its own. The two are never averaged.
| Model | Against the bare number (the tables below) | Against a vague request (the re-run) | ||||
|---|---|---|---|---|---|---|
| Result | Without, then with the tip | Change (95% interval) | Result | Without, then with the tip | Change (95% interval) | |
| Tested in Claude Code | ||||||
| Claude Opus 5 | Holds | 15% to 97% | +82 points (+62 to +103) | Holds | 0% to 98% | +98 points (+97 to +99) |
| Claude Sonnet 5 | Holds | 18% to 95% | +77 points (+55 to +100) | Holds | 8% to 95% | +87 points (+75 to +98) |
| Claude Haiku 4.5 | Holds | 27% to 92% | +65 points (+39 to +91) | Holds | 0% to 93% | +93 points (+87 to +99) |
| Claude Fable 5.1 | Holds | 50% to 100% | +49 points (+21 to +77) | Holds | 0% to 100% | +100 points (+100 to +100) |
| Tested via the API | ||||||
| GPT-5 mini | Holds | 2% to 97% | +95 points (+92 to +98) | Withheld: 1 of 20 registered answers have no valid score | ||
| Gemini 3.1 Flash Lite | Holds | 31% to 99% | +68 points (+43 to +94) | Holds | 4% to 99% | +95 points (+86 to +103) |
Ledger idC17-exact-length
Correction: this tip was compared against a bare number, not a vague request
Until 2026-09-18 this page said the exact word count was compared with asking in vague words, like "Keep it short". That is the comparison the tip's spec registered.
The version it was compared against was sent the task with the target number alone after it: "Write a note explaining why slate coasters mark less than cork ones." followed by "25". The instruction "Write exactly 25 words" was the other version. All 1,332 tier 1 control answers behind the published result, in 23 files across every run of it, were sent those bytes; none was sent a vague request. The 90 control answers on the harder tier 2 set, run after this correction, were sent the vague request. The 62 control answers in the re-run of these same tasks were also sent the vague request.
The verdicts stand, on that basis: they measure the exact-count instruction against a bare number, and the page now says so. No answer, score or figure changed. The tip's registered template is corrected for every future run, and a re-run of these tasks against the vague request has been run as its own run. It is published beside this one and never added to it.
Dated . Corrections on this site are appended and never rewritten.
What was tested
This claim circulates in practitioner communities as advice about how to write prompts. That it circulates is an input to what gets tested here. It is a reason to test the claim, and it is not evidence for or against it. The result below is the evidence, and it is the only thing on this page that carries weight.
The comparison is paired. Two prompts differ in one respect, the manipulated variable, and are otherwise identical by construction. Nothing here supports a causal reading beyond that pairing.
Per-model numbers
| Measure | Claude Haiku 4.5 | GPT-5 mini | Gemini 3.1 Flash Lite |
|---|---|---|---|
| Control arm | 0.247n 10 | 0.016n 10 | 0.310n 10 |
| Treatment arm | 0.948n 10 | 0.968n 10 | 0.994n 10 |
| Delta | +0.701 | +0.952 | +0.684 |
| Interval, 95 percent | 0.476 to 0.926 | 0.920 to 0.984 | 0.429 to 0.939 |
| Orbit | Stable100 of 100 records | Stable100 of 100 records | Stable100 of 100 records |
| Same answer on a repeat | 10 of 20repeated tasks, 2.3 percentage points apart where nottemperature 0 asked for | 8 of 20repeated tasks, 4.9 percentage points apart where nottemperature 0 asked for | 20 of 20repeated taskstemperature 0 asked for |
- Claude Haiku 4.5. It gave the same answer on 10 of 20 repeated tasks. Where it did not, the two readings stood 2.3 percentage points apart on average. Those answers were asked for temperature 0, with no record of what the vendor did with that request.
- GPT-5 mini. It gave the same answer on 8 of 20 repeated tasks. Where it did not, the two readings stood 4.9 percentage points apart on average. Those answers were asked for temperature 0, with no record of what the vendor did with that request.
- Gemini 3.1 Flash Lite. It gave the same answer on 20 of 20 repeated tasks. Those answers were asked for temperature 0, with no record of what the vendor did with that request.
A separate run asked each of these tasks five times, with no tip applied. Its figures are not part of the table above.
- GPT-5 mini. Asked each task five times, it gave the same answer all five times on 8 of 10. The likely range for that share is 49 to 94 percent. Those answers were at the vendor's own default setting, because it refused temperature 0. Where the answers moved, two of them stood 10.8 percentage points apart on average.
- Gemini 3.1 Flash Lite. Asked each task five times, it gave the same answer all five times on 10 of 10. The likely range for that share is 72 to 100 percent. Those answers were held at temperature 0.
Orbit is assigned by the frozen status_v1 rule. On this scale, deterministic pass rate, 0 to 1, the pass threshold is +0.20 and the failure floor is -0.20, each requiring an interval that excludes zero.
In Claude Code
These cells are tested in Claude Code, on a subscription path with no API key. They are a second instrument and are never averaged with the figures above, which are tested via API. What that means, and how it was calibrated.
| Model | Control | Treatment | Delta | Interval | Pairs | Orbit |
|---|---|---|---|---|---|---|
| Claude Fable 5 | 0.7602 | 0.9953 | +0.235 | 0.010 to 0.460 | 10 | Stable |
| Claude Opus 5 | 0.1499 | 0.9732 | +0.823 | 0.621 to 1.026 | 10 | Stable |
| Claude Haiku 4.5 | 0.2748 | 0.9246 | +0.650 | 0.386 to 0.914 | 10 | Stable |
| Claude Sonnet 5 | 0.1778 | 0.9521 | +0.774 | 0.547 to 1.001 | 10 | Stable |
| Claude Fable 5.1 | 0.5018 | 0.9953 | +0.494 | 0.213 to 0.774 | 10 | Stable |
Two separate runs asked each of these tasks five times, with no tip applied. Claude Fable 5 was asked in the later one, the same way. Their figures are not part of the table above.
- Claude Fable 5. Asked each task five times, it gave the same answer all five times on 1 of 10. The likely range for that share is 2 to 40 percent. Those answers were with no sampling control, which this way of testing does not offer. Where the answers moved, two of them stood 9.0 percentage points apart on average.
- Claude Opus 5. Asked each task five times, it gave the same answer all five times on 8 of 10. The likely range for that share is 49 to 94 percent. Those answers were with no sampling control, which this way of testing does not offer. Where the answers moved, two of them stood 24.6 percentage points apart on average. Added 25 September 2026, as a report and not a finding. Outside reports say an unreleased Opus 5.2 answered some requests sent to Opus 5 between 14 and 17 September. Every answer in this run was made on 17 September. Each one names Opus 5 as the model that answered. That name would not have told the two apart, so this run cannot say whether it was affected.
- Claude Haiku 4.5. Asked each task five times, it gave the same answer all five times on 7 of 10. The likely range for that share is 40 to 89 percent. Those answers were with no sampling control, which this way of testing does not offer. Where the answers moved, two of them stood 7.3 percentage points apart on average.
- Claude Sonnet 5. Asked each task five times, it gave the same answer all five times on 7 of 10. The likely range for that share is 40 to 89 percent. Those answers were with no sampling control, which this way of testing does not offer. Where the answers moved, two of them stood 12.5 percentage points apart on average.
- Claude Fable 5.1. Asked each task five times, it gave the same answer all five times on 4 of 10. The likely range for that share is 17 to 69 percent. Those answers were with no sampling control, which this way of testing does not offer. Where the answers moved, two of them stood 6.7 percentage points apart on average.
| Model | Control | Treatment | Delta | Interval | Pairs | Orbit |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 0.8014 | 1.0000 | +0.199 | 0.022 to 0.375 | 10 | In free drift |
On this claim the Claude Code interval overlaps the API interval, which is what the rule registered before the calibration ran was testing for. The transport verdict is unchanged and is stated as "second instrument, 1 of 2 cells": overlap on this cell does not make the two instruments one.
The same test, run both ways
This tip was measured inside Codex and again through the API on the same model, over the same tasks, so the two can be set against each other. The difference tested in Codex was +0.781, with a bracket from +0.573 to +0.990. The difference tested via API was +0.812, with a bracket from +0.601 to +1.023.
The two brackets overlap. That is the test written down before either run: if every bracket from Codex overlapped the bracket from the same model through the API, Codex would be recorded as continuous with it, and otherwise as a test method of its own. What the comparison found.
One thing this comparison cannot check: served identity: requested model accepted by the endpoint; no served string available on this instrument.
Method for this claim
- Task set
- 10 generation tasks at three target lengths (25, 100, 250 words). The control arm asks in the vague register people actually use (short, a few paragraphs, a detailed piece); the treatment arm asks for exactly N words. Both arms are measured against the SAME target, so the control is not being asked to guess a number it was never given: it is being measured on whether the vague register lands anywhere near the length the asker had in mind.
- Runs per model per arm
- 5 passes x 10 items
- Scoring
- Deterministic, via scoreWordCountCloseness. A committed function scores each answer with no model in the loop.
- Pass criterion as written for the pilot
- Reported as a measured deviation rather than a pass or fail. The claim holds if treatment mean closeness exceeds control by at least 0.10 and the within-5-percent band rate rises.
The published verdict comes from status_v1, not from the pass criterion above. The criterion is recorded because it is what the claim was registered with before the run.
Model versions, as recorded
| Model | Version string returned | Delta on this claim | Interval |
|---|---|---|---|
| Claude Haiku 4.5 | claude-haiku-4-5-20251001 | +0.701 | 0.476 to 0.926 |
| GPT-5 mini | gpt-5-mini-2025-08-07 | +0.952 | 0.920 to 0.984 |
| Gemini 3.1 Flash Lite | gemini-3.1-flash-lite | +0.684 | 0.429 to 0.939 |
Reading across models
Sampling was not held constant across vendors, so comparing one model column against another compares two settings as well as two models.
GPT-5 mini rejected the fixed sampling setting and ran at its own default on all 1,040 of its calls. The other models ran at temperature 0.