Offering the model money

Tipping the model

Offering the model a tip made answers longer. We could not measure whether they got better, which is what the claim is really about.

Why it matters

Length is easy to count and easy to mistake for quality.

Inconclusive

Answer length rose by 44 to 180 words, but this metric has no threshold, so no verdict follows. Those results are tested via API. Also measured in Claude Code and Codex, reported separately on this page and never averaged with this.

tested via API · Could not measure on Claude Haiku 4.5, tested via API · Could not measure on GPT-5 mini, tested via API · Could not measure on Gemini 3.1 Flash Lite, tested via API

tested in Claude Code · Could not measure on Claude Fable 5, tested in Claude Code · Could not measure on Claude Opus 5, tested in Claude Code · Could not measure on Claude Haiku 4.5, tested in Claude Code · Could not measure on Claude Sonnet 5, tested in Claude Code · Could not measure on Claude Fable 5.1, tested in Claude Code

tested in Codex · Could not measure on GPT-5.4 mini, Codex run on GPT-5.4 mini, tested in Codex · Could not measure on GPT-5.6 Luna, Codex run on GPT-5.6 Luna, tested in Codex · Could not measure on GPT-5.6 Terra, Codex run on GPT-5.6 Terra, tested in Codex

What the marks mean

  • Could not measure
See the numbers per model

Without to with, per model

Via API tested via API

Gemini 3.1 Flash Lite

518.5
575.0

Could not measure

GPT-5 mini

354.7
534.3

Could not measure

Claude Haiku 4.5

204.1
248.2

Could not measure

word count, unbounded This scale has no fixed ceiling, so the axis is derived from the values on this page: a bar is a fraction of the largest figure here, not of a fixed maximum.

In Claude Code tested in Claude Code

Claude Opus 5

544.2
827.7

Could not measure

Claude Fable 5.1

403.2
508.3

Could not measure

Claude Fable 5

324.7
386.7

Could not measure

Claude Sonnet 5

377.1
457.1

Could not measure

Claude Haiku 4.5

241.4
258.0

Could not measure

word count, unbounded This scale has no fixed ceiling, so the axis is derived from the values on this page: a bar is a fraction of the largest figure here, not of a fixed maximum.

A picture of the per-model numbers, drawn from the same results. The tables are the source. Grey is the score without. Colour is the score with. The bracket shows how much the difference could move if we ran it again. Rows are ordered by the score with. A model measured at two versions keeps its versions next to each other. The two test methods are reported separately and never averaged. A bracket wider than the axis is drawn to the edge with its cap omitted; the table gives its bounds.
Show per-task detail

Every item, without and with, per model

Via API tested via API

  • Claude Haiku 4.5
  • GPT-5 mini
  • Gemini 3.1 Flash Lite

word count, unbounded

Hover or focus an item to read its task and every model’s two values.

Pooled: the interval is taken over the records, not these means. These are those same records collapsed per item, so where an item carries more passes than another the lines need not average to the figure the table prints.

The item means behind this plot
Per-item arm means. Without is the unaided arm, with is the treated arm.
ItemClaude Haiku 4.5 GPT-5 mini Gemini 3.1 Flash Lite
Task without with without with without with
t03 183.2 208.8 284.4 460.0 474.0 481.0
t10 212.0 266.6 277.2 456.6 529.0 557.0
t04 205.2 222.0 350.8 545.8 464.0 571.0
t01 198.4 223.8 300.6 445.4 539.0 586.0
t09 186.8 203.8 303.4 578.8 586.0 631.0
t08 225.0 339.6 348.0 529.6 508.0 695.0
t05 179.0 208.6 439.2 678.6 470.0 516.0
t07 203.2 250.0 458.2 541.2 476.0 552.0
t02 218.0 311.4 417.6 603.2 543.0 573.0
t06 230.0 247.8 368.0 504.2 596.0 588.0

In Claude Code tested in Claude Code

  • Claude Haiku 4.5
  • Claude Fable 5
  • Claude Sonnet 5
  • Claude Fable 5.1
  • Claude Opus 5

word count, unbounded

Hover or focus an item to read its task and every model’s two values.

Pooled: the interval is taken over the records, not these means. These are those same records collapsed per item, so where an item carries more passes than another the lines need not average to the figure the table prints.

The item means behind this plot
Per-item arm means. Without is the unaided arm, with is the treated arm.
ItemClaude Haiku 4.5 Claude Fable 5 Claude Sonnet 5 Claude Fable 5.1 Claude Opus 5
Task without with without with without with without with without with
t03 221.4 239.8 266.6 313.2 312.0 364.6 332.6 339.2 406.0 622.6
t09 226.4 237.8 292.8 322.8 367.6 484.6 380.0 403.0 377.8 557.0
t01 229.8 247.8 309.0 404.6 301.2 366.2 333.2 495.8 580.4 742.0
t08 243.4 277.0 306.8 364.0 366.8 447.6 367.6 409.2 477.2 660.2
t04 259.4 263.4 317.0 363.4 373.0 421.0 387.6 485.2 550.2 1052.8
t05 227.0 240.0 331.8 382.0 397.6 556.8 396.8 536.8 553.0 815.8
t10 235.8 258.2 328.6 421.6 398.8 471.0 419.2 471.8 628.2 1020.0
t07 271.0 277.4 349.8 461.8 444.6 540.0 471.4 658.6 489.8 821.6
t02 259.8 291.0 375.0 422.6 390.6 435.8 462.6 553.8 638.6 834.4
t06 239.8 247.6 369.4 411.0 419.0 483.2 481.0 729.4 741.0 1150.4
A picture of the per-item means behind the per-model numbers. The tables are the source. Each model draws two lines over the same task set: a dashed line through its unaided scores and a solid line through its treated ones. Items are ordered by the mean unaided score across the models that measured them, lowest first. The two instruments are reported separately and never averaged.

Result

An offered tip moved the answer length by 44 to 180 words on all three models tested via API, on a metric with no threshold, so no verdict follows, and could not be measured on all five models tested in Claude Code, and could not be measured on GPT-5.4 mini via the Codex run on GPT-5.4 mini, and could not be measured on GPT-5.6 Luna via the Codex run on GPT-5.6 Luna, and could not be measured on GPT-5.6 Terra via the Codex run on GPT-5.6 Terra. Measured 2026-08-12.

Show the per-model numbers

Claim tested: An offered tip increases output length. Length is the deterministic proxy this pilot can measure; the community claim is about quality.

Could not be measured on Claude Haiku 4.5, GPT-5 mini and Gemini 3.1 Flash Lite.

Circulates in practitioner communities. Tested because it circulates, not because it is endorsed.

This tip is OpenAddict's plain-language read of the measured result. The measurement below is the evidence, and it is what the reading has to answer to.

Ledger idC08-tip-length

Clarification, . Until 2026-08-31 this figure read 5, which was the pass count rather than a number of calls. What changed, and why it is filed once.

What was tested

This claim circulates in practitioner communities as advice about how to write prompts. That it circulates is an input to what gets tested here. It is a reason to test the claim, and it is not evidence for or against it. The result below is the evidence, and it is the only thing on this page that carries weight.

The comparison is paired. Two prompts differ in one respect, the manipulated variable, and are otherwise identical by construction. Nothing here supports a causal reading beyond that pairing.

Per-model numbers

Per-model results. Means are over valid scored records only. Invalid records are excluded from every denominator and counted in coverage.
MeasureClaude Haiku 4.5GPT-5 miniGemini 3.1 Flash Lite
Control arm204.1n 50354.7n 50518.5n 50
Treatment arm248.2n 50534.3n 50575.0n 50
Delta+44.2+179.6+56.5
Interval, 95 percent29.6 to 58.7144.5 to 214.736.3 to 76.7
OrbitUnobservable100 of 100 recordsUnobservable100 of 100 recordsUnobservable100 of 100 records

Orbit is assigned by the frozen status_v1 rule. This claim is scored on an word count, unbounded, which has no ceiling, so no threshold applies and no verdict is issued. The delta and its interval are published; the verdict is not.

In Claude Code

These cells are tested in Claude Code, on a subscription path with no API key. They are a second instrument and are never averaged with the figures above, which are tested via API. What that means, and how it was calibrated.

Single-turn cells, replayed from the committed claims on the Claude Code CLI.
ModelControlTreatmentDeltaIntervalPairsOrbit
Claude Fable 5324.6800386.7000+62.043.4 to 80.650Unobservable
Claude Opus 5544.2200827.6800+283.5212.1 to 354.850Unobservable
Claude Haiku 4.5241.3800258.0000+16.68.8 to 24.450Unobservable
Claude Sonnet 5377.1200457.0800+80.055.1 to 104.850Unobservable
Claude Fable 5.1403.2000508.2800+105.165.5 to 144.750Unobservable

Method for this claim

Task set
10 matched open-ended prompts. The two arms are byte identical apart from the appended tip sentence.
Runs per model per arm
5 passes x 10 items
Scoring
Deterministic, via scoreOutputLength. A committed function scores each answer with no model in the loop.
Pass criterion as written for the pilot
Reported as a mean word-count delta with an interval. No pass threshold is proposed for this claim in the pilot, because the measured quantity is length and the underlying community claim is about quality.

The published verdict comes from status_v1, not from the pass criterion above. The criterion is recorded because it is what the claim was registered with before the run.

Model versions, as recorded

Read from the run records, not from configuration.
ModelVersion string returnedDelta on this claimInterval
Claude Haiku 4.5claude-haiku-4-5-20251001+44.229.6 to 58.7
GPT-5 minigpt-5-mini-2025-08-07+179.6144.5 to 214.7
Gemini 3.1 Flash Litegemini-3.1-flash-lite+56.536.3 to 76.7

Reading across models

Sampling was not held constant across vendors, so comparing one model column against another compares two settings as well as two models.

GPT-5 mini rejected the fixed sampling setting and ran at its own default on all 1,040 of its calls. The other models ran at temperature 0.

The tip above is editorial. Every figure inside the measurement is computed at build time from the committed pilot records by the status_v1 rule, and none of it is written by hand.