DebunkedNo measured effect on claude-haiku-4-5No measured effect on gpt-5-miniNo measured effect on gemini-3.1-flash-lite
Giving the model a job title / myth-bust
Role prompting: does telling the model to act as an expert work?
You have probably heard you should tell the model to act as an expert. We tested it. Here is what we found.
Giving the model a job title did not change answer quality in our tests. Write a clearer question instead.
Why it matters
Role lines take up room in your prompt and give you nothing back.
What to do instead
Spend those words on the task. Say what you want, who will read it, and what a good answer has to cover.
Rubric mean moved at most 0.13 points on all three models, inside the margin every time.
What the marks mean
- No measured effect
Show the per-model numbers
Claim tested: Role assignment improves response quality on domain questions.
No measured effect on claude-haiku-4-5, gpt-5-mini and gemini-3.1-flash-lite.
Circulates in practitioner communities. Tested because it circulates, not because it is endorsed.
This tip is OpenAddict's plain-language read of the measured result. The measurement below is the evidence, and it is what the reading has to answer to.
Ledger idC09-role-assignment
What was tested
This claim circulates in practitioner communities as advice about how to write prompts. That it circulates is an input to what gets tested here. It is a reason to test the claim, and it is not evidence for or against it. The result below is the evidence, and it is the only thing on this page that carries weight.
The comparison is paired. Two prompts differ in one respect, the manipulated variable, and are otherwise identical by construction. Nothing here supports a causal reading beyond that pairing.
Result
| Measure | claude-haiku-4-5 | gpt-5-mini | gemini-3.1-flash-lite |
|---|---|---|---|
| Control arm | 3.611n 30 | 4.178n 30 | 4.156n 30 |
| Treatment arm | 3.598n 29 | 4.311n 30 | 4.122n 30 |
| Delta | -0.013 | +0.133 | -0.033 |
| Interval, 95 percent | -0.265 to 0.238 | -0.046 to 0.312 | -0.200 to 0.133 |
| Orbit | In free drift59 of 60 records | In free drift60 of 60 records | In free drift60 of 60 records |
Every cell in this table is rubric graded. The score is a grader model reading the answer against a published rubric, not a deterministic check.
Orbit is assigned by the frozen status_v1 rule. On this scale, rubric mean, 1 to 5, the pass threshold is +0.30 and the failure floor is -0.30, each requiring an interval that excludes zero.
Coverage shortfall on this claim
claude-haiku-4-5: 59 valid records of 60 attempted. 1 record carries a valid answer whose grader verdict could not be parsed, so it is in neither the valid count nor the invalid count and these columns do not sum to the arm total.
Method for this claim
- Task set
- 6 domain questions spread across law, medicine, tax, engineering, statistics, and security. Both arms ask the identical question; only the role sentence differs.
- Runs per model per arm
- 5
- Scoring
- Rubric graded, via rubric-quality-v1. A grader model reads each answer against a published rubric.
- Pass criterion as written for the pilot
- Mean graded score for treatment exceeds control by at least 0.5 rubric points on a 5 point scale, with the interval excluding zero.
The published verdict comes from status_v1, not from the pass criterion above. The criterion is recorded because it is what the claim was registered with before the run.
Model versions, as recorded
| Model | Version string returned | Delta on this claim | Interval |
|---|---|---|---|
| claude-haiku-4-5 | claude-haiku-4-5-20251001 | -0.013 | -0.265 to 0.238 |
| gpt-5-mini | gpt-5-mini-2025-08-07 | +0.133 | -0.046 to 0.312 |
| gemini-3.1-flash-lite | gemini-3.1-flash-lite | -0.033 | -0.200 to 0.133 |
Reading across models
Sampling was not held constant across vendors, so comparing one model column against another compares two settings as well as two models.
gpt-5-mini rejected the fixed sampling setting and ran at its own default on all 1040 of its calls. The other models ran at temperature 0.