Tips
I got tired of hunting for prompting tips that actually work, so I built a place to collect and test them.
18 claims tested3 models4,938 records
17 tips, from 18 tested claims: the examples family reports two claims on one page.
Each tip rests on a claim tested as a paired comparison on a fixed task set. Across 54 tested pairs the verdicts are 23 In free drift, 16 Stable, 14 Unobservable, 1 Past the horizon.
In free drift: could not be told from no effect; Stable: held; Unobservable: the test could not measure it; Past the horizon: measured worse.
Showing 17 of 17 tips.
Stop made-up answerslead
How to use itAdd a line saying that if the text does not answer the question, replying I do not know is fine. Put it next to your question.
Pass rate up 80 to 100 points on all three models.
Show the per-model numbersBetter step-by-step answersco-lead
How to use itAdd a line asking for the steps before the answer on any task with chained arithmetic or logic. Check that it helps on the model you use.
Pass rate up 20 points on Claude and 34 points on GPT. Inside the margin on Gemini.
Show the per-model numbersGetting clean JSON back
How to use itPut a small example of the structure in your prompt. List every field you need and what kind of value it holds.
Pass rate rose from 0% to 100% on all three models.
Show the per-model numbersShowing examples
How to use itPaste one example in the format you need. Add more only for formats that are tricky to describe in words. This helped a little on every model we tested, under our bar for a clear win.
Pass rate up 18 to 19 points on all three models.
Show the per-model numbersAsking for a rewrite
How to use itIn one prompt, ask for a draft, then a short critique, then a final version. Compare it against what a single pass gives you.
Rubric mean up 0.30 points on Claude. Inside the margin on GPT and Gemini.
Show the per-model numbersSplitting big tasks
How to use itAsk for the first step, then feed its answer into the next prompt. Best on tasks with several parts. Check that it helps on the model you use.
Pass rate up 10 to 25 points on all three models.
Show the per-model numbersGetting the length right
How to use itSay exactly how many words you want rather than keep it short. Expect the answer to land within a few percent of the number you asked for.
Pass rate up 68 to 95 points on all three models.
Show the per-model numbersGiving the model a job title
You have probably heard you should tell the model to act as an expert.
What to do insteadSpend those words on the task. Say what you want, who will read it, and what a good answer has to cover.
Rubric mean moved at most 0.13 points on all three models, inside the margin every time.
Show the per-model numbersAdding pressure
You have probably heard that telling the model your job depends on it gets you a better answer.
What to do insteadDescribe the task and the standard you want. Concrete requirements do the work that urgency does not.
Rubric mean moved at most 0.11 points on all three models, inside the margin every time.
Show the per-model numbersBeing polite
You have probably heard that being polite, or being blunt, changes what you get back.
What to do insteadBe as plain or as warm as you like. Spend the effort on stating the task clearly.
Rubric mean moved at most 0.13 points on all three models, inside the margin every time.
Show the per-model numbersRules for the whole chat
You have probably heard that rules in the system prompt stick better than rules in the message.
What to do insteadPut rules wherever is convenient, and repeat them if the chat runs long.
Measured on two models; the rest could not be measured on this task set.
Show the per-model numbersPrompt, then revise
You have probably heard that asking the model to critique and revise its own draft improves it.
What to do insteadWrite one careful prompt. Revise only when you can name what is wrong.
Measured on two models; the rest could not be measured on this task set.
Show the per-model numbersQuestions about recent events
You have probably heard that telling the model its knowledge may be out of date makes it more careful.
What to do insteadOnly add it for questions about recent events, and check the marks: on Claude it made answers worse.
Pass rate up 41 points on GPT and 30 points on Gemini, and scored worse on Claude by 23 points.
Show the per-model numbersInstructions before or after
Every model scored full marks with and without it, so the task set separated nothing.
Show the per-model numbersUsing tags and formatting
Every model scored full marks with and without it, so the task set separated nothing.
Show the per-model numbersInstructions in long prompts
Every model scored full marks with and without it, so the task set separated nothing.
Show the per-model numbersOffering the model money
Answer length rose by 44 to 180 words, but this metric has no threshold, so no verdict follows.
Show the per-model numbersShowing 17 of 17 tips.