InconclusiveCould not measure on claude-haiku-4-5Could not measure on gpt-5-miniCould not measure on gemini-3.1-flash-lite
Using tags and formatting
XML tags in prompts: do they beat markdown headers?
We could not measure this one. Both ways of marking up the prompt scored full marks, so the test could not tell them apart.
Why it matters
When everything passes, the test is measuring the task, not the technique.
Every model scored full marks with and without it, so the task set separated nothing.
What the marks mean
- Could not measure
Show the per-model numbers
Claim tested: XML tag delimiters improve instruction compliance over markdown headers on a multi-constraint task.
Could not be measured on claude-haiku-4-5, gpt-5-mini and gemini-3.1-flash-lite.
Circulates in practitioner communities. Tested because it circulates, not because it is endorsed.
This tip is OpenAddict's plain-language read of the measured result. The measurement below is the evidence, and it is what the reading has to answer to.
Ledger idC03-xml-delimiters
What was tested
This claim circulates in practitioner communities as advice about how to write prompts. That it circulates is an input to what gets tested here. It is a reason to test the claim, and it is not evidence for or against it. The result below is the evidence, and it is the only thing on this page that carries weight.
The comparison is paired. Two prompts differ in one respect, the manipulated variable, and are otherwise identical by construction. Nothing here supports a causal reading beyond that pairing.
Result
| Measure | claude-haiku-4-5 | gpt-5-mini | gemini-3.1-flash-lite |
|---|---|---|---|
| Control arm | 1.000n 50 | 1.000n 50 | 1.000n 50 |
| Treatment arm | 1.000n 50 | 1.000n 50 | 1.000n 50 |
| Delta | +0.000 | +0.000 | +0.000 |
| Interval, 95 percent | 0.000 to 0.000 | 0.000 to 0.000 | 0.000 to 0.000 |
| Orbit | Unobservable100 of 100 records | Unobservable100 of 100 records | Unobservable100 of 100 records |
Orbit is assigned by the frozen status_v1 rule. On this scale, deterministic pass rate, 0 to 1, the pass threshold is +0.20 and the failure floor is -0.20, each requiring an interval that excludes zero.
Method for this claim
- Task set
- 10 writing tasks, each carrying three independently checkable constraints. The constraint text is identical between arms; only the delimiter style around the sections differs. Hardened for instrument v2: v1 carried three constraints in a short task, which both delimiter styles satisfied every time. v2 carries five constraints, embeds them in a longer brief that also contains prose the model could mistake for instructions, and places a decoy sentence between the task and the rules. Delimiter clarity now has work to do.
- Runs per model per arm
- 5
- Scoring
- Deterministic, via scoreConstraintCompliance. A committed function scores each answer with no model in the loop.
- Pass criterion as written for the pilot
- Treatment full-compliance rate exceeds control by at least 10 percentage points on the same 10 tasks.
The published verdict comes from status_v1, not from the pass criterion above. The criterion is recorded because it is what the claim was registered with before the run.
Model versions, as recorded
| Model | Version string returned | Delta on this claim | Interval |
|---|---|---|---|
| claude-haiku-4-5 | claude-haiku-4-5-20251001 | +0.000 | 0.000 to 0.000 |
| gpt-5-mini | gpt-5-mini-2025-08-07 | +0.000 | 0.000 to 0.000 |
| gemini-3.1-flash-lite | gemini-3.1-flash-lite | +0.000 | 0.000 to 0.000 |
Reading across models
Sampling was not held constant across vendors, so comparing one model column against another compares two settings as well as two models.
gpt-5-mini rejected the fixed sampling setting and ran at its own default on all 1040 of its calls. The other models ran at temperature 0.