Claude skills for Product Management, tested
Skills for product management work, measured on spec writing tasks. Each was the same task, run twice, once with the skill loaded and once without it, and what is below is what changed. Ranked by how many models the rule found a clear effect on, never by the size of one.
Measured on 3 models (Claude, GPT & Gemini): Claude Haiku 4.5, GPT-5 mini, Gemini 3.1 Flash Lite.
The model is given a feature request and asked for a product spec meeting a stated contract: required sections and testable acceptance criteria. Scored two ways: against the contract, and by a grader comparing the with and without outputs blind. Both readings appear.
What helped, and on how many models
First
skills/pm-spec-writing rampstackco/claude-skills*
Spec writing · Blind comparison of the same task, run twice, with ties counting half
Helped clearly on Claude Haiku 4.5, Gemini 3.1 Flash Lite and GPT-5 mini. Measured against no skill loaded.
A judge preferred its output 25 times in 30 over the no-skill version. Instruction arm: not applicable to a blind preference.
- Claude Haiku 4.5 · helped
- Gemini 3.1 Flash Lite · helped
- GPT-5 mini · helped
Cost per task, with the skill loaded: about $0.00657. Adds about 6,800 tokens of context to every request.
Second
skills/product-capability affaan-m/ECC
Spec writing · Blind comparison of the same task, run twice, with ties counting half
Helped clearly on Claude Haiku 4.5 and Gemini 3.1 Flash Lite; unclear on GPT-5 mini. Measured against no skill loaded.
A judge preferred its output 27 times in 30 over the no-skill version. Instruction arm: not applicable to a blind preference.
- Claude Haiku 4.5 · helped
- Gemini 3.1 Flash Lite · helped
- GPT-5 mini · unclear. The model scored 30% without it.
Cost per task, with the skill loaded: about $0.00305. Adds about 1,100 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/product-capability affaan-m/ECC
Spec writing · Deterministic, scored against a committed answer key
Nothing left to measure. The models scored 98% without the skill. Measured against the same task with a one-line instruction.
Found 98% of the required sections and acceptance criteria, against 98% with no skill loaded. Against 97% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · Nothing left to measure. The model scored 99% without the skill.
- GPT-5 mini · Nothing left to measure. The model scored 97% without the skill.
In Claude Code
- Claude Fable 5, First run, single attempt · Nothing left to measure. The model scored 93% without the skill.
- Claude Opus 5, First run, single attempt · Nothing left to measure. The model scored 92% without the skill.
- Claude Fable 5, Second run, every task twice · Nothing left to measure. The model scored 97% without the skill.
- Claude Haiku 4.5, Second run, every task twice · Nothing left to measure. The model scored 98% without the skill.
- Claude Opus 5, Second run, every task twice · Nothing left to measure. The model scored 93% without the skill.
- Claude Sonnet 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5.1, Fable 5.1, single attempt · Nothing left to measure. The model scored 97% without the skill.
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 92% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 97% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 99% without the skill.
Cost per task, with the skill loaded: about $0.0022. Adds about 1,000 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/spec-driven-development addyosmani/agent-skills
Spec writing · Deterministic, scored against a committed answer key
Could not be measured on Gemini 3.1 Flash Lite. Measured against the same task with a one-line instruction.
Found 98% of the required sections and acceptance criteria, against 98% with no skill loaded. Against 98% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · not measured
- GPT-5 mini · Nothing left to measure. The model scored 96% without the skill.
In Claude Code
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 99% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 93% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 96% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 99% without the skill.
Cost per task, with the skill loaded: about $0.00257. Adds about 2,800 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
pm-execution/skills/create-prd phuryn/pm-skills
Spec writing · Deterministic, scored against a committed answer key
Could not be measured on Gemini 3.1 Flash Lite. Measured against the same task with a one-line instruction.
Found 98% of the required sections and acceptance criteria, against 97% with no skill loaded. Against 98% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · not measured
- GPT-5 mini · Nothing left to measure. The model scored 95% without the skill.
In Claude Code
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 91% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 94% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
Cost per task, with the skill loaded: about $0.00197. Adds about 900 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
engineering/skills/spec-driven-workflow alirezarezvani/claude-skills
Spec writing · Deterministic, scored against a committed answer key
Could not be measured on Gemini 3.1 Flash Lite. Measured against the same task with a one-line instruction.
Found 97% of the required sections and acceptance criteria, against 97% with no skill loaded. Against 97% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · not measured
- GPT-5 mini · Nothing left to measure. The model scored 95% without the skill.
In Claude Code
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 96% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 99% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 93% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
Cost per task, with the skill loaded: about $0.00554. Adds about 15,000 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/product-capability affaan-m/ECC
Spec writing · Deterministic, scored against a committed answer key
Nothing left to measure. The models scored 98% without the skill. Measured against no skill loaded.
Found 95% of the required sections and acceptance criteria, against 98% with no skill loaded. Instruction arm: not measured.
Via API
- Claude Haiku 4.5 · Nothing left to measure. The model scored 100% without the skill.
- Gemini 3.1 Flash Lite · Nothing left to measure. The model scored 99% without the skill.
- GPT-5 mini · Nothing left to measure. The model scored 94% without the skill.
In Claude Code
- Claude Fable 5, First run, single attempt · Nothing left to measure. The model scored 93% without the skill.
- Claude Opus 5, First run, single attempt · Nothing left to measure. The model scored 92% without the skill.
- Claude Fable 5, Second run, every task twice · Nothing left to measure. The model scored 97% without the skill.
- Claude Haiku 4.5, Second run, every task twice · Nothing left to measure. The model scored 98% without the skill.
- Claude Opus 5, Second run, every task twice · Nothing left to measure. The model scored 93% without the skill.
- Claude Sonnet 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5.1, Fable 5.1, single attempt · Nothing left to measure. The model scored 97% without the skill.
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 92% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 97% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 99% without the skill.
Cost per task, with the skill loaded: about $0.00305. Adds about 1,100 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/prd github/awesome-copilot
Spec writing · Deterministic, scored against a committed answer key
Could not be measured on Gemini 3.1 Flash Lite. Measured against the same task with a one-line instruction.
Found 95% of the required sections and acceptance criteria, against 97% with no skill loaded. Against 98% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · not measured
- GPT-5 mini · Nothing left to measure. The model scored 95% without the skill.
In Claude Code
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 91% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · Nothing left to measure. The model scored 100% without the skill.
Cost per task, with the skill loaded: about $0.00198. Adds about 1,100 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/pm-spec-writing rampstackco/claude-skills*
Spec writing · Deterministic, scored against a committed answer key
Nothing left to measure. The models scored 99% without the skill. Measured against the same task with a one-line instruction.
Found 95% of the required sections and acceptance criteria, against 99% with no skill loaded. Against 98% with a one-line instruction instead.
Via API
- Gemini 3.1 Flash Lite · Nothing left to measure. The model scored 99% without the skill.
- GPT-5 mini · Nothing left to measure. The model scored 99% without the skill.
In Claude Code
- Claude Fable 5, First run, single attempt · Nothing left to measure. The model scored 94% without the skill.
- Claude Opus 5, First run, single attempt · Nothing left to measure. The model scored 93% without the skill.
- Claude Fable 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Haiku 4.5, Second run, every task twice · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Second run, every task twice · Nothing left to measure. The model scored 92% without the skill.
- Claude Sonnet 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5.1, Fable 5.1, single attempt · Nothing left to measure. The model scored 98% without the skill.
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 97% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · not measured
Cost per task, with the skill loaded: about $0.00357. Adds about 6,700 tokens of context to every request.
Tied for third · shared with 7 others, and the tie is broken by nothing
skills/pm-spec-writing rampstackco/claude-skills*
Spec writing · Deterministic, scored against a committed answer key
Nothing left to measure. The models scored 98% without the skill. Measured against no skill loaded.
Found 94% of the required sections and acceptance criteria, against 98% with no skill loaded. Instruction arm: not measured.
Via API
- Claude Haiku 4.5 · Nothing left to measure. The model scored 99% without the skill.
- Gemini 3.1 Flash Lite · Nothing left to measure. The model scored 99% without the skill.
- GPT-5 mini · Nothing left to measure. The model scored 95% without the skill.
In Claude Code
- Claude Fable 5, First run, single attempt · Nothing left to measure. The model scored 94% without the skill.
- Claude Opus 5, First run, single attempt · Nothing left to measure. The model scored 93% without the skill.
- Claude Fable 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Haiku 4.5, Second run, every task twice · Nothing left to measure. The model scored 100% without the skill.
- Claude Opus 5, Second run, every task twice · Nothing left to measure. The model scored 92% without the skill.
- Claude Sonnet 5, Second run, every task twice · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5.1, Fable 5.1, single attempt · Nothing left to measure. The model scored 98% without the skill.
- Claude Fable 5.1, Wider run, three ways · Nothing left to measure. The model scored 98% without the skill.
- Claude Haiku 4.5, Wider run, three ways · Nothing left to measure. The model scored 97% without the skill.
- Claude Opus 5, Wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Sonnet 5, Wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Fable 5, Fable 5, wider run, three ways · Nothing left to measure. The model scored 95% without the skill.
- Claude Opus 5.5, Opus 5.5 series, wider run, three ways · not measured
Cost per task, with the skill loaded: about $0.00657. Adds about 6,800 tokens of context to every request.
The numbers
How to read these tables
How to read these tables
- Without the skill
- The score on the task set before any skill was loaded. Where this is already near the top of the scale, there was little room for a skill to show an effect, and the result says so instead of reporting no difference.
- Without -> with
- The score on the task set with no skill loaded, then with the skill loaded. For classes scored against an answer key, this is the share of seeded items found. For classes scored by blind comparison, it is how often a grader preferred the skill's output over the baseline's, with the baseline shown as the complement.
- Effect (range)
- With minus without, averaged over the models tested, with the lowest and highest per-model value in brackets. Per-model intervals are on each skill's page.
- Result
- helped: delta at or above the pass threshold, and the interval excludes zero
- worse: delta at or below the failure floor, and the interval excludes zero
- unclear: the interval spans zero, or the delta sits between the floor and the threshold
- not measured: the cell cannot be classified: sub-case (a) an unbounded scale, or sub-case (b) both arms at the same bound
A fifth value, Decaying, is defined and appears once a skill has been re-tested: a cell previously Stable that has fallen below the pass threshold on a later run.
- Cohort
- The repository that publishes the skill. Rows marked with an asterisk come from the repository the operator of this site maintains, so those are the operator measuring its own work. The rest were chosen by a stated rule and not by preference.
- Commit
- The exact version tested. Later commits are not covered by this verdict.
- Injected context tokens
- How much text the skill adds to the model's context. This is cost, not quality; a bigger number is not a better one. Measured from the input tokens the API reported on the with arm, averaged over that skill's records.
- Tested
- The date. Verdicts age as models change.
Spec writing
Each figure is how many points the skill added over no skill on that model, at the tier shown. A ring marks a clear lift. “Nothing left to measure” means the model already scored 90% or more with no skill.
Via API
From one run: first run, with and without the skill. One other run is in the tables below.
| Skill | Claude Haiku 4.5 | GPT-5 mini | Gemini 3.1 Flash Lite |
|---|---|---|---|
| Product capability deterministic | nothing left to measure 100% with no skill, tier 1 | nothing left to measure 94% with no skill, tier 1 | nothing left to measure 99% with no skill, tier 1 |
| Product capability paired | +100 points helped tier 1 | +40 points unclear tier 1 | +100 points helped tier 1 |
| PM spec writing deterministic | nothing left to measure 99% with no skill, tier 1 | nothing left to measure 95% with no skill, tier 1 | nothing left to measure 99% with no skill, tier 1 |
| PM spec writing paired | +80 points helped tier 1 | +60 points helped tier 1 | +60 points helped tier 1 |
| Spec driven development deterministic | not run | in another run | in another run |
| Create PRD deterministic | not run | in another run | in another run |
| Spec driven workflow deterministic | not run | in another run | in another run |
| PRD deterministic | not run | in another run | in another run |
In Claude Code
From one run: wider run, three ways. 5 other runs are in the tables below.
| Skill | Claude Haiku 4.5 | Claude Fable 5.1 | Claude Opus 5 | Claude Sonnet 5 |
|---|---|---|---|---|
| Product capability | nothing left to measure 100% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 | nothing left to measure 92% with no skill, tier 1 | nothing left to measure 97% with no skill, tier 1 |
| PM spec writing | nothing left to measure 97% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 | nothing left to measure 95% with no skill, tier 1 | nothing left to measure 95% with no skill, tier 1 |
| Spec driven development | nothing left to measure 99% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 | nothing left to measure 93% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 |
| Create PRD | nothing left to measure 100% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 | nothing left to measure 91% with no skill, tier 1 | nothing left to measure 94% with no skill, tier 1 |
| Spec driven workflow | nothing left to measure 99% with no skill, tier 1 | nothing left to measure 96% with no skill, tier 1 | nothing left to measure 93% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 |
| PRD | nothing left to measure 100% with no skill, tier 1 | nothing left to measure 98% with no skill, tier 1 | nothing left to measure 91% with no skill, tier 1 | nothing left to measure 95% with no skill, tier 1 |
| Skill | Cohort | Without the skill | Without -> with | Effect (range) | Result | Injected context tokens | Tested |
|---|---|---|---|---|---|---|---|
| No skill loaded Blind comparison of the same task, run twice, with ties counting half, First run, with and without the skill | baseline | 13% | 13% | 0 by definition | 0 | 2026-08-30 | |
| One-line instruction instead Blind comparison of the same task, run twice, with ties counting half, First run, with and without the skill | baseline | instruction arm: not applicable to a blind preference | 0 by definition | 0 | 2026-08-30 | ||
skills/product-capability from affaan-m/ECC d8409a4b0813 Blind comparison of the same task, run twice, with ties counting half First run, with and without the skill Best result in this test | affaan-m/ECC | 10% | 10% -> 90% baseline shown as the complement | +0.80 [+0.40, +1.00] | helped on 2 of 3 | 1,082 | 2026-08-30 |
skills/pm-spec-writing from rampstackco/claude-skills in the catalog 047924252254 Blind comparison of the same task, run twice, with ties counting half First run, with and without the skill | rampstackco/claude-skills* | 17% | 17% -> 83% baseline shown as the complement | +0.67 [+0.60, +0.80] | helped | 6,845 | 2026-08-30 |
| No skill loaded Deterministic, scored against a committed answer key, Wider run, three ways | baseline | 98% | 98% | 0 by definition | 0 | 2026-09-05 | |
| One-line instruction instead Deterministic, scored against a committed answer key, Wider run, three ways | baseline | 98% | 0 by definition | 0 | 2026-09-05 | ||
skills/product-capability from affaan-m/ECC d8409a4b0813 Deterministic, scored against a committed answer key Wider run, three ways Best result in this test | affaan-m/ECC | 98% at the ceiling | 98% -> 98% | +0.01 [-0.01, +0.03] | unclear | 1,047 | 2026-09-05 |
skills/spec-driven-development from addyosmani/agent-skills d2c37ef6225d Deterministic, scored against a committed answer key Wider run, three ways | addyosmani/agent-skills | 98% at the ceiling | 98% -> 98% | +0.00 [+0.00, +0.00] | Mixed, see page | 2,777 | 2026-09-05 |
pm-execution/skills/create-prd from phuryn/pm-skills 18468a95b427 Deterministic, scored against a committed answer key Wider run, three ways | phuryn/pm-skills | 97% at the ceiling | 97% -> 98% | +0.00 [+0.00, +0.00] | Mixed, see page | 941 | 2026-09-05 |
engineering/skills/spec-driven-workflow from alirezarezvani/claude-skills 19392f7a0826 Deterministic, scored against a committed answer key Wider run, three ways | alirezarezvani/claude-skills | 97% at the ceiling | 97% -> 97% | -0.00 [-0.01, +0.00] | Mixed, see page | 15,043 | 2026-09-05 |
skills/prd from github/awesome-copilot c956566a35c3 Deterministic, scored against a committed answer key Wider run, three ways | github/awesome-copilot | 97% at the ceiling | 97% -> 95% | -0.03 [-0.06, +0.00] | Mixed, see page | 1,147 | 2026-09-05 |
skills/pm-spec-writing from rampstackco/claude-skills in the catalog 047924252254 Deterministic, scored against a committed answer key Wider run, three ways | rampstackco/claude-skills* | 99% at the ceiling | 99% -> 95% | -0.04 [-0.06, -0.01] | unclear | 6,653 | 2026-09-05 |
| No skill loaded Deterministic, scored against a committed answer key, First run, with and without the skill | baseline | 98% | 98% | 0 by definition | 0 | 2026-08-30 | |
| One-line instruction instead Deterministic, scored against a committed answer key, First run, with and without the skill | baseline | instruction arm: not measured | 0 by definition | 0 | 2026-08-30 | ||
skills/product-capability from affaan-m/ECC d8409a4b0813 Deterministic, scored against a committed answer key First run, with and without the skill Best result in this test | affaan-m/ECC | 98% at the ceiling | 98% -> 95% | -0.02 [-0.04, +0.00] | unclear | 1,082 | 2026-08-30 |
skills/pm-spec-writing from rampstackco/claude-skills in the catalog 047924252254 Deterministic, scored against a committed answer key First run, with and without the skill | rampstackco/claude-skills* | 98% at the ceiling | 98% -> 94% | -0.04 [-0.12, +0.00] | unclear | 6,845 | 2026-08-30 |
| Averaged over three models; per-model results on each skill's page. Ranked within this class only. | |||||||
Ranked within this topic by how many models the rule found a clear effect on, which is a count of verdicts and never the size of one. Two classes are two answer keys on two scales, so no effect here is compared with another. Every class, with its full table.