AI for Product Managers, Tested
Four jobs product managers hand to AI, each run on named models with no help, with one plain instruction, and with skills from public catalogs. One has results so far. The other three are written down and not run yet.
The PM jobs
Customer feedback analysis
Customer feedback analysis means reading a pile of feedback or interview notes and grouping it into themes, with how often each comes up.
Registered, not run yet.
RICE prioritization
RICE prioritization means scoring a backlog on reach, impact, confidence and effort, then ranking it.
Results are in.
User stories
Writing user stories means turning a feature brief into stories with acceptance criteria a team can test.
Registered, not run yet.
PRD drafting
Drafting a PRD means turning a brief into a product requirements document that keeps the facts it was given.
Registered, not run yet.
Skills from public catalogs
Each job is run once more with each skill a public catalog offers for it. A skill that does not help is listed as not helping.
| Job | Skills | What we found |
|---|---|---|
| Customer feedback analysis | 5 to try | Not run yet |
| RICE prioritization | 3 tried | Do not load one of these skills for this job: none did better than the one plain sentence on any model, and two did worse. Every skill on the RICE prioritization page. |
| User stories | 3 to try | Not run yet |
| PRD drafting | 5 to try | Not run yet |
Which model for PM work
Each pick is made inside one job and one test method, from what the models did with no help.
| Job | Test method | Best on its own | Cheapest within range of the best |
|---|---|---|---|
| Customer feedback analysis | Not run yet | ||
| RICE prioritization | In Claude Code | Claude Fable 5.1, with Claude Opus 5.5 too close to call | Claude Opus 5.5, $0.05680 a backlog |
| RICE prioritization | Via API | None ranked first: each model had its own backlog size | Not ranked |
| User stories | Not run yet | ||
| PRD drafting | Not run yet | ||
How these are tested
Every task, its right answer and how it is scored are written down before any model sees it. Each job also hides at least one gap, and nothing in the task says it is there. The full method is on the methodology page.
Three jobs are not run yet. When they run, each of 9 models gets 10 tasks at each of 3 inputs of different sizes, asked 5 times.