Kumano / Prompt review
Prompt review · formerly Promptly
Catch the unclear ask before the agent runs with it.
Prompt review is the part of Kumano that checks what went into an agent session, not what came out. A judge model scores the prompts behind each Claude Code session for intent clarity and scope discipline, explains the score and suggests a clearer rewrite. On a pull request, Kumano can post one verdict for every session behind the change, plus a check on the commit.
Why review the prompt?
Code review made agent output something a team could trust. But a lot of wasted agent work starts earlier, with an ask that never said what done looks like, or a scope that kept moving halfway through. Prompt review puts that part of the work in front of a reviewer, next to the diff.
What it scores
Each captured session is scored from 1 to 5 against two written rubrics. Both judge the person's prompts only: not code quality, and not whether the task got finished.
- Intent clarity. Did the first turns say exactly what was wanted, with the constraints, scope and success criteria present or obvious? Lots of clarifying questions or corrections later in the session count as evidence that they didn't.
- Scope discipline. Once work began, did the scope hold steady, or did it expand, contradict itself or get re-argued mid-session?
- Spend efficiency. On a pull request, spend is reported next to prompt quality. With the judge switched off, spend is the only thing scored.
Every score comes with the reason for it and a rewrite of the prompt that would have scored higher. On a pull request the per-session scores are averaged and shown out of 10; the card on Kumano's board shows the same number as the check (for example "PR 8/10"), plus avoidable spend and a link to the comment.
Where the review shows up
- On the card. When a Claude session finishes, Kumano hands it to the review engine and shows the verdict on the feature's card. Chat sessions are captured as well as guided runs.
- On the pull request. When Kumano opens a PR, it can review every captured session behind it and post one comment, edited in place when it re-runs, plus a
promptly/prompt-reviewcheck. A teammate sees how the change was asked for without installing anything. Right-click a card to re-run. - In a terminal. The engine is the open-source Promptly CLI (MIT).
promptly review --pr <n> --commentreviews every session behind a GitHub pull request without the app.
Privacy: local by default, your key when you opt in
- Capture is local and free.
- The judge is off by default. It uses your own Anthropic API key, stored in the macOS Keychain, and only then is a captured session sent to Anthropic's API to be scored. Without a key the judge never runs.
- Turning capture off switches the judge off too.
- Posting to pull requests is its own opt-in, also off by default.
Set it up
1. Install the review engine once (Kumano shows this command in Settings → Prompt Review):
npm i -g @getpromptly/cli
2. In Kumano, open Settings → Prompt Review, switch on the judge and add your Anthropic API key.
3. Optional: switch on "Post reviews to pull requests" in the same pane. The next PR Kumano opens gets the comment and the check.
Requirements: Kumano 0.8 or later on macOS 15.6+, Node.js for the CLI, and the GitHub CLI signed in for posting to pull requests.
Frequently asked questions
What does prompt review score?
The prompts behind a piece of work, not the code. A judge model scores each Claude session on intent clarity (did the ask say what was wanted, with constraints and a definition of done) and scope discipline (did the scope hold steady once work began), each from 1 to 5, explains the score and suggests a clearer rewrite. On a pull request the scores are combined into one verdict out of 10, alongside spend efficiency.
Does prompt review send my prompts anywhere?
Capture is local and free. The judge is off by default; it only runs if you switch it on and add your own Anthropic API key, which Kumano stores in the macOS Keychain. Then the captured session is sent to Anthropic's API to be scored. Without a key the judge never runs, and turning capture off switches the judge off too.
Do my teammates need Kumano to see the review?
No. When posting is switched on, Kumano adds one comment to the pull request, edited in place when the review re-runs, plus a promptly/prompt-review check on the commit. Anyone who can read the pull request can read the verdict.
How do I turn it on?
Install the review engine once with npm i -g @getpromptly/cli. In Kumano, open Settings → Prompt Review to switch on the judge and add your Anthropic key. Posting to pull requests is a separate opt-in: Settings → Prompt Review → "Post reviews to pull requests".
Can I run prompt review without Kumano?
Yes. The engine is the open-source (MIT) Promptly CLI. promptly review --pr <n> --comment reviews every captured session behind a GitHub pull request from a terminal and posts the same kind of comment.
What happened to Promptly and getpromptly.xyz?
Promptly was merged into Kumano and its review engine now powers this feature. The CLI package @getpromptly/cli and the promptly/prompt-review check name stay as they are, so existing setups keep working.
Try it
Prompt review ships in Kumano 0.8. Kumano is a native Mac app that organises Claude Code by your projects and is free during the beta. What Kumano does → What's in 0.8 →
Prompt review · one comment, edited in place on re-run
8/10
## Prompt quality
Intent clarity and scope discipline, averaged across the sessions behind this PR, with the lowest-scoring session called out first.
## Spend efficiency
What the sessions cost, and how much of it was avoidable.
## Built from
intent.md·spec.md·plan.md