Turn a bad LLM output into a test you never break again
Paste the input, what should have happened, and what your model actually said. Save it as a case. Write a revised prompt. See — case by case — whether the revision fixes it without quietly breaking everything that already worked.
Deterministic evaluation mode · no live model provider required
| Case | Rev A · baseline | Rev B · candidate |
|---|---|---|
| Refuses valid refund window | Fail | PassFixed |
| Leaks internal policy id | Fail | PassFixed |
| Handles duplicate order id | Pass | Pass |
| Declines out-of-window request | Pass | FailRegressed |
Illustrative example — not real customer data.
How it works
One cohesive path from a real failure to a saved, comparable result.
1. Save the failure
Paste the input, the expected behavior, and the bad output your model produced. It becomes a reusable case.
2. Draft revisions
Write two prompt revisions — your current one and a candidate fix — inside the same project.
3. Run the comparison
A bounded, deterministic evaluation scores each saved case against both revisions.
4. Read the verdict
See exactly which cases got fixed, which stayed broken, and which regressed — before you ship.
Not another prompt playground
Regressor starts from a real failure you already have, not a blank prompt box. Every case in your suite is something that actually went wrong, so the comparison you see is grounded in your product's real edges — not a generic benchmark.
Revision-to-revision, not one-shot
The result view is built around comparing two revisions side by side, so a fix that quietly breaks an older case is as visible as a fix that works. That visibility is the whole point.