How to write a PRD for an AI agent, starting from real work

What is a PRD for an AI agent, and how is it different?

A PRD for an AI agent, sometimes called an agent spec, does what any product requirements document does: it says what to build, for whom and why. It also says how you will know the agent is right, what it must never do, and when it must hand the work back to a person.

Those extra parts exist because an agent doesn’t behave like ordinary software. Its output varies: the same case can come out differently on different runs, which is why Anthropic’s engineers test each task more than once. So the PRD defines “correct” with test cases, a rubric and a pass rate. And an agent acts on its own, so the PRD has to say where it must stop. OpenAI’s guide to building agents lists stopping as a core trait: when an agent fails, it can halt and hand control back to the user.

An agent also goes live in steps, from shadow mode to acting with spot checks, and it is never finished, because a new model, prompt or policy can change how it behaves.

Why start an agent PRD from real work, not a wish list?

Because the agent will meet the work as it really runs. Ask a team what an agent should do and you get a wish list, built from the process document and from memory. The real work has more steps, more exceptions and rules nobody wrote down.

Daniel Kornum of Varick Agents describes the usual failure: a company scopes an agent from the SOP, ships it, and the agent collides with the actual workflow. He calls the gap between the documented process and the real one the conformance gap, and puts it at 30% or more in Varick’s typical engagements. Build for the document, he writes, and you automate 70% of the volume and break on the other 30%.

OpenAI’s guide suggests building an agent’s instructions from existing procedures and policy documents. That works only when the documents match the work. So before you write the PRD:

1. Map the workflow from the people who do it, the systems they use and the documents they follow, exceptions included. See how to map a process for AI.
2. Sort every step into the four buckets: delete it, run it as plain code, give it to an agent, or keep a person on it. The PRD covers the agent’s steps and the handoffs around them. Plain-code steps need an ordinary spec, and deleted steps need nothing.

The map also supplies the PRD’s evidence: how often the work happens, how long it takes, where it goes wrong, and the past cases you will test on. In our revenue review example, the process document lists five steps, four observed Mondays show 20, and only four of those need an agent.

What goes in a PRD for an AI agent?

Thirteen sections, grouped by the five questions they answer. Most need only a few lines, and the eval cases can go in an appendix.

Four of the early sections need extra care:

• Problem, with evidence. State the volume, the time and the cost today, and link each number to where it came from. If the map can’t fill this section, everything after it is guesswork.
• Owner and users. Name one person who answers for the agent’s results and approves its limits, usually whoever owns the workflow. Then name who uses the output and who reviews it.
• Inputs and data sources. List every source the agent reads, with the access it needs: read, draft or write. Start everything at read-only, and add access only where a requirement calls for it.
• Tools and permissions. Give the agent only the tools the work needs. The OWASP Gen AI Security Project lists excessive functionality, permissions and autonomy as the root causes of “excessive agency,” one of its 2025 Top 10 risks for applications built on large language models. OpenAI suggests rating each tool’s risk as low, medium or high, based on whether it writes, whether its effects can be undone, the permissions it needs and its financial impact. The rating then decides when the agent pauses for checks or escalates to a person.

How do you define success for an AI agent?

Define it as an eval the agent must pass before it acts on real work: real cases, a rubric that says what correct means, and pass thresholds set against how your people do the same work today. Write it before the first run, so the goalposts can’t move.

“Early on, evals force product teams to specify what success means for the agent.”

— Anthropic, “Demystifying evals for AI agents,” January 2026

• Cases. Take them from the mapped work, awkward ones included: the late renewal, the missing purchase order, the figures that conflict. Anthropic’s engineers say 20 to 50 simple tasks drawn from real failures is a great start. Include cases where the agent should hand the work back and cases where it shouldn’t, so you test both sides of every rule.
• Rubric. Turn “good” into yes-or-no checks, such as the right total, every conflict flagged and nothing sent. That is a rubric. Anthropic’s test for a well-written task is that two experts would reach the same verdict on their own, and it advises grading what the agent produced rather than the exact path it took.
• Thresholds. Set the pass rate against your people’s results on the same cases, not against perfection. A human baseline gives you that number; see how to benchmark AI on your own work. Count serious errors on their own, with a bar of zero. Because results vary, run each case several times, and count the critical checks as passed only when they pass on every run.
• Budget. Set a ceiling on cost per task: model usage across every attempt, plus review time and fixing mistakes. Cap retries too. OpenAI suggests limits on retries and actions, with a person taking over when the agent goes past them.

How do you write guardrails and escalation rules for an agent?

Write three kinds of limit, and make each one testable:

• Never. These are the agent’s guardrails: actions it must not take in any case, such as messaging customers, moving money or editing a system of record. Enforce each one with permissions or code, in the agent’s harness and in the systems it touches, because a model can be wrong, or be manipulated by something it reads. OWASP advises checking authorization in those systems rather than letting the model decide what is allowed.
• Hand back when. These escalation rules stop the agent and send the case to a named person. OpenAI’s guide names two triggers: going past failure limits, such as too many retries, and high-risk actions, which should stay under human oversight until confidence in the agent grows. Add the triggers your map turned up, such as conflicting figures or an amount over a set limit. Each rule needs a condition, a threshold and a recipient.
• Approvals. These say which outputs a person signs off before they take effect, and who signs. OpenAI’s agent documentation calls this human review: the run pauses until a person approves or rejects the action. Make each approval specific: show the reviewer what the agent checked, what it is unsure of and which decision is theirs. See human-in-the-loop.

The rollout plan says how those limits loosen. In a staged rollout, the agent starts in shadow mode, working real cases and sending nothing. Then it suggests work that a person approves, and then it acts with spot checks. Write down the evidence each step needs, and what moves the agent back down, such as one serious error.

The monitoring plan says what you track after launch: the eval pass rate, hand-backs, outputs people changed, and cost per task. Re-run the evals whenever the model, a prompt, a tool or the process changes. Anthropic’s engineers note that monitoring live work catches drift and real-world failures that tests miss, and they suggest reading a sample of transcripts every week. See AI agent observability and governance.

How do you write requirements an agent and an eval can both use?

Write each requirement so the agent can follow it and a test can check it. When GitHub introduced its open-source Spec Kit, it gave the reason: language models are “exceptional at pattern completion, but not at mind reading.” A vague requirement leaves the agent to guess what you meant, and leaves the eval nothing to check. Four rules help:

1. Be specific. Name the sources, the output, the deadline and the threshold.
2. Make it checkable. A person or an automated grader can mark it pass or fail from the output alone.
3. Link it to evidence. Each requirement cites where it came from: the observed weeks, the volumes, the past exceptions. When the evidence changes, you know which requirement to revisit.
4. Test both sides of every limit. Write cases just inside and just outside each threshold.

If you can’t write the test, rewrite the requirement. Anthropic’s engineers call defining eval tasks one of the best ways to find out whether requirements are concrete enough to start building.

Worked example: a PRD for a revenue review agent

Here is a shortened PRD for the weekly revenue review at the illustrative company from Nolli’s product demo. Six people on Revenue Ops spend about 14 hours each week assembling it from five apps. That is 56 hours a month, worth about $44,000 a year at $66 an hour. Three figure conflicts came up last month. After sorting its steps, four went to an agent, and the final recommendation stayed with the Revenue Ops lead.

Three things make it work. Every requirement can be tested: a 09:30 deadline, a 5% threshold, read-only access. Every line shows where it came from, so a reviewer can see why it is there. And the limits come from the person who owns the work: the Revenue Ops lead set the never rule, and the recommendation, always written by hand, stays out of scope.

The eval cases come from the same evidence: the four observed Mondays, last month’s three conflicts and the two weeks when late renewals were missing from the deck. The same lines also give you the agent brief, the short version the agent works from: its goal, what it can use, when it hands back and what it must never do.

What are the most common mistakes in agent PRDs?

Watch for six mistakes:

• Writing from the SOP. The agent meets the real process, so map it first.
• Defining success in words. “Accurate” and “timely” can’t be tested. Anthropic warns that teams can waste weeks on AI features that seem to work in early testing but miss expectations nobody wrote down.
• Testing only the easy cases. If the eval set has no conflicts, late arrivals or cases near a threshold, the hand-back rules have never been tested.
• Putting limits only in the prompt. If the agent could still break a never rule, the rule is only a request. Back it with permissions.
• Giving the agent too much. Steps that plain code can do, or that shouldn’t exist, don’t belong to the agent. Anthropic recommends the simplest solution that works, which sometimes means no agent at all.
• Treating the PRD as finished. Add every correction as a new eval case, and update the PRD when the model, the tools or the process change.

Sources

1. Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares and Jiri De Jonghe. “Demystifying evals for AI agents.” Anthropic, January 9, 2026.
2. “A practical guide to building agents.” OpenAI.
3. Daniel Kornum. “If AI is so great, why isn’t it working?” Varick Agents.
4. “LLM06:2025 Excessive Agency.” OWASP Gen AI Security Project, accessed October 4, 2026.
5. “Guardrails and human review.” OpenAI API documentation, accessed October 4, 2026.
6. Den Delimarsky. “Spec-driven development with AI: Get started with a new open source toolkit.” The GitHub Blog, September 2, 2025.
7. Erik S. and Barry Zhang. “Building effective agents.” Anthropic, December 19, 2024.