Writing Testable User Stories for an Enterprise Knowledge Assistant

Welcome back. Last lesson, you mapped how a Harborline support agent finds returns guidance today and how an assistant might improve that journey. The map also exposed important branches: documents may disagree, guidance may be outdated, and an agent may have no source they can safely use.

This lesson turns those branches into testable user stories. A story will state what an agent needs and why; its acceptance criteria will make the proposed behavior checkable. By the end, you should be able to tell the difference between a promising idea for the assistant and a requirement that an agent and delivery team could actually verify.

Move from a journey to a user need

A journey map follows a task across several phases. A user story selects a small, valuable capability within that journey. The usual form is:

As a [specific user], I want [capability or outcome] so that [benefit].

For Harborline, a first draft might be:

As a support agent handling a returns case, I want to find relevant policy guidance so that I can decide how to answer the customer.

The agent, need, and reason are all visible. The story does not say that a language model must use a particular search service or that every question must receive an answer. Those would either dictate an implementation prematurely or conflict with the uncertain cases on our map.

The Business Analysis Doctor’s explanation adds a useful discipline: a story is not just the sentence on a card. It also requires a conversation about its meaning and a way to confirm that the delivered behavior meets it.

The conversation matters especially here. Harborline has reported that agents search a shared drive, contact specialists about conflicting documents, and sometimes copy outdated PDF text. We do not yet know how approved guidance is designated or what an agent is permitted to see. A neatly worded story cannot settle those questions; it can make them precise enough to investigate.

Notice also the difference between a user story and a development task. “Build a search index” is work a team might eventually perform. “Find relevant guidance for my case” describes something the agent needs to accomplish. The latter gives the team room to consider solutions while keeping the user’s purpose in view.

Add a way to confirm the story

The draft story above gives direction, but what would count as finding relevant guidance? Would an answer with no source count? What if the only matching document is withdrawn? These are questions for acceptance criteria: observable conditions that must hold for the story to be accepted.

Read Atlassian’s distinction between a story and its criteria before looking at the Harborline examples. The article’s general examples are useful for form, though our enterprise case needs additional attention to source status and access.

One helpful format for a behavioral criterion is Given / When / Then. Given establishes the test situation, When names the action, and Then states the observable result. It is a format, not a requirement that every criterion use those words. A short rule such as “A displayed citation opens the supporting source passage” can also be tested, provided “supporting” is defined against an agreed example.

For an enterprise knowledge assistant, testable does not necessarily mean checking that a generated answer contains an exact sentence. Two sound answers may use different wording. A test can instead check agreed properties: whether a source link resolves, whether the cited passage supports a particular claim, and whether the assistant refrains from making a policy claim when the test materials contain no supporting guidance. The team must still agree on the test materials and who judges support.

Draft stories from Harborline’s decision points

We can now split the proposed journey into capabilities an agent could recognize. The examples below are candidate stories, not confirmed Harborline requirements. They deliberately use controlled test documents: documents whose contents, status, and access permissions have been agreed for testing. Whether Harborline can identify those properties in its real collection remains an open question.

1. Locate guidance for a case

As a support agent handling a returns case, I want to find potentially applicable guidance so that I can identify material to check before replying.

• Given an agreed test collection with an accessible returns document that addresses the question, when the agent asks that question, then the assistant presents that document as a candidate source the agent can open.
• Given a test question with no accessible, relevant source in the collection, when the agent asks it, then the assistant does not present an unrelated document as if it answered the question.

“Potentially applicable” is intentional. Finding a matching document does not establish that it is approved for this case. The story covers the find guidance phase; it does not quietly promise that retrieval alone resolves a policy decision.

2. Inspect the basis for suggested guidance

As a support agent, I want to inspect the source behind suggested returns guidance so that I can judge whether it applies to my customer’s case.

• Given a suggestion based on an accessible test document, when the agent opens its citation, then the citation identifies the source and leads to the passage used to support the suggestion.
• Given a suggested policy claim and its cited passage, when the passage is reviewed, then it supports that claim; a citation to a merely related passage does not pass.
• Given a test source whose status is explicitly marked withdrawn, when the assistant presents it, then the assistant does not describe it as approved current guidance.

The last condition depends on a business definition, not just a software feature. A policy owner would need to establish what “withdrawn” and “approved” mean and how those statuses are recorded. If Harborline cannot reliably supply source status, we should not claim that the assistant can certify a document as current.

3. Handle absent or conflicting support

As a support agent, I want to know when the assistant cannot establish supported returns guidance so that I can seek policy help rather than send an unsupported answer.

• Given a test question for which no accessible, approved source supports a policy answer, when the agent asks it, then the assistant indicates that it cannot establish an answer from available guidance and shows the agreed next step for seeking help.
• Given two approved test sources that make incompatible statements about the same case, when the agent asks about that case, then the assistant does not present either statement as the definitive policy without qualification and shows the agreed next step.

This story preserves the specialist branch from the journey map. “Shows the agreed next step” is testable once Harborline specifies that step: for example, which team to contact and how. Until then, it is an explicit point for conversation, not an invitation for the delivery team to invent a policy-escalation route.

4. Respect the agent’s access

As a support agent, I want guidance shown to me to be limited to documents I am permitted to access so that I can use the assistant within my role.

• Given two test agents with different document permissions and a returns document available only to one of them, when each asks a question about that document, then only the permitted agent can receive its contents or a citation that exposes them.
• Given that the restricted document is the only source supporting an answer, when the agent without permission asks the question, then the assistant does not disclose the answer from that document, including through a summary or source preview.

This is still written from the agent’s experience; it does not dictate how access checks are implemented. It also reveals a dependency for the project: Harborline must confirm which permissions apply to the documents and users. A citation is not safe merely because it looks like a link—the answer and preview must respect the same boundary.

Together, these stories cover distinct moments in the agent’s task: locating material, assessing its basis, handling uncertainty, and working within access rules. They need not imply that each can be released entirely independently; they are separated so that their value, questions, and acceptance tests remain visible.

Refine without mistaking precision for certainty

A common first attempt at this backlog might read:

As a user, I want an AI-powered search chatbot that instantly gives accurate answers to everything so that work is faster.

Almost every part needs discussion. “User” hides whose task matters. “Chatbot” chooses an interface before establishing the need. “Instantly,” “accurate,” and “everything” offer no workable pass/fail test and erase the possibility that a safe answer is “I cannot establish that.” The Harborline stories above instead identify a role, a bounded situation, and observable outcomes—including unsuccessful paths.

Another tempting criterion is “The assistant reduces a returns lookup to under 12 minutes.” The earlier brief contained a 12-minute estimate, not a verified baseline or an agreed target. Speed may become important, but it requires a defined measure and evidence. In the next lesson, we will set MVP success criteria—business, user-experience, and technical measures—rather than hiding an unvalidated project-wide target inside one story’s acceptance test.

Before a team treats these candidate stories as ready to build, its conversations with agents, policy owners, and access-control owners should settle questions such as: Which documents count as approved? What should happen if two approved sources conflict? Where exactly does an escalation go? Which identity and permissions determine what an agent may see? The answers may change the stories or their criteria. That is refinement working as intended, not a failure to write the first draft perfectly.

Conclusion

A user story names a specific person’s need and the value of meeting it. Acceptance criteria state what someone could observe or test to decide whether that need has been met. For Harborline, good criteria must cover more than the happy path: they should make unsupported guidance, conflicting sources, misleading citations, and unauthorized disclosure visible as failures.

The resulting stories are a starting agreement for discussion, not proof that the underlying documents and policies are ready. Next, we will define measurable success criteria for the MVP as a whole, distinguishing evidence of business and user benefit from the pass/fail conditions on individual stories.