From Acceptance Criteria to Acceptable Product
We write requirements to communicate to development teams what they need to build to achieve goals for our company. These requirements let the team know what needs to be accomplished by users of our products or to enable the goals of stakeholders to be achieved.
With an outside-in perspective, most requirements focus on the goals of our product users. From the developer’s point of view, the requirements need to provide enough precision for the developer to know when the requirement has been satisfied.
When the requirements are written as user stories, the definition of what it means to satisfy the requirement is captured in the acceptance criteria. This is the definition that developers use to know if what they have delivered is acceptable. By including unambiguous definitions of what is acceptable, you give the developers a crisp definition of done. They know if what they have done meets the expectations explicitly defined for the user story.
While this protects a development team that is accountable for doing what they are asked to do, there is still a risk that the team has not been asked to do the right thing. Writing acceptance criteria not only gives you a mechanism for clarifying what you want to be built, but it also gives you a mechanism for verifying that you are asking the development team to build the right thing.
Teresa Torres' good article on writing acceptable acceptance criteria focuses on the criteria of acceptability to the user. A couple of good tips from Torres emphasize the importance of capturing criteria that reflect how usable your product is—both by a single individual and under load—when multiple people are using the product (think website) simultaneously.
This is enough to create a product that is good for a user—assuming you have captured the acceptance criteria effectively. This is not enough, however, to create a product that meets the goals of your company. You have to ask the next question—“If we satisfy the users, will the product be successful?”
That’s where Eric Reis’ Lean Startup gives us guidance. You need to form a hypothesis (and test it). One example hypothesis could be “If we satisfy SAT-prep requirements of the top 10% (academically) of high school juniors, we will be able to sell our SAT-prep service to at least 25% of those students.”
The acceptance criteria in the user stories define what the team is asked to build for the SAT-prep service. By engaging those students and testing the acceptance criteria, we become comfortable that what we are asking the development team to build is what we need them to build to satisfy the students' requirements.
To fully test this hypothesis, the team would have to build and sell the entire product. That’s not very lean. By recurring with the hypothesis-formulation process, a product manager can narrow the scope of experimentation. For example, “If we assume that a third of the students are particularly interested in improving their scores on the verbal portion of the SAT, then satisfying their requirements would allow us to sell our product to 8% of our target market.” That gives the team an area of focus.
Market research could lead to additional narrowing of scope—that the area needing the most improvement is in knowing synonyms, and that doubling the number of correct answers in that portion of the SAT would raise verbal scores by an average of 50 points out of 800—satisfying half of the students. Building a minimally viable product that helps the subset of the market with this subset of the problem is more tractable. The team can build this solution, measure the effectiveness of what they built, and measure the effectiveness of selling this solution in the market.
Using acceptance criteria to define an acceptable product works all the way up the stack from clarifying what the development team is being asked to do, to understanding what the users really need, to defining the minimum viable product that allows the company to achieve its goals.
