Agile Axiom #5 – Understanding and ahdering to iteration acceptance critieria is critical to the overall quality of a release.

July 13, 2008

One of the key benefits of Agile is the reduction in the difference between the level of quality for which we strive and the level of quality we actually achieve. Realizing this benefit requires careful attention to iteration acceptance criteria. Driven by the fact that most Agile methods rely on “inside the iteration” testing as their primary test cycle, the impact of ambiguous or not followed acceptance criteria can have a devastating impact on the quality of a release. Over the past several years, there has been much discussion around what it means to be “releasable” at the end of an iteration. Often, teams will set fairly ambiguous acceptance criteria for marking a story or requirement as complete. The level of discipline here is directly related to the level of quality of a final release. So what is good acceptance criteria? “Good” acceptance criteria needs to meet the following in order to be effective in ensuring a given story or requirement is ready for its consumers.

  1. Can the user or system consumer of a story or requirement complete the task for which the story is to enable? In other words, does it work? This is fairly binary criteria. If the user or consumer can complete the action, then this one is passed. One should keep in mind that if the result of a given story or requirement cannot be measured with this binary view, the story itself may need revision. “Testability” is one of the key characteristics of a “good” requirement.
  2. Are the scalability aspects of the story defined and met? This point is likely the most critical and least understood. An example of a scalability aspect of a requirement might be the number of objects a given UI needs to be able to manipulate. Scalability is often a separate consideration than “inside the iteration” testing usually provides. One could argue whether or not it should be (a good topic for another post). However, regardless of how the overall scalability of a release is managed and tested, it is critical that the scalability needs are considered at design and construction time, within the iteration. Therefore, the acceptance criteria needs to include the scale levels for which a given story must operate. Without an understanding of the scalability requirements of a given story, there is significant risk of needing to rework code after the end of an iteration.
  3. Are the non-functional requirements for a given story met? Other non-functional requirements include but are not limited to platform support (operating systems etc.), foreign language support, coexistence requirements, compatibility and interaction with third-party software, etc. Understanding how each of these impact a given story and to what level they need to be tested within an iteration will improve the overall quality of the release.
  4. Does the story meet the overall quality objectives of a release? This is more of a general criteria but must be examined. It is possible to be build the code to support a story that passes the criteria above but still causes the product quality to be poor. Lets take a simple example. If there is a story that says “Create a file selection dialog which allows a user to select which files to open for editing”. It could be the dialog exists, works with lots of files, and meets the defined non-functional requirements but does not function properly when the user presses the cancel button. Maybe the user sees a stack trace when pressing cancel. This case may not be captured as a specific acceptance criterion. However, it will likely need to be addressed (or should be). I would suggest there is an ambiguous criterion which requires that products must be at a “reasonable” level of quality beyond the defined acceptance criteria. This notion does not imply a level of quality less than the traditional criteria states, it implies the quality criteria will likely not cover all of the cases which should be tested. Agile helps with this situation as the product is being tested as it is being coded and these issues will be addressed within an iteration before the code is committed or causes other problems. In waterfall methods, these issues would be found much later in the process and would certainly be much more costly to address. In either case, success with this ambiguous level of quality requires a team that establishes a high threshold for quality and simply refuses to deliver code which does not meet the basic quality objectives of a given release.

When teams transition to Agile, the lack of a full testing cycle at the end of the release creates a substantial risk to the overall quality of a release unless the stories and features are tested at the end of the iteration. This is done through the proper defining and testing of acceptance criteria for the stories and requirements that are marked complete for a given iteration. Additionally, stories and features should be at a level of quality which meet the objectives for a given release. For highly successful Agile teams looking to evolve into test driven development, the need and ability to define of detailed acceptance criteria “the tests” is a prerequisite.


Is it possible to be half Agile?

July 5, 2008

With the all the hype in recent years regarding Agile software development, one does not have to look very far to find a team or organization claiming to be Agile when in fact their methodology is anything but Agile. Recently, I was discussing this phenomenon with a colleague. The conversation focused how Agile should be defined. What is the true test as to whether or not an engineering team is Agile? What does it mean to be Agile? I thought I would take a few minutes and share with you what it means to me.

There are many Agile-like characteristics which can be evaluated as to a team’s level of agility. However, I believe there are three key principles that must exist in order for a team to be Agile.

When your Agile project starts have you committed to a fixed set of features using a fixed set of resources to be completed on a fixed date? If the answer to this is “Yes”, then it is impossible to be Agile when making changes to the project scope to ensure a high-quality on time release. Sure to be controversial, it is my belief that projects with these three variables fixed simply cannot be Agile.

Does your project include a series of increments or iterations that require the completion of “releasable” or “working” software” (defined, developed, and tested) at the end of each increment? Completing working software in an incremental approach enables the team to re-evaluate the project scope at the end of each iteration. This is the foundation for having the ability to make decisions at the end of each iteration. If none of the features are complete and tested until the end of the project, it is too late to re-asses and ensure an on-time successful release.

Does your project allow for the team members to operate in an empowered fashion in which each team member has some ability to influence the methodology and engineering practices? One of the key principles in Agile is believing the people who manage and build the project day to day know the most about the changes that need to be made. If the team does not have the empowerment to self-adjust then there is no mechanism to be Agile.

While it is true most Agile methodologies call for much more than these three items, these represent (in my opinion) the foundation for operating in an Agile way. Adherence, to these key principles as one transitions to Agile is critical to the long term success of the transformation. In short, “Variability”, “Incremental Completion”, and “Empowerment” are the most critical Agile characteristics. All other Agile concepts support at least one of these. For example, automated testing enables incrementally releasable software.

So, is it possible to be half-Agile? It may very well be. However, projects not containing the three principles above will struggle to be successful with their transformation to this new and exciting methodology.