Comments (5)

By Lynn McKee, November 19, 2010 12:40 pm

Test planning is an important element to all testing efforts although it appears in many forms. For some projects test planning is formal exercise focused on the development of a comprehensive, pre-emptive document labelled the Test Plan. For other projects, test planning is a dynamic, lightweight exercise constantly adapting to the changes in the project and organizational needs. Templates are often developed for test planning in an attempt to assure “best practices” and standardization. It is important to consider that this standardization is likely to restrict our ability to apply critical and creative thinking and to effectively respond to specific circumstances.

Test planning on many projects is challenging especially when it is tasked early in the project, with relatively little information, and with the expectation for the planning to be pre-emptive. In other words, testers are expected to foresee all of the risks, assumptions, constraints, dependencies, and quality issues in order to account for them. The problem is amplified by the sheer volume of documentation created and the need for stakeholder approval on a detailed plan that is a projection at best.

Templates are commonly used within organizations in their test planning practices. Let’s consider a few of the potential pros and probable cons with using test planning templates.

It is very interesting to consider the false securities extensive test planning templates generally offer. There is an intense desire from management to believe these tools add value and provide an assurance against poorly executed testing efforts. The reality is often a stark contrast to the perceived value. Am I suggesting there is no valid case for test planning templates? Absolutely not. However, it is very important to understand the value the templates are truly providing your teams and more importantly the limitations.

Many times I hear that templates are in place to serve as a guideline for test teams. While this is a great concept it would be much more effective to create an actual guideline instead. Perhaps a listing of key considerations that serves to educate not prescribe. The problem with templates is people often get “stuck” working within the existing headings and worrying about providing content so no section is left empty. This can restrict our ability to apply critical and creative thinking.

The other major challenge with test planning templates is the idea of “comprehensive planning”. It is as if by creating more sections there is an assurance that the planning will be more complete and concrete. Pre-emptive planning does not work because of what is known as the “Unknown Unknown”. Donald Rumsfeld elaborated on this by saying “There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.” In order to respond to evolving knowledge about the project and product under test, test planning needs to be adaptive and context specific.

With the diversity in organizations and projects, no two projects are ever the same. Templates and standardization can confine thinking and testers need to focus on effective, adaptive, context driven test planning. Take a step back from the templates and the objective remains to define a strategy for gathering insightful and timely, quality-related information for stakeholders.

Transition the focus of your test planning from the templates to understanding the following fundamental questions:

• Mission – Why are you testing? What are you trying to learn?
• Strategy – How should you organize your work to achieve your mission?
• Oracles – How will you know whether the program passed or failed the tests?
• Impossibility of Completeness – What would it take to do a complete testing job?
• Measurement – How much testing is enough?

* Fundamental Questions excerpted from the Black Box Software Testing Fundamentals course, by the Association for Software Testing.

Document your test planning in a lightweight, effective method that meets the needs of your organization and is adaptable. Changes in our understanding of the project’s context is not a matter if but when. Enable your strategy to adapt as your understanding of the project evolves and quality issues are revealed.

Adjusting your current test planning approach may require communication and education within your development organization on the benefits. Be prepared to articulate and deliver upon the increased value your testing effort will provide by being adaptive.

Additional Relevant Reading

• Five Orders of Ignorance by Philip G. Armour
• Unknown Unknown, Wikipedia