Buggy Thoughts for Today

I strongly feel that  adding ATDD to TDD is a good testing combination. Given the right focus from all stakeholders, it can drive the entire development process to come up with quality software products whose codes are testable, reliable, stable and easy to maintain. I don’t claim to be an expert on any QA methodology but I try to justify my testing strategy by adequate research, seeking advice from the testing community (online and offline) complemented by my own experience. This will be a series of journey stories to find out if we can truly combine Test Driven Development ( TDD) and Acceptance Test Driven Development (ATDD) to come up with a successful Software Application Project using Agile.

Let the journey begins …

Journey 1 :  Thinking about  ATDD + TDD

Once upon a time, writing a Test Plan  seems an easy task if you are on the same project for the same company for a long time. You just fill up a simple template and you can come up with a completed Test Plan within a day or two.

But this simple one day activity can turn into weeks of planning when you are assigned to a new client ( or company) or a new project. Thou there may be a Test Plan template , it is not as simple as it seems. The area of the Test Plan which I found to be the hardest to come up with is the “Testing Approach and Strategy”. This is where the QA has to describe what is the best way  to test the application. Any QA with years of experience will tell you that there is no single one QA process that fits all. This is reality but misunderstood and even mocked. You need to give a serious look into a couple of factors that would affect your choice of testing strategy. This article is not about this issue so I will skip this part.

What I would want to talk about  is how I came up with a Test Strategy for one project :

• I started studying the business model and the application to be developed for about 3 days. I gather as much information from the PMs, BAs, other QAs. I read every existing artifacts.
• Then I worked with the Lead Developer alone for another 3 days, brainstorming which approach is do-able given all the resources, time and other known constraints we have. We both believed that we need to “pair” so that the testing approach we picked will be something that we both feel comfortable using. I would give much of the credit to him (SK,thank you ) for analyzing each of the testing activities I suggested and giving his technical opinion. ( Yes, I am truly lucky to work with very brilliant developers who put so much value to testing …)
• On the 7th day, I finalized the first draft of the plan with a high level Testing Approach and Strategy.

The high level strategy is like this (I copied this verbatim from my Test Plan)

• Our Team has agreed to use the Agile methodology where QA Analysts (QA) work closely with Developers, Business Analysts (BA), Project Managers (PM) and other stakeholders all throughout the project life cycle. We will implement Test Driven Development (TDD), continuous integration and pair programming.
• In addition to TDD, to ensure full unit test coverage for developed code, we will also follow the process of Acceptance Test Driven Development (ATDD) to ensure full test coverage. TDD’s approach is technology facing while ATDD is business facing. Acceptance tests are written at the beginning of an iteration and becomes an executable form of the requirements.
• Simple Process Flow of  ATDD + TDD

So why  ATDD + TDD?

• The Development Team has been using  Agile already so everyone is familiar with the iterative development life cycle.
• TDD is now a generally accepted practice so the “Test First” concept is not hard to sell to the developers.
• ATDD strategy will logically complement the PM’s practice of translating requirements into Epic Stories then User Stories.
• Our team has previously agreed that we will rely heavily on Story Acceptance Criteria to mark any task “DONE”. The Story Acceptance Criteria are converted into Automated Acceptance Tests. The story cannot be accepted as complete until the automated acceptance tests for that story pass.
• Other non technical stakeholders like the Business Analysts and the Customers are able to participate in writing automated acceptance tests minus the technical barriers. The write Tests as “Specs” in human readable language.
• The use of ATDD ensures expectations are set around testing. There is no disconnect between what the QAs think should be tested and what the Business think should be tested.
• As each automated acceptance tests are added to the continuous build, a fully automated regression suite is developed as part of the development process.
• The fast feedback mechanism is tremendously beneficial to everyone. The business has visibility into what is tested at all stages of development.

What is ATDD and other concepts

• Common names : Acceptance Test Driven Development, Story Driven Development, Behavior Driven Development, Example Driven Development
• Definition : “ATDD is an advanced extension of TDD, keeping the software’s development guided by the principles of satisfying the customer’s needs through acceptance criteria and automated testing. “ This is so far the best definition of ATDD (1)
• Format of a typical  TDD + ATDD acceptance tests:

BDD  (What is needed)

Story: [ DESCRIPTION OF STORY]
As a [ROLE]
I want [FEATURE]
So that [MOTIVATION]

ATDD  (How is the story supposed to work to arrive at the expected outcome)

Scenario: [SOME DESCRIPTION OF SCENARIO]
Given [SOME CONTEXT]
When [SOME EVENT OCCURS]
Then [ENSURE SOME DESIRED OUTCOME]

• ATDD allows us to take the customer’s stories and scenarios and add them directly into the code
• ATDD is business facing ( software that matters) while TDD is technology facing
• ATDD uses the testing direction of Outside-In
• Our software application now has more “business life” which can easily evolve and change to adapt to unknowns in the future requirements.

Challenges of ATDD + TDD to QA

I would avoid using Pros and Cons in evaluating the  ATDD + TDD approach. I’d rather combine them as “challenges” which we might face, but to which we can always find a good solution.

• Management support is essential so we need to be prepared to justify our recommendation. If management is not willing to invest on ATDD, you can still be creative. I have some thoughts on how to go about this but I would reserve that in another post.
• ATDD is somewhat a newer concept so this needs to be explained to the Team before implementation.
• It requires practice and discipline to write acceptance tests especially in a very short iteration cycle of 2 weeks or less.
• During the first few iterations, the team may suffer from poor velocity. So set your initial expectations realistically. As everyone starts getting used to writing acceptance criteria and acceptance tests, the velocity for each story should also improve.
• More tests for the developers ( and QA ) to implement and maintain.
• Automation is a must. ATDD is not as successful if it is manually implemented.
• Lack of automation tools that would fit some applications under test. The most commonly used open source tools are FIT, FITNESSE, Cucumber, NBehave, StoryQ and EasyB. For this project, I have narrowed down the choices to : NBehave and StoryQ . I will let you know in another post which tool I picked.
• Again, the selection of the automation tool is crucial . One team I know had initial success using EasyB for almost one year. Eventually, the maintenance and the load it created on build  time seemed to  offset some of the positive returns earlier. I have not used EasyB myself, so please don’t take this statement as a bad review for this free tool.
• I highly suggest that the QA works closely with the developer during the testing tool selection. Try a couple, but don’t fall in love with the initial results. Think ahead. Consider refactoring, consider integration with other project streams. Consider the maintenance overhead.

Next, please follow our journey to the bottom of ATDD + TDD

• Journey 2 : Presenting  ATDD + TDD to the entire Project Team
• Journey 3 : First Iteration Implementation of  ATDD + TDD

Thanks everyone!

Tess Rupprecht

——————

References

(1) http://testdrivendeveloper.com/2008/03/14/ATDDAcceptanceTestDrivenDevelopment.aspx