Collection of Notes & Experiences: Bangalore Workshop on Software Testing 1

Approved for Publication by all Participants

Theme: Changing the way people test and think about testing
May 2^nd, 2009
Venue Sponsor: Edista Testing Institute & Test Republic


Participants displaying their K - cards
For more pics, click here

Participants ( not in the o: Ajay Balamurugadas, Aishwarya D Shukla, Guruprasad, Manjunath, Rahul Verma, Rahul Mirakhur, Shrini Kulkarni, Manoj Nair, Ravisurya, Santhosh Tuppad, Shikhar Singh, Raghu Sahay, Sharath Byregowda and Pradeep Soundararajan

Bangalore Workshop on Software Testing officially started with the announcement on my blog on April 12, 2009. We had about 15 people rushing in their registration and abstracts for presentation. When you see the list above, you would know who finally made it.

We tried following the LAWST style peer conference and we think we did. I played the role of host and facilitator. I am sure the next time someone else will play the facilitation. I learned facilitation by observing Paul Holland do it in TWST and CAST. He is an amazing tester and facilitator. We used K-cards and that really worked. You would see in photograph, people holding K cards.

We had a check in and introduction (as there were a few faces who did not know each other) at 9:30 AM followed by Ajay kicking off the first presentation.

Ajay’s presentation: Rapid Software Testing

Ajay talked about the change he had after attending my workshop on Exploratory Testing – A Rapid Software Testing Approach and how it helped him log several hundreds of bugs when he started doing a structured exploratory testing. He explained how he struggled with the templates that he had to adhere and how those test cases filled in templates failed to yield coverage and value.

Ajay brought in an interesting thing: “When I started to log more bugs, people were skeptical of how I was doing it. They used to think I was stocking them and releasing it as required”. In this case, Ajay funded himself for the training.

It surprises me that some organizations get their resources trained to help changes to occur to their work and grow skeptical when they get to witness the change.

Before he finished his presentation, he mentioned how the management started seeing Ajay’s work output as a different value than what they expected out of him. What they expected from Ajay was to run as many test cases as possible and what he ended up producing was finding defects and reporting them in their bug tracking system. That is an interesting one – test cases are there, wherever they are, to help testers find bugs and some managers might be misusing it to show progress of testing. I have heard it happens everywhere in the world and not just in India.

During the facilitated discussion that happened after Ajay’s presentation, Shrini mentioned, “The value depends on what stakeholders defines it as”. However, sometimes some stakeholders might need education from a tester to have more meaningful value definition.

Rahul Mirakhur (RM) brought in the idea of a tester wearing different hats and being able to think the value that different stakeholders might want from a tester.

When he interacted with Support teams in an earlier organization, it opened up a completely new world of information source and that, changed the way he did testing thereafter. It also gave a glimpse on how different functions on a project bring in value and in the case of Support teams case, value is more "after" product release.

As the discussion drifted to bugs, Rahul Verma said, “I see bugs as questions we can ask”

Manoj Nair’s presentation on An attempt for better testing

Manoj talked about the opportunity he got in his project where time crunch and amount of testing to be done, enabled his manager to bestow him with a freedom to do free style exploratory testing (without much documentation) for an important feature in his project. The defects and the information he came up by performing free style exploratory testing made the management to postpone the release realizing the danger. He felt happy with his effort, until he was required to explain the details of test coverage across the feature and provide test case document for the tests he conducted for those features. The test case document he prepared were subset of the entire range of tests he performed because he did not have time to document all those tests. He realized the value of tests he did could possibly diminish through that activity. Looking back at it he felt, if he had used Session Based Test Management, he could had done a much better job in providing test ideas document.

During the discussion, I talked about the idea of a Wiki for changes to the product open to all team members and the value it could have for the test team.

Aishwarya, an undergraduate student asked a striking question, “How much information would be an overflow of information?”

The discussion then digressed to – When is the right time to get testers on the project for which all participants had different ideas and experiences. Many organizations have tried bringing in testers early and late and most of them are not yet sure when to bring them in.

I argued that it depends on the kind of a tester an organization is trying to bring in and other factors. Not all testers are the same. I have witnessed organizations that actively ignore testers in requirements review meeting as they have had a bad experience of bringing in a tester. It makes me think that if I lose credibility, it affects other team members in my test group. That is how important it is for each team member to have and build credibility.

Manjunath’s Presentation of Review of Review of Review of Bug Reports

He talked about his experience in one of his previous organization in the context of bug report reviews conducted to help testers log better and credible bug reports. This exercise was undertaken based on the success of improved bug report quality, post a review of each bug reported from the previous project.

Interestingly, in the second project there was another level of review that happened over the review comments of bug reports. The tester, whose bug reports were review had an option to appeal for escalation if he found the review comments did not fit his (her) bug report. It got many testers to defend each of the review comment of the bug reports they logged. Thus spending more time on a single bug report, instead of finding more bugs. That's opportunity cost. That’s like review of review of reviewed bug reports. Yes, we did have a good laugh!

This process was then declared a failure for the second project as its implementation went wrong and 
ceased to exist thereafter, maybe. 

The process can become so worse that it might eat so much time for so little value. On probing, Manjunath revealed that the review process was actually initiated to help testers write better bug reports. All participants were convinced that the way in which the mission was executed might have been poor and what works in one context does not fit another. I am sure we are going to remember Review of Review of Review of Bug Reports experience report.

Doesn’t it help us understand that the best practices do not work for another project within the same organization because the context keeps changing?

During the discussion, we discussed ways of credible bug reporting and shared our experiences of reporting bugs through Video recording of the bug and the value it has had for people who experimented that. It cuts across communication barriers and I am surprised why this practice is not widely used. Oh! 

Someone hasn't talked about it as "best practice" yet? It then digressed to how much English should testers know and then RM talked about practicing English as a daily activity as opposed to a one time upgrade of English skills.

He provided inferences that it’s (relatively) harder to learn language (or command over it) when we get old compared to when we are young. Shrini mentioned that it is not a "rule" by citing his personal experience of how despite being relatively old (compared to most of the audience), he is constantly on the lookout to learn new things.

We got back to reviewing and then discussed on reviewing as a skill and review effectiveness.

Lunch Time folks!

At about 1:15 PM, we walked to a nearby hotel down the street and carried over facilitation-less discussions over lunch table. A heavy lunch

Then when we returned, something helped us get the heat back… Rahul Verma’s presentation on Confessions of a Fallible Tester

This was a very energetic presentation we had for that day and thanks to Rahul Verma, he did it right at the time it was needed ( post lunch )

He made confessions of the mistakes he did right from the start of the career where he thought testing was not his cup of tea. He just did not confess but also explained how he learned from those mistakes and lived better after each big mistake.

One of his confession sounded very cool - “When senior testers in the team said XXXX XXXXXX was the best tool, I believed it” and then he explained how the project suffered because of that tool or the idea of thinking that as THE BEST Tool.

He explained how he cleared that trap and ended up writing a customized plug in to work with XXXX XXXXXX to make it suitable for his testing context.

He then confessed on another mistake of how two blank lines that caused the test environment to fail to enable the product to be tested over it. He or his team members make turned the case by defocusing the customer to something else when the customer observed it. The message was – Test your Test Environment. I think those who do that today are the ones who have realized the value of it. Many testers just setup and start running their test cases without bothering if test environment is what they intended it to be.

The discussion kicked off and we started discussing about tools. The topics varied from Automation Readiness to Vendors bribing managers to sell their tools to Good code versus Measurable Code.

From Rahul Verma to Rahul Mirakhur’s presentation on My career and experiences in testing

He started with a cool question, "When people say WHY will you want to be a software tester, I ask WHY NOT" and I think that was exciting. RM had some interesting observations about IT people in India – “By the time people realize what they are good at they would be in some other domain and would have beefed up living a luxurious life that doesn’t allow them to shift to something they want to do”

He asked another question “Anyone can be trained on software BUT what can they really do?” and discussed that every human is a born explorer and testing is an innate thing and can be developed by proper training.

He then talked about his experience of witnessing developers who moved to testing because they were interested in testing. He then said based on his experience, “Jump into development first if you are serious about becoming a tester”. He shared the experience and value of hiring a person who earlier worked in a support job.

The discussion post this presentation helped in understanding how many different ways are there to get into testing and the value of taking each way. We also talked about the value of learning testing from other fields and that reminded me of CAST2008 conference theme and presentations from there.

RM went back from BWST and started a blog and has promised to keep posting. Another credible blog from an Indian tester on the way - check out.

Sharath Byregowda’s experience of Session Based Test Management

Sharath talked about his experience of experimenting SBTM on the project he worked for and how he influenced other testers, the management and customers with it. He has detailed it in his blog post and I am sure you'd read go there and read it or maybe even learn from it. His team won an award for achieving great results in his organization that has thousands of testers. In addition, Sharath got another award from Michael Bolton.

We had an interesting comment that - By the time an organization implements some new practice as a "best practice" and penetrate every team, the idea is outdated. It reminded me of Ted Neward’s presentation at Oredev08 where he talks about the time difference of programming language birth to becoming an industry usage.

Shrini Kulkarni’s experience of test automation consulting

Shrini talked about his experience of becoming a tester and then about how he grew from being a fresher in software testing to wherever he currently is. He then talked about an interesting experience of Test Automation Readiness Assessment that he was supposed to do for a client in North America and the challenges he had. Shrini talked about his confidence in doing it although his colleagues thought of it as an impossible task. Shrini detailed the value of collaborating with people like Michael Bolton, James Bach, Cem Kaner and others that have helped him evolve to a Test Automation Expert.

 Shrini also talked about his consulting experience and mentioned that almost all test consulting assignments he witnessed, the client was interested in knowing some "universal" or industry standard benchmark related to testing - be it dev to test ratio or % of project cost that could be associated to testing. His experience was that he rarely witnessed anyone challenging those industry standards. In his opinion - That is hindrance for our profession.

Discussions on Test Automation, mindset and skill set followed Shrini’s presentation until the bell rang at 5:30 PM

We had a quick check out and got on to the socializing evening and facilitation-less discussions. We decided on a Food Court in Koramangala and spent about 3 hours there munching food and discussions on Cricket, Movies, Testing, Food, Beer and Wine.

What a fantastic learning to all of us. I must thank Shikhar Singh and Raghu Sahay for coming down from Mumbai and Pune to attend BWST-1. It was fantastic to note that Aishwarya Shukla, an undergraduate student, wanting to do a software testing project as his final year project after attending BWST along with Santhosh Tuppad who completed Practical Software Testing Training at Edista and has been working on several testing assignments as a freelancer. Every participant asked questions that helped other people learn and discover more about themselves and the work they do.

Once again, special thanks to Mohan Panguluri, Pradeep Chennavajhula and Rachna for providing us the space @ Edista Testing. I wish India has more leaders like them who facilitate knowledge sharing and learning in software testing without just being business minded.

Shikhar went back to Mumbai and wrote an e-mail to me regarding Mumbai Workshop on Software Testing that he is planning. Watch out India!

Here is the list of Post It Notes that all participants contributed as an additional reference points to the discussions we had:

• Only testers can talk about Testability
• Book suggestion: Perfect Software and other Illusions about Testing – Jerry Weinberg
• “Every bug is a duplicate of a master bug that the product fails to do something that is important to work for someone” – if that sounds bad - that's still me.
• Philosophy in Golf & Testing – Adam Goucher
• Bug Advocacy – Cem Kaner
• How to investigate Intermittent Problems – James Bach blog post
• “The best tester is the one who gets the most the right bugs fixed” – Cem Kaner in Bug Advocacy
• Do confess when you screw up
• Turning Numbers into Knowledge – A book by Jonathan Koomey
• Lessons Learned in Software Testing – Kaner, Bach, Pettichord
• “Automation could be a dangerous word for a tester” – Rahul Verma
• “If you have failed big, it indicates you rose to a great height” – Rahul Verma
• Value of Checklists – Cem Kaner’s presentation in CAST 08
• Video Record Your Bugs
• The intake I have had from BWST-1, has questioned what I have learned so far. It has helped me to unlearn and learn consistently - Ravisurya
  
• "When I spoke to support teams, it changed the way I tested, forever" - Rahul Mirakhur “
• After I attended Rapid Software Testing Workshop, my testing improved” – Ajay Balamurugadas
• “The value you add as a tester is like NAV of an investment. It changes every day” – Shrini
• General Systems Thinking – A book by Jerry Weinberg