"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Friday, May 29, 2009

Testing Experience Magazine : The voice of ISTQB

I agree you need money to run a business but then...

If you are subscribed to Testing Experience - The magazine for professional testers, you would have got an e-mail recently ( a day or two before this post ) announcing first year anniversary of the magazine. I am subscribed to them from their third issue and in an year they have grown to about 350,000 subscribers ( as per their own website claim )

I congratulated Jose Diaz for running the magazine and for completing one year with it, and offered my respect to him for doing it. I also wrote in the same e-mail to him that I see the magazine becoming too much of ISTQB stuff to which he seemed to disagree with me. I am surprised with that.

I decided to investigate if I could be wrong and presented my findings to Jose over an e-mail:
( modified the construct for your reading and not the content)

Here is my experience of going through Testing Experience magazine
  • When I open the website www.testingexperience.com I am forced to look at a pop up ad for ISTQB. Only after closing that I am allowed to browse through the pages. [ As on May 29, 2009 2300 IST ] Do you see anything different?
  • Let's look into the latest issue : Security testing: Page: 2 : The opening ad of Testing Experience is ISTQB. It kind of puts in my mind the thing that Testing Experience means ISTQB.
  • Page 3: Jose Diaz has personally covered ISTQB in letter from Editor section and he says, "Last but not least, I want you to pay attention to our new e-learning portal www.testingexperience.learntesting.com . You can register for ISTQB Foundation Level and very soon for Advanced Level. Enjoy Learning". When the chief says pay attention to something it indicates to me that Testing Experience is supporting ISTQB because they are feeding Testing Experience with enough money to do that.
  • Page 6: Yet another ad of ISTQB in German language or something similar to that. Hey look, before I even get to read the first article in a magazine, I am asked to cross 4-5 instances of ISTQB.
  • Page 13: Another instance of ISTQB half page attractive ad
  • Page 21: ISTQB again
  • Page 24: Erik Van Veenendaal writes about ISTQB and wooing people to go for it. Here is an excerpt from that article: "The ISTQB Advanced Level scheme offers a complete 5-day module on technical testing which may seem irrelevant to testers today, but I'm convinced will become more and more important to the near future or even already today".
  • Page 27: Interview with Mike Smith: Yet another ISTQB masala flavor in the form of LearnTesting. 3 pages full of that. As though - Learn Testing means only about learning ISTQB way of testing.
  • Page 30: Yet another ad for Learn Testing who in turn promotes ISTQB and run by Testing Experience.
  • Page36: An article from Rex Black, a popular founder and face of ISTQB. To his credit, he doesn't talk about ISTQB.
  • In what way is his article on page 66 to 70 that talks about Advanced Software Test Design Techniques is related to Security theme of the magazine? Do we have so less people on Security Testing that we'd have to compensate it by a non security article.
  • At least aren't there any other brilliant ISTQB certified testers who can write good articles on Security Testing or maybe even the Advanced Test Case Design Techniques?
  • Page 52 & 53 has the ISTQB and information such as "We are the community with 1,10,000 people. Are you on the right side?". Its a Shame. Rex Black speaks against the idea of Schools of Testing but has ads asking testers, Are you on the right side? with a link to ISTQB below it. What is the message?
  • Page 58: ISTQB in France
  • Page 70: Another article from Rex Black
  • Page 78: RBCS & ISTQB
  • Page 99: There is ISTQB, yet again and a full fledged article on it for 2 pages.
  • Page 103: Is yet another mention about ISTQB being there and being here for those who didn't know it will soon be everywhere .
  • Oh and Page 104 also ends with ISTQB training and other details. ( There are only 104 pages of this magazine )
  • Compare their first edition of magazine and there on, you would see ISTQB increasing with every edition. I wouldn't be surprised that the future editions is only about ISTQB and I see the future in their latest edition.

Testing is Questioning

Where is all the money that many testers across the globe paying to ISTQB certification and training going? Asks Michael Bolton on James Bach's Blog

Is the magazine becoming an official mouth piece of ISTQB or there is so much of bias that it makes me think so?

Whose money is all this? Your money. You didnt know you can make someone as rich as you wanted to be?

I think anyone who is running a business can cling to a brand to build their own brand but not to an extent that the other brand swallows it. If that is the mission, I think Testing Experience is headed on the right track.

I don't know who is in the review board and I suspect that it might not be diversified and have only ISTQB promoters.

What do you think?
  • So, you reader of this post - What is your opinion? After seeing all this would you think the magazine is neutral or is serving the purpose of promoting ISTQB?
  • Should magazines like these that claim to be for Professional Testers be promoting one kind of stuff or be helpful to the community in allowing different kinds of people to mix with?

Test Republic is a social community for software testers and a not for profit service by Mohan Panguluri, COO, Edista Testing Institute. You can witness that by visiting it. It has events of ISTQB posted in the past, it has CAST conference details and QAI conferences. It has a blend of all kinds of thought process in software testing and arguments. They allow all kinds of information to be posted from all kind of communities and schools. They keep aside their business and do this for benefiting the community. Another example is STC by Rosie Sherry. I think there is lots to learn from people like Mohan Panguluri and Rosie Sherry, for the rest of the world who also claims to be doing it for Professional testers.

The philosophy

Jose seemed to be OK with my above response of facts about ISTQB in Testing Experience's latest issue but wasn't comfortable after I mentioned to him that I plan to blog about it. He cited that his lawyers might have a look into my blog if I do so.

So they might be reading this post along with you. If I question ISTQB, the dynasty of ISTQB will hire lawyers to see if I can be attacked for questioning them?

How cheap? Software Testing is such a nascent field, we haven't even crossed 50 years and already someone is saying "The best practice and right ways of testing" and "we control the field and there is no room for disagreement".

Kudos ISTQB certified testers. You are sinking the field you are working on without knowing you are doing it. Fantastic, go ahead get certified!

Tuesday, May 19, 2009

Collection of Notes & Experiences: Bangalore Workshop on Software Testing 1

Approved for Publication by all Participants

Theme: Changing the way people test and think about testing
May 2nd, 2009
Venue Sponsor: Edista Testing Institute & Test Republic

Participants displaying their K - cards
For more pics, click here
Participants ( not in the o: Ajay Balamurugadas, Aishwarya D Shukla, Guruprasad, Manjunath, Rahul Verma, Rahul Mirakhur, Shrini Kulkarni, Manoj Nair, Ravisurya, Santhosh Tuppad, Shikhar Singh, Raghu Sahay, Sharath Byregowda and Pradeep Soundararajan

Bangalore Workshop on Software Testing officially started with the announcement on my blog on April 12, 2009. We had about 15 people rushing in their registration and abstracts for presentation. When you see the list above, you would know who finally made it.
We tried following the LAWST style peer conference and we think we did. I played the role of host and facilitator. I am sure the next time someone else will play the facilitation. I learned facilitation by observing Paul Holland do it in TWST and CAST. He is an amazing tester and facilitator. We used K-cards and that really worked. You would see in photograph, people holding K cards.
We had a check in and introduction (as there were a few faces who did not know each other) at 9:30 AM followed by Ajay kicking off the first presentation.
Ajay’s presentation: Rapid Software Testing

Ajay talked about the change he had after attending my workshop on Exploratory Testing – A Rapid Software Testing Approach and how it helped him log several hundreds of bugs when he started doing a structured exploratory testing. He explained how he struggled with the templates that he had to adhere and how those test cases filled in templates failed to yield coverage and value.
Ajay brought in an interesting thing: “When I started to log more bugs, people were skeptical of how I was doing it. They used to think I was stocking them and releasing it as required”. In this case, Ajay funded himself for the training.

It surprises me that some organizations get their resources trained to help changes to occur to their work and grow skeptical when they get to witness the change.

Before he finished his presentation, he mentioned how the management started seeing Ajay’s work output as a different value than what they expected out of him. What they expected from Ajay was to run as many test cases as possible and what he ended up producing was finding defects and reporting them in their bug tracking system. That is an interesting one – test cases are there, wherever they are, to help testers find bugs and some managers might be misusing it to show progress of testing. I have heard it happens everywhere in the world and not just in India.

During the facilitated discussion that happened after Ajay’s presentation, Shrini mentioned, “The value depends on what stakeholders defines it as”. However, sometimes some stakeholders might need education from a tester to have more meaningful value definition.

Rahul Mirakhur (RM) brought in the idea of a tester wearing different hats and being able to think the value that different stakeholders might want from a tester.

When he interacted with Support teams in an earlier organization, it opened up a completely new world of information source and that, changed the way he did testing thereafter. It also gave a glimpse on how different functions on a project bring in value and in the case of Support teams case, value is more "after" product release.

As the discussion drifted to bugs, Rahul Verma said, “I see bugs as questions we can ask”

Manoj Nair’s presentation on An attempt for better testing

Manoj talked about the opportunity he got in his project where time crunch and amount of testing to be done, enabled his manager to bestow him with a freedom to do free style exploratory testing (without much documentation) for an important feature in his project. The defects and the information he came up by performing free style exploratory testing made the management to postpone the release realizing the danger. He felt happy with his effort, until he was required to explain the details of test coverage across the feature and provide test case document for the tests he conducted for those features. The test case document he prepared were subset of the entire range of tests he performed because he did not have time to document all those tests. He realized the value of tests he did could possibly diminish through that activity. Looking back at it he felt, if he had used Session Based Test Management, he could had done a much better job in providing test ideas document.

During the discussion, I talked about the idea of a Wiki for changes to the product open to all team members and the value it could have for the test team.

Aishwarya, an undergraduate student asked a striking question, “How much information would be an overflow of information?”

The discussion then digressed to – When is the right time to get testers on the project for which all participants had different ideas and experiences. Many organizations have tried bringing in testers early and late and most of them are not yet sure when to bring them in.

I argued that it depends on the kind of a tester an organization is trying to bring in and other factors. Not all testers are the same. I have witnessed organizations that actively ignore testers in requirements review meeting as they have had a bad experience of bringing in a tester. It makes me think that if I lose credibility, it affects other team members in my test group. That is how important it is for each team member to have and build credibility.

Manjunath’s Presentation of Review of Review of Review of Bug Reports

He talked about his experience in one of his previous organization in the context of bug report reviews conducted to help testers log better and credible bug reports. This exercise was undertaken based on the success of improved bug report quality, post a review of each bug reported from the previous project.
Interestingly, in the second project there was another level of review that happened over the review comments of bug reports. The tester, whose bug reports were review had an option to appeal for escalation if he found the review comments did not fit his (her) bug report. It got many testers to defend each of the review comment of the bug reports they logged. Thus spending more time on a single bug report, instead of finding more bugs. That's opportunity cost. That’s like review of review of reviewed bug reports. Yes, we did have a good laugh!

This process was then declared a failure for the second project as its implementation went wrong and 
ceased to exist thereafter, maybe. 

The process can become so worse that it might eat so much time for so little value. On probing, Manjunath revealed that the review process was actually initiated to help testers write better bug reports. All participants were convinced that the way in which the mission was executed might have been poor and what works in one context does not fit another. I am sure we are going to remember Review of Review of Review of Bug Reports experience report.

Doesn’t it help us understand that the best practices do not work for another project within the same organization because the context keeps changing?
During the discussion, we discussed ways of credible bug reporting and shared our experiences of reporting bugs through Video recording of the bug and the value it has had for people who experimented that. It cuts across communication barriers and I am surprised why this practice is not widely used. Oh! 

Someone hasn't talked about it as "best practice" yet? It then digressed to how much English should testers know and then RM talked about practicing English as a daily activity as opposed to a one time upgrade of English skills.

He provided inferences that it’s (relatively) harder to learn language (or command over it) when we get old compared to when we are young. Shrini mentioned that it is not a "rule" by citing his personal experience of how despite being relatively old (compared to most of the audience), he is constantly on the lookout to learn new things.

We got back to reviewing and then discussed on reviewing as a skill and review effectiveness.

Lunch Time folks!

At about 1:15 PM, we walked to a nearby hotel down the street and carried over facilitation-less discussions over lunch table. A heavy lunch

Then when we returned, something helped us get the heat back… Rahul Verma’s presentation on Confessions of a Fallible Tester

This was a very energetic presentation we had for that day and thanks to Rahul Verma, he did it right at the time it was needed ( post lunch )

He made confessions of the mistakes he did right from the start of the career where he thought testing was not his cup of tea. He just did not confess but also explained how he learned from those mistakes and lived better after each big mistake.

One of his confession sounded very cool - “When senior testers in the team said XXXX XXXXXX was the best tool, I believed it” and then he explained how the project suffered because of that tool or the idea of thinking that as THE BEST Tool.

He explained how he cleared that trap and ended up writing a customized plug in to work with XXXX XXXXXX to make it suitable for his testing context.

He then confessed on another mistake of how two blank lines that caused the test environment to fail to enable the product to be tested over it. He or his team members make turned the case by defocusing the customer to something else when the customer observed it. The message was – Test your Test Environment. I think those who do that today are the ones who have realized the value of it. Many testers just setup and start running their test cases without bothering if test environment is what they intended it to be.

The discussion kicked off and we started discussing about tools. The topics varied from Automation Readiness to Vendors bribing managers to sell their tools to Good code versus Measurable Code.
From Rahul Verma to Rahul Mirakhur’s presentation on My career and experiences in testing

He started with a cool question, "When people say WHY will you want to be a software tester, I ask WHY NOT" and I think that was exciting. RM had some interesting observations about IT people in India – “By the time people realize what they are good at they would be in some other domain and would have beefed up living a luxurious life that doesn’t allow them to shift to something they want to do”
He asked another question “Anyone can be trained on software BUT what can they really do?” and discussed that every human is a born explorer and testing is an innate thing and can be developed by proper training.

He then talked about his experience of witnessing developers who moved to testing because they were interested in testing. He then said based on his experience, “Jump into development first if you are serious about becoming a tester”. He shared the experience and value of hiring a person who earlier worked in a support job.

The discussion post this presentation helped in understanding how many different ways are there to get into testing and the value of taking each way. We also talked about the value of learning testing from other fields and that reminded me of CAST2008 conference theme and presentations from there.
RM went back from BWST and started a blog and has promised to keep posting. Another credible blog from an Indian tester on the way - check out.

Sharath Byregowda’s experience of Session Based Test Management
Sharath talked about his experience of experimenting SBTM on the project he worked for and how he influenced other testers, the management and customers with it. He has detailed it in his blog post and I am sure you'd read go there and read it or maybe even learn from it. His team won an award for achieving great results in his organization that has thousands of testers. In addition, Sharath got another award from Michael Bolton.

We had an interesting comment that - By the time an organization implements some new practice as a "best practice" and penetrate every team, the idea is outdated. It reminded me of Ted Neward’s presentation at Oredev08 where he talks about the time difference of programming language birth to becoming an industry usage.

Shrini Kulkarni’s experience of test automation consulting

Shrini talked about his experience of becoming a tester and then about how he grew from being a fresher in software testing to wherever he currently is. He then talked about an interesting experience of Test Automation Readiness Assessment that he was supposed to do for a client in North America and the challenges he had. Shrini talked about his confidence in doing it although his colleagues thought of it as an impossible task. Shrini detailed the value of collaborating with people like Michael Bolton, James Bach, Cem Kaner and others that have helped him evolve to a Test Automation Expert.

 Shrini also talked about his consulting experience and mentioned that almost all test consulting assignments he witnessed, the client was interested in knowing some "universal" or industry standard benchmark related to testing - be it dev to test ratio or % of project cost that could be associated to testing. His experience was that he rarely witnessed anyone challenging those industry standards. In his opinion - That is hindrance for our profession.

Discussions on Test Automation, mindset and skill set followed Shrini’s presentation until the bell rang at 5:30 PM

We had a quick check out and got on to the socializing evening and facilitation-less discussions. We decided on a Food Court in Koramangala and spent about 3 hours there munching food and discussions on Cricket, Movies, Testing, Food, Beer and Wine.

What a fantastic learning to all of us. I must thank Shikhar Singh and Raghu Sahay for coming down from Mumbai and Pune to attend BWST-1. It was fantastic to note that Aishwarya Shukla, an undergraduate student, wanting to do a software testing project as his final year project after attending BWST along with Santhosh Tuppad who completed Practical Software Testing Training at Edista and has been working on several testing assignments as a freelancer. Every participant asked questions that helped other people learn and discover more about themselves and the work they do.

Once again, special thanks to Mohan Panguluri, Pradeep Chennavajhula and Rachna for providing us the space @ Edista Testing. I wish India has more leaders like them who facilitate knowledge sharing and learning in software testing without just being business minded.
Shikhar went back to Mumbai and wrote an e-mail to me regarding Mumbai Workshop on Software Testing that he is planning. Watch out India!

Here is the list of Post It Notes that all participants contributed as an additional reference points to the discussions we had:
  • Only testers can talk about Testability
  • Book suggestion: Perfect Software and other Illusions about Testing – Jerry Weinberg
  • “Every bug is a duplicate of a master bug that the product fails to do something that is important to work for someone” – if that sounds bad - that's still me.
  • Philosophy in Golf & Testing – Adam Goucher
  • Bug Advocacy – Cem Kaner
  • How to investigate Intermittent Problems – James Bach blog post
  • “The best tester is the one who gets the most the right bugs fixed” – Cem Kaner in Bug Advocacy
  • Do confess when you screw up
  • Turning Numbers into Knowledge – A book by Jonathan Koomey
  • Lessons Learned in Software Testing – Kaner, Bach, Pettichord
  • “Automation could be a dangerous word for a tester” – Rahul Verma
  • “If you have failed big, it indicates you rose to a great height” – Rahul Verma
  • Value of Checklists – Cem Kaner’s presentation in CAST 08
  • Video Record Your Bugs
  • The intake I have had from BWST-1, has questioned what I have learned so far. It has helped me to unlearn and learn consistently - Ravisurya
  • "When I spoke to support teams, it changed the way I tested, forever" - Rahul Mirakhur “
  • After I attended Rapid Software Testing Workshop, my testing improved” – Ajay Balamurugadas
  • “The value you add as a tester is like NAV of an investment. It changes every day” – Shrini
  • General Systems Thinking – A book by Jerry Weinberg

Monday, May 04, 2009

Video : Against Best Practices in Software Testing :: Oredev 08

Here is my talk at one of the coolest conferences: Oredev, Malmo, Sweden, November 2008.

Great* testing stories from India
( Created by *not* following any best practices )

Highlights of the talk :
  • I was fired from Motorola trying to follow James Bach's advice
  • Test cases make people mad and sad.
  • Sharing 230 stories in 45 minutes.
  • The value of doing things that that context demands.
  • Questioning things that happens in India.
  • Truth of how many organizations get CMMi and ISO certifications.
  • Independence day for software testers
  • Brain is also a tool used in software testing, consider recognizing it.
Blogs about the talk so far:
I thought of making the video private instead of public and wrote to Michael Tiberg and he pointed out that this talk had received high rating from the audience and that I nailed some important points. I then decided to let it continue to be in public mode. I hope you'd enjoy the talk. Thanks to Michael Tiberg and Emily Holweck for putting up a great conference.

Oh you should watch James Bach's power packed keynote on Renaissance in Software Testing at Oredev. It left me crying for a couple of minutes.