"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Wednesday, July 30, 2008

Most test cases fail

  • Most testers who write test cases carry a hope that most test cases they write would help in finding most bugs.
  • Most testers who execute them carry a hope that most test cases they execute would help in finding most bugs.
  • Most testers who add more test cases carry a hope that they will help in finding most bugs.
  • Most customers insist on most of the testing to be done from test cases.
  • Most managers who ask testers to write test cases carry a hope that their testers will catch most bugs with the help of test cases.
  • Most managers who asked them to write test cases ask a question: How many test cases passed? when considering to ship.
  • Most testers in those contexts reply, "Most".

"If a test case didn't help in finding a bug then the purpose of that test case failed. Products are shipped when most test cases fail their purpose of existence."

------------------------------------------------------------------------------------------------------------------------------------------

  • Test cases fail to help most testers who write and execute them realize that testing is not a monotonous job as they are doing.
  • Test cases fail to help most testers realize that there are other ways to achieve what they are trying to achieve or there are other ways to do much better testing.
  • Test cases fail to help most managers understand that the number of test cases passed is a misleading metric because most test cases means few documented tests from a possible hundred billion tests.
  • Test cases fail to help most customers understand that there are ways to gain more value for the cost they are paying.
  • Test cases fail to help testers earn more money because in scripted testers view - an expert tester is one who writes a lot of test cases and executes a lot of test cases.
Test cases are human ideas and all human ideas can fail. Most testers who write and execute it fail to realize it.

If human ideas to test can't fail so will the software, not fail. Don't test.

------------------------------------------------------------------------------------------------------------------------------------------

To participate ( or just enjoy ) in a discussion involving exploratory and scripted testers on test cases , click here .

-- Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --


Update: The comments posted as Angel and Devil are by me ( Pradeep Soundararajan ). Wondering why I posted comments on the post I wrote - you would know the answer if you go through those comments.

The Angels and Devils concept in Testing is inspired by a presentation from Michael Bolton and Jonathan Kohl . Here is an excerpt of the presentation and also illustrates Elizabeth Kohl's design skills.

Monday, July 21, 2008

Heroes of software testing - Do you know about them and their work?

My initial plan was to board the flight to Toronto on July 11th from Mumbai to attend CAST 08 software testing conference. Sangeeta, a tester from Mumbai influenced me to change my plan and organized a lecture in Ness Technologies in Mumbai.

I liked the offer and accepted it, as I knew I would enjoy the experience. It's fun to meet new testers, talk to them and learn new things. I am in constant search for budding testers like Sharath Byregowda , Ajay Balamurugadas , Sathish Kumar Chinappa , Sangeeta, Sandesh , Girija, Bhargavi and Nattu ( who accompanied me to CAST 08 ) and those who are passionate and do things about seeing a better tester community like Mohan Panguluri, COO of Edista Testing ... I never know where I can find similar ones. One of the ways that has been of great help to me, to find such testers and thinkers is, while at lectures and my workshops. I offer whatever help I am capable of doing to expose them to good stuff and I do it when they ask me for it.


Here is an excerpt from the talk:

"We are in India, the land where cricket is considered close to being a religion. It has super heroes like Sachin Tendulkar and Dhoni who have inspired a lot of people in India to play better cricket. They could inspire because a lot of people spent their time watching them play great cricket at tough situations. As Indians, you might have seen a lot of people who enjoy playing cricket the way Sachin and Dhoni play. As testers, have you seen anyone doing testing that has inspired you to test better?"

Response: Silence

and then a question comes up from a corner, "Who did Sir Don Bradman see as an inspiration for he is the best known cricketer of centuries?" and all other testers burst to laughter thinking that I was cornered with that question.

I took a while for the audience to settle down and replied,
"I know who inspired Sir Don Bradman" and then a pause to add, "It was himself."
"How many of you are inspired by your own testing?"


Response: Silence

End of excerpt




Reading good stuff as a bad practice and reading bad stuff as the best practice

The above state is what most Indian testers to my knowledge are stuck in. This state resulted because most of testers don't have the habit of reading as though its a bad practice. Most of those who read, are the ones who search in Google and land up at bad sites ( that includes my blog at times ). Most of those who Google search are the ones who are not interested at better thought process but at ready made best practice answers.

There are good and great stuff from internet. I once searched for expert testers and found James Bach. That google search changed my life and the way I test. There were a lot of junk stuff written to attract such search results that I had to pass through before finding James Bach's website. I was sure that I needed help about thought process from an expert tester and not ready made solutions from time to time.

The road to Toronto

I had to face a lot of trouble to get to Toronto. My Visa was rejected ( at the first attempt ), I didn't have sufficient funds ( as per the VISA officials and my bank statement ) , I was denied permission to board the plane as my ticket was booked via United States and I didn't have a US Visa. The Canadian customs had a discussion that bothered me if they would let me in after I landed at Toronto and then my baggage went missing and finally landed as the last one on the belt. Amidst all this was my hope to meet the heroes in real space who have inspired me and to meet those whom I can draw more inspiration from. I believe in God and also believe that God takes a human form to help humans. A great human who lives in 61, Ashburnham in Toronto and the one who lived in my house ( my dad ) helped me and cleared traps so that I reached Toronto and was able to attend CAST 08.


Chasing dreams

Finally I got a chance to meet the heroes who have been inspiring me to test and think better. My super hero James Bach wasn't present at the conference and hence I couldn't meet him. I would be meeting him in November at a conference where we both are invited to speak on testing at a developers conference in Malmo Sweden.

By meeting my heroes, I learned some important things that I could not have learned, had I not met them. I got an insight into the way they live, some ways they think and they communicate. It was several dreams of several years that came true, all in one place - CAST 08.

Not all learning is fun.


If you have understood what I have written in this post so far, then my English writing skill has improved a lot since I wrote my first blog post. It pained a lot to know that I was writing terribly bad English but had I not learned it, I wouldn't have been able to make sense to you in this post.

One of the important things I learned in CAST is the cost of making a few mistakes and the cost of wasting someone's time. I did some mistakes a few months ago but experienced the cost of making those mistakes when I could not stand in front of those people whose time I had wasted. I felt too ashamed of myself but I know I would not feel ashamed for a life time as I am recovering from those mistakes.

Learning is fun as long as you don't learn about yourself and how bad you are in somethings that you want to be really good at. The more I learn that I am bad, I am inspired to work hard enough to get over it. Being a student of James Bach means I keep searching for the answer, "How do I know what I know?" and often I discover what I don't know when I meet and discuss with great minds like the ones I met in CAST 08. I make it a practice to learn what I don't know or what more I'd want to learn on what I want to be good at.


Not all pain is bad

I went all way from Bengaluru ( Bangalore ) to Toronto to know more about myself and how bad I am in things I want to be good at. It hurts but there is no short cut to learning. For those who might disagree that learning could hurt - as testers we provide information about the quality of the product to the management. If we find a lot of bugs, it might hurt the management when they know it especially when it upsets their shipping schedules. It hurts them because they have learned something about their own product. This pain is then converted to measures to not get into a similar situation again. That is what I think some great man said, "No pain, No gain". The things that I learned from my heroes at the conference and other discussions would be reflected in my blog for years to come, so unfortunately keep reading them.


Person dependent software projects and organization

India depended on Sachin Tendulkar for winning a lot of matches. Cricket ( a team sport ) became a person dependent one. Is it a problem for Sachin?

Not really. Actually it demanded him to be at a very good knock and he loved that challenge as it challenged its consistency. It could also be stated that - he faced tough situations and cleared balls beyond boundaries which is why he is admired by millions worldwide.

Similarly, if your organization doesn't want to make their testing a person dependent thing it is because they don't like you to be Sachin Tendulkar . They can't stop from you becoming a Sachin of software testing because it depends on your learning as a tester and not they giving a learning opportunity to you.

You will be interested to read what James Bach wrote about the need for super heroes for software projects long back.

--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --