All workshops that I conduct has lots of elements of hands on testing and thinking exercises. I myself benefited a lot from hands on testing training approach that James and Michael do in their Rapid Software Testing class.
Over the last few months, I have been working on models of hands on training on software testing for freshers and people who are interested to be good enough to start testing computer software and for organizations who have a tough time getting people to test with lesser supervision than the current magnitude.
I have always been worried about the value of showing thousands of slides on software testing without actually getting a person to test, especially when a person hasn't had any testing experience.
"An ounce of practice is better than tons of theory" - Swami Sivanada
So, here comes a 30 day course on software testing that is hands on training from Satisfice India in partnership with Edista Testing Institute. Edista would be happy to place the participants of this kind of training to their clients who have been wanting freshers and candidates who can test as their resources.
If you were to make a choice between a A+ certified tester who got certified A+ merely based on answering 100 questions versus someone who has a certificate of successful completion of a hands on testing training, whom would you choose?
If you know of any fresher in India, or, if you want someone to benefit from this training kindly ask them to call Noorie @ +91 96322 22326 or e-mail her at noorie.a@edistatesting.com and ask for the hands on testing training for freshers.
I wish I can take a bet against those certification people to challenge those who attend this kind of hands on training for the same 30 days that the certified people take and ask them to see the value. I think I definitely can when I show you the results in the coming months.
Now, what I have written here might not be so wooing as a commercial ad of a testing training but I believe there is more value for the cost and is more worthy of someone's time if they want themselves to be trained good enough to start testing a software.
Oh! I didn't talk about the cost. This training costs twice as much as my 2 day Exploratory Testing class. Yes, its that affordable! And, there are only 6 more seats to be filled.
The good news is, I interviewed Testing Heads/ Vice President - Testing / Founders / CXO of organizations like - Wipro, Applabs, Patni, HP, Target, National Stock Exchange, Infosys, AIG, Edista Testing, VM Logix, ITKO Lisa, Deloite Consulting... at STC 2008 conference last fortnight and asked them a question about hands on training than mere slide show. You guess what their answer would have been or watch the videos at Test Republic in a couple of days.
One of the change India needs is here!
--
Pradeep Soundararajan :: Software Testing Videos: http://www.viddler.com/explore/testertested
Note: If you are interested to attend a 2 hour session on Exploratory Testing in Chennai on Dec 9th e-mail Noorie
Thursday, December 04, 2008
Tuesday, November 25, 2008
Grammy Award for Michael Bolton and others
Michael Bolton, the singer has won Grammy Awards. This Michael Bolton , a little known mandolin player and largely known singer of good intellectual software testing has been awarded for his innovation and contribution to the community you and me live in, at EuroStar 2008, one of the biggest conferences in the world.
Wait a minute, T H E Y won the award!
I just want to celebrate this more than any of my own success because it makes me so proud that my singing is influenced by THEY. Read more ->
Some Indian testers won thought leadership award at Test 2008 conference. Wondering who they are?
Plus, at STC2008 conference that concluded today, Sharath Byregowda, ( another proud moment for me ) won the Best Contributor award for Test Republic
What does all this say to you?
If you want to be a tester who goes beyond the traditional, great recognition awaits you BUT if you do things just for recognition you won't get it. Come out and communicate if you haven't been communicating or else you will see yourself vanishing.
I am sandwiched between my guru and my student winning the award at around the same time. What a fantastic moment for me. During the toughest of the times, I kept hoping that software testers who focus on human skills should shine in the future and those who aren't willing to challenge their minds should vanish.
This is just the beginning of a greater success of these people. Wake up or Vanish!
Wait a minute, T H E Y won the award!
I just want to celebrate this more than any of my own success because it makes me so proud that my singing is influenced by THEY. Read more ->
Some Indian testers won thought leadership award at Test 2008 conference. Wondering who they are?
Plus, at STC2008 conference that concluded today, Sharath Byregowda, ( another proud moment for me ) won the Best Contributor award for Test Republic
What does all this say to you?
If you want to be a tester who goes beyond the traditional, great recognition awaits you BUT if you do things just for recognition you won't get it. Come out and communicate if you haven't been communicating or else you will see yourself vanishing.
I am sandwiched between my guru and my student winning the award at around the same time. What a fantastic moment for me. During the toughest of the times, I kept hoping that software testers who focus on human skills should shine in the future and those who aren't willing to challenge their minds should vanish.
This is just the beginning of a greater success of these people. Wake up or Vanish!
Friday, November 07, 2008
Rating testers based on number of bugs they find - It's definitely possible
In a CMM Level 5 organization I worked for about 2 years back, I got an e-mail from my Senior Manager that testers would be rated based on number of bugs they find. At that time I was an ace bug slogger ( means, I logged a lot of bugs ) for the team.
I was the first one to revolt on that idea of rating a tester based on the number of bugs they find despite knowing the fact that it could benefit me a lot. I was aware of the importance of other testers in the team who weren’t as good bug sloggers like me but had some skills that I did not have and were important skills that a test team required. For instance, I wouldn’t execute test cases. A combination of scripted and exploratory testing was important for that team. A customer insisted that the test cases ( about 140 ) were executed and report being sent to him without discarding the idea that we could do exploratory testing. Some testers in the team had brilliant ideas to create test files that helped the team in finding a lot of bugs. Some testers were documentation savvy guys who used to sit hours together patiently to generate release reports. We shared our tasks pretty well.
I replied to the e-mail saying, “Despite I see that this policy could benefit me, I see this as a threat to the value we testers can add and have been adding. I wouldn’t mind to quit the organization if this policy is serious and be put to action”.
The idea of losing a tester like me on a project whose business is a lot of US Dollars sounded bad to the management and they changed the policy to “Testers will be rated based on the number of bugs that customers find” for which, my reply was, “This doesn’t make me any comfortable”.
In this context, I started to look for articles that talk about testers being rated based on number of bugs found by them and bumped into Dr Cem Kaner’s article Don’t Use Bug Counts to Measure Testers and the conclusion is super fantastic “If you really need a simple number to use to rank your testers, use a random number generator. It is fairer than bug counting, it probably creates less political infighting, and it might be more accurate”.
I was very much impressed about the ideas shared there and shared that article with my management. No measurement of bugs to rate testers happened to my knowledge.
Till a couple of days before, I was thinking that testers should not be rated based on number of bugs they find.
Well, its possible.
I read a post from Sharath Byregowda, a tester whom I respect and collaborate with inBangalore
Can I still use bug count to measure testers although I know it’s a bad idea and make a good judgment out of it ?
Then an idea struck, “Measure testers based on their reaction to being rated based on number of bugs they find”
All good and great testers that I know and those who understanding testing better than the major population of testers would oppose the idea. There you are – you know who understands testing better than your other staff and whom to retain and who cannot be easily replaced.
Those who test whatever information they get are likely to test the product better than those who don’t test the information they have been given.
If you give a specification document, test plan document, test case document to someone who doesn’t test the information ( which most management does ) in it, you have a problem with your hiring and staff. I have witnessed testers who think of those documents as Bible, The Holy Quran and Bhagvad Gita.
Watch the following video. While you watch the video answer my question:
Isn’t what you see in the video strikingly similar to what most testers do?
Step 1: Open that
Step 2: Click Here,
Expected Result: This should happen and then you get a peanut.
Many such peanuts satisfy your hunger.
Such humans can definitely be rated on the number of times they find things that they are expected to find through a tightly written script. Now you know why some testers use the term “monkey testing” as something that they do and also now you know why their being paid peanuts.
Disclaimer: The usage of monkey training video isn’t to talk ill about the training that was happening to those monkeys there. I respect and appreciate that those monkeys are being trained for a noble job of helping physically handicapped people get their jobs done. Kudos to the team at http://www.monkeyhelpers.org and for their brilliant idea of using monkeys to help physically handicapped humans.
--
Pradeep Soundararajan - Software Testing Videos: http://www.viddler.com/explore/testertested
I was the first one to revolt on that idea of rating a tester based on the number of bugs they find despite knowing the fact that it could benefit me a lot. I was aware of the importance of other testers in the team who weren’t as good bug sloggers like me but had some skills that I did not have and were important skills that a test team required. For instance, I wouldn’t execute test cases. A combination of scripted and exploratory testing was important for that team. A customer insisted that the test cases ( about 140 ) were executed and report being sent to him without discarding the idea that we could do exploratory testing. Some testers in the team had brilliant ideas to create test files that helped the team in finding a lot of bugs. Some testers were documentation savvy guys who used to sit hours together patiently to generate release reports. We shared our tasks pretty well.
I replied to the e-mail saying, “Despite I see that this policy could benefit me, I see this as a threat to the value we testers can add and have been adding. I wouldn’t mind to quit the organization if this policy is serious and be put to action”.
The idea of losing a tester like me on a project whose business is a lot of US Dollars sounded bad to the management and they changed the policy to “Testers will be rated based on the number of bugs that customers find” for which, my reply was, “This doesn’t make me any comfortable”.
In this context, I started to look for articles that talk about testers being rated based on number of bugs found by them and bumped into Dr Cem Kaner’s article Don’t Use Bug Counts to Measure Testers and the conclusion is super fantastic “If you really need a simple number to use to rank your testers, use a random number generator. It is fairer than bug counting, it probably creates less political infighting, and it might be more accurate”.
I was very much impressed about the ideas shared there and shared that article with my management. No measurement of bugs to rate testers happened to my knowledge.
Till a couple of days before, I was thinking that testers should not be rated based on number of bugs they find.
Well, its possible.
I read a post from Sharath Byregowda, a tester whom I respect and collaborate with in
Can I still use bug count to measure testers although I know it’s a bad idea and make a good judgment out of it ?
Then an idea struck, “Measure testers based on their reaction to being rated based on number of bugs they find”
All good and great testers that I know and those who understanding testing better than the major population of testers would oppose the idea. There you are – you know who understands testing better than your other staff and whom to retain and who cannot be easily replaced.
Those who test whatever information they get are likely to test the product better than those who don’t test the information they have been given.
If you give a specification document, test plan document, test case document to someone who doesn’t test the information ( which most management does ) in it, you have a problem with your hiring and staff. I have witnessed testers who think of those documents as Bible, The Holy Quran and Bhagvad Gita.
Watch the following video. While you watch the video answer my question:
Isn’t what you see in the video strikingly similar to what most testers do?
Step 1: Open that
Step 2: Click Here,
Expected Result: This should happen and then you get a peanut.
Many such peanuts satisfy your hunger.
Such humans can definitely be rated on the number of times they find things that they are expected to find through a tightly written script. Now you know why some testers use the term “monkey testing” as something that they do and also now you know why their being paid peanuts.
Disclaimer: The usage of monkey training video isn’t to talk ill about the training that was happening to those monkeys there. I respect and appreciate that those monkeys are being trained for a noble job of helping physically handicapped people get their jobs done. Kudos to the team at http://www.monkeyhelpers.org and for their brilliant idea of using monkeys to help physically handicapped humans.
--
Pradeep Soundararajan - Software Testing Videos: http://www.viddler.com/explore/testertested
Tuesday, October 28, 2008
Testing the number of years of software testing experience
I am a software testing coach. I am not the kind of software testing coach who runs through hundred of slides about how to drive a car for those aspiring to drive a car or for those who aspire to drive better. I coach by putting each one of them in drivers seat. That's how James Bach and Michael Bolton coach software testers. I believe they picked up such an approach from Jerry Weinberg.
I am young and will remain young. Some people who sit in my class have a problem with my age. Some people who sit in my class have a problem with their age. They see me as young as they were about a decade ago or half a decade ago and wonder what would I be able to teach them.
This makes my job all the more challenging. I try to crack tough nuts even before I get started off with my class. One such thing happened 2 weeks back in Pune while training for a corporate client when a Test Lead attempted to walk out of my class yelling "this is the most useless class I have ever taken".
Well, I just wondered if he ever walked out of a movie just watching the first minute and deciding the whole movie was useless?
Jerry Weinberg said "Words just carry 10% and the rest is music", which is so true.
I infer from the kind of music I heard from him and the number of white hair he had grown, the problem was not with the class but with his age and mine. Do you ever go out and say "Well, you are inexperienced to teach me?" Don't you just show that in the behavior?
That evening I was conversing about the incident with Manoj Nair and was trying to question "The number of years of experience syndrome".
How many days make an year?
365. 25 ( that's three sixty five and a quarter days )
How many hours make a day?
24 hours ( that's twenty four hours )
When you ask someone about their experience, what metric do they use?
X Number of years of experience
Is that true?
When people say I have 10 years of experience:
Are they trying to say, they worked for 365.25 x 10 = 3652.5 days
Are they trying to say, they worked for 3652.5 x 24 = 87660 hours
We all know that Saturdays and Sundays are holidays and we also have a leave/holiday package an employer gives to all its employees.
So, for those who claim 10 years of experience, ignoring Saturdays, Sundays and average leave/holiday package of 30 days per year plus sick leaves, offs, training time, office events, party - they actually work for about 220 days.
Hold on - 220 are working days.
Assuming their project manager and management goofed up metrics and made their employees work for about 12 hours a working day during those working days, it brings down the actual number of days they worked in an year to 110 days.
Does 110 days make an year?
Well, it may but I am wondering in which planet would that be.
If in one year they actually are at office for 110 days then for 10 years their stay at office is 110 x 10 = 1100 days.
Now how many years of experience does someone who claims to have 10 years of experience has = 1100 / 365.25 = 3.01 years.
Now how much time does a person stays productive out of the 12 hours we calculated?
(You figure out further questions)
Assuming someone hanged around for 5 years ( their idea of an year ) , get promoted to Test Lead and stop running tests. How much time did they actually spend running tests? About an year?
Isn't that their true software test execution experience?
That's why those people who have a problem in my class have more than a problem - with my age and with my questioning skill that makes them feel they are just 3 years of experience and not the huge number that they claim.
"Experience is not the time that has elapsed by. It is what you have done to the time that has elapsed by" - Attributed to some great soul whose name I am unable to find through Google search. This quote impacted my teenage and future thereafter.
Also, that's why people like Michael Bolton, Jerry Weinberg, James Bach, Cem Kaner, Ben Simo, Scott Barber, Jonathan Kohl, Shrini Kulkarni ... are more experienced than what some of us can ever achieve working 110 days an year.
You may continue to say your experience in number of years because it makes you feel good or talk about the things you did and let people figure out what your experience is.
After I finished the workshop and came out of my client's office, I saw someone holding a book "Effective methods of software testing" and I asked her, "Do you know of any ineffective methods to test software?" and her immediate question was: What experience do you have?
My reply : Five million four hundred and thirty six mistakes that I learnt from.
Her response: WHAT!
I definitely might have sounded stupid to her but certainly not as stupid as mentioning I have 10 years of experience although my true stay time at office was 3 years.
--
Pradeep Soundararajan - http://testertested.blogspot.com - http://www.viddler.com/explore/testertested
Question of the day: Did you read copyrights section in this blog?
I am young and will remain young. Some people who sit in my class have a problem with my age. Some people who sit in my class have a problem with their age. They see me as young as they were about a decade ago or half a decade ago and wonder what would I be able to teach them.
This makes my job all the more challenging. I try to crack tough nuts even before I get started off with my class. One such thing happened 2 weeks back in Pune while training for a corporate client when a Test Lead attempted to walk out of my class yelling "this is the most useless class I have ever taken".
Well, I just wondered if he ever walked out of a movie just watching the first minute and deciding the whole movie was useless?
Jerry Weinberg said "Words just carry 10% and the rest is music", which is so true.
I infer from the kind of music I heard from him and the number of white hair he had grown, the problem was not with the class but with his age and mine. Do you ever go out and say "Well, you are inexperienced to teach me?" Don't you just show that in the behavior?
That evening I was conversing about the incident with Manoj Nair and was trying to question "The number of years of experience syndrome".
How many days make an year?
365. 25 ( that's three sixty five and a quarter days )
How many hours make a day?
24 hours ( that's twenty four hours )
When you ask someone about their experience, what metric do they use?
X Number of years of experience
Is that true?
When people say I have 10 years of experience:
Are they trying to say, they worked for 365.25 x 10 = 3652.5 days
Are they trying to say, they worked for 3652.5 x 24 = 87660 hours
We all know that Saturdays and Sundays are holidays and we also have a leave/holiday package an employer gives to all its employees.
So, for those who claim 10 years of experience, ignoring Saturdays, Sundays and average leave/holiday package of 30 days per year plus sick leaves, offs, training time, office events, party - they actually work for about 220 days.
Hold on - 220 are working days.
Assuming their project manager and management goofed up metrics and made their employees work for about 12 hours a working day during those working days, it brings down the actual number of days they worked in an year to 110 days.
Does 110 days make an year?
Well, it may but I am wondering in which planet would that be.
If in one year they actually are at office for 110 days then for 10 years their stay at office is 110 x 10 = 1100 days.
Now how many years of experience does someone who claims to have 10 years of experience has = 1100 / 365.25 = 3.01 years.
Now how much time does a person stays productive out of the 12 hours we calculated?
(You figure out further questions)
Assuming someone hanged around for 5 years ( their idea of an year ) , get promoted to Test Lead and stop running tests. How much time did they actually spend running tests? About an year?
Isn't that their true software test execution experience?
That's why those people who have a problem in my class have more than a problem - with my age and with my questioning skill that makes them feel they are just 3 years of experience and not the huge number that they claim.
"Experience is not the time that has elapsed by. It is what you have done to the time that has elapsed by" - Attributed to some great soul whose name I am unable to find through Google search. This quote impacted my teenage and future thereafter.
Also, that's why people like Michael Bolton, Jerry Weinberg, James Bach, Cem Kaner, Ben Simo, Scott Barber, Jonathan Kohl, Shrini Kulkarni ... are more experienced than what some of us can ever achieve working 110 days an year.
You may continue to say your experience in number of years because it makes you feel good or talk about the things you did and let people figure out what your experience is.
After I finished the workshop and came out of my client's office, I saw someone holding a book "Effective methods of software testing" and I asked her, "Do you know of any ineffective methods to test software?" and her immediate question was: What experience do you have?
My reply : Five million four hundred and thirty six mistakes that I learnt from.
Her response: WHAT!
I definitely might have sounded stupid to her but certainly not as stupid as mentioning I have 10 years of experience although my true stay time at office was 3 years.
--
Pradeep Soundararajan - http://testertested.blogspot.com - http://www.viddler.com/explore/testertested
Question of the day: Did you read copyrights section in this blog?
Wednesday, October 22, 2008
Your search for Bug Free Software ends here
Test 2008 was a scintillating software testing conference. I never slept before 2 AM every night around the conference dates. I had great discussions with interesting testing minds like Rahul Verma, Aswin Palaparthi, Shrini Kulkarni, Dhanashekaran , Ravindran, Shyam Sridhar and others.
During one of our conversations the topic of "bug free software" popped up and someone mentioned that he had hope to see bug free software within his life span as technology is growing rapidly.
I have heard a lot of people talk about bug free software so have you. What have your replies to them been? Have you been one such who hoped for bug free software?
Here are my answers:
Here is another example - I am a perfect thinker.
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --Yours truly
During one of our conversations the topic of "bug free software" popped up and someone mentioned that he had hope to see bug free software within his life span as technology is growing rapidly.
I have heard a lot of people talk about bug free software so have you. What have your replies to them been? Have you been one such who hoped for bug free software?
Here are my answers:
- Oh yeah, you don't need to wait for the technology to advance, bug free software already exists. Have you shipped software products to your customers? Were there bugs in the products you shipped? Did you charge your customers for the bugs? Did you make the bugs free for them? So, you did ship a bug free software! So, all software we ship is bug free!
- I shall give you the link to download a software that remains in bug free state in a specific context. Unfortunately the bugs start showing up if you try downloading the file, opening the file, install the software or start using it. I can guarantee you its bug free if you don't do anything of that sort with it.
- It might be a good idea to read Jerry Weinberg's book on Perfect Software and other Illusions about Testing in which he makes an attempt to make some of us conscious of the fact that - humans are imperfect thinkers, how can you expect an imperfect thinker to produce something that is perfect?
- I think, even, accidentally humans can't produce bug free software. The idea of bug free software itself is the result of imperfect thinking and imperfect understanding of software and bugs. We might end up making better inferences and conjectures about what they are if we are on a continuous learning path.
- "Software systems can easily become complex. Computers allow us mortals to create complex systems that are beyond our ability to fully understand. We testers seek out software problems based on what we understand. We cannot completely test the software. We use a variety of tools and approaches to learn as much as possible. However, we are unlikely to completely understand a complex software system." - Ben Simo
Here is another example - I am a perfect thinker.
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --Yours truly
Monday, September 15, 2008
Exploratory Software Testing Demonstration Videos
You have read enough things from Pradeep Soundararajan. Some of you even believed what he wrote. Some of you thought it was stupid. Some of you follow it very closely. Some of you got irritated by it. Some of you relied on what he wrote for guidance. Some of you enjoyed the writing and others hated it. Some of you thought he was faking. Some of you didn't bother to read what he was writing all these days. Some of you thought he is talking about ideal world. Some of you thought what he talks isn't practical. Some of you will always think good or bad about him no matter what he does. No matter whatever you think of him, he will keep doing things because what fuels him is, himself, primarily.
Now here is my testing open to public scrutiny. A Rapid Tester stands up to scrutiny. Here is something that can stand up to public scrutiny:
That's just one of the 7 videos I have come out with. If you are reading this post, you could find others as well. 7 isn't my limit but at a start. From now on, every week or two, you could find a new video. Not all of them would be published on my blog so don't rely on my blog to get updates. Figure out a way. Also you could download those videos if you wish to watch it offline or to watch it more than once.
Jonathan Kohl wrote about the importance of testing videos here .
For all things you couldn't believe about exploratory testing, human testing skills, rapid testing, context driven approach, pair testing, etc... here is one of your chances. Alternatively James Bach, Mike Kelly and other testers have their own videos which you could find on Googling.
The horse can be taken near the water. The horse won't drink the water if it thinks its not thirsty or it can live without drinking that water or for whatever reason the horse decides not to drink. I know, you aren't a horse or at least you don't appear so.
If you wish to be a part of these videos, drop an e-mail to me. Thanks to Ajay Balamurugadas and Sharath Byregowda for their enthusiasm in helping me with these videos and for pair and trio testing with me.
Operation Roshan has just begun.
Update: 18 Sep: When we watch a movie, we usually fall in love with the actor and actress and forget the work of people who made the movie to happen being at the backstage. Similarly, I forgot to include our host for about 2 days since I posted this who facilitated the video creation, place, resources and provided space to sleep overnight - Edista Testing Institute and Test Republic. When I asked what Edista means, I got a reply as "Facilitation" and this is an organization living up to what its name means.
Thank you folks!
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Now here is my testing open to public scrutiny. A Rapid Tester stands up to scrutiny. Here is something that can stand up to public scrutiny:
That's just one of the 7 videos I have come out with. If you are reading this post, you could find others as well. 7 isn't my limit but at a start. From now on, every week or two, you could find a new video. Not all of them would be published on my blog so don't rely on my blog to get updates. Figure out a way. Also you could download those videos if you wish to watch it offline or to watch it more than once.
Jonathan Kohl wrote about the importance of testing videos here .
For all things you couldn't believe about exploratory testing, human testing skills, rapid testing, context driven approach, pair testing, etc... here is one of your chances. Alternatively James Bach, Mike Kelly and other testers have their own videos which you could find on Googling.
The horse can be taken near the water. The horse won't drink the water if it thinks its not thirsty or it can live without drinking that water or for whatever reason the horse decides not to drink. I know, you aren't a horse or at least you don't appear so.
If you wish to be a part of these videos, drop an e-mail to me. Thanks to Ajay Balamurugadas and Sharath Byregowda for their enthusiasm in helping me with these videos and for pair and trio testing with me.
Operation Roshan has just begun.
Update: 18 Sep: When we watch a movie, we usually fall in love with the actor and actress and forget the work of people who made the movie to happen being at the backstage. Similarly, I forgot to include our host for about 2 days since I posted this who facilitated the video creation, place, resources and provided space to sleep overnight - Edista Testing Institute and Test Republic. When I asked what Edista means, I got a reply as "Facilitation" and this is an organization living up to what its name means.
Thank you folks!
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Wednesday, August 20, 2008
Happy testing and Sad Testing
I don't know from where I heard "Happy Testing", the first time and fear to think if it echoed within me. I also don't know how I caught those words to sign all my e-mails to testers I communicate, writing "Happy Testing" at the end of the e-mail. I noticed that a lot of other testers to whom I communicated also started doing that in their e-mail communication to me and in their blog posts. I didn't know what "Happy Testing" meant when I started using it long back but I think this post explains what I mean by Happy Testing.
For the moment, forget about people ( that includes me ) asking you to do good testing, better testing, great testing, pleasing customers, pleasing managers, and getting great hike. Think about doing happy testing.
As a tester what makes me happy is when I find 'a' bug. What makes me more happy is when I find more than a bug. What makes me the happiest is when I find more and more bugs in every product I test.
I come across a lot of Sad testers and observe a pattern of the bugs they find. Most Sad testers that I come across are the ones who follow test scripts or cases to find bugs. You might observe that those Sad testers say "These test cases found those bugs". I am sure if test cases have life, they would be happy set of organisms because a lot of Sad testers owe credit to the test case for whatever bugs they ( humans ) find.
Its not that those Sad testers are always unhappy about the bugs they find, its that they are happy and not happy enough to recognize how they can be more happy.
If finding a bug doesn't make you happy, how sad a tester you are!
Don't ask me the secret of being a "Happy Tester". I am sure a lot of Sad testers won't believe my reply, "Rapid Software Testing, Exploratory Testing and Context Driven Testing that provides me freedom to think, experiment and find a lot of bugs".
In every Skilled Exploratory Testing corporate workshop I do, I challenge testers on testing their product for 90 minutes and find a lot of bugs within that short span. One of the recent experience was in SAS, Pune, where testers started clapping after the 90th minute witnessing the Happy Testing I did. It felt like a Hero to receive clapping from testers for demonstrating Rapid Software Testing.
What I heard from Rajesh K , one of the managers who nominated his team and attended the workshop in SAS over an e-mail was "The bug count has certainly gone up". Happy Testing Rajesh, Vikram, Manoj Nair and their teams.
Jerry Weinberg revamped his website recently and a sentence in his revamped website made me think what Jerry was trying to say and here is the sentence for you "Dedicated to Helping Smart People be Happy". It made me wonder if smart people can be unhappy. I then thought about all those talented testers in India ( and probably other countries, too ) who have been forced to write and execute test cases and follow best practices that might have worked for someone else. That reminds me to say, I get happy when I read Jerry Weinberg's books because reading his work makes smart and I made myself a lot happier when I bought Jerry's books for about 300 US dollars during my trip to Canada.
Speaking of all that, let me ask a question to myself: Am I never unhappy?
Oh yes, whenever I come across people who are unhappy and they don't want to listen to stuff that can make them realize they are smart and can be happy enough, too.
Am I talking as though I am Mr Perfect?
Oh well, I forgot to share with you that an important lesson I learned is that - Humans are fallible and so are their ideas. Everything is a heuristic and it is a choice of heuristics in a suitable context that makes a person smart. If you think Perfect Software and Software Testing is possible, read Jerry Weinberg's Perfect Software and other Illusions about Testing.
Unfortunately, I want so see those sad testers I come across as happy as me. Unfortunately, they aren't happy that I am claiming to be happy by doing things that they think is impractical or hard.
Fortunately there are some sad testers who are smart and want to be more happy. I live to help them. Oh! Did I forget to tell you that I get happy when I help testers find more bugs.
-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
For the moment, forget about people ( that includes me ) asking you to do good testing, better testing, great testing, pleasing customers, pleasing managers, and getting great hike. Think about doing happy testing.
As a tester what makes me happy is when I find 'a' bug. What makes me more happy is when I find more than a bug. What makes me the happiest is when I find more and more bugs in every product I test.
I come across a lot of Sad testers and observe a pattern of the bugs they find. Most Sad testers that I come across are the ones who follow test scripts or cases to find bugs. You might observe that those Sad testers say "These test cases found those bugs". I am sure if test cases have life, they would be happy set of organisms because a lot of Sad testers owe credit to the test case for whatever bugs they ( humans ) find.
Its not that those Sad testers are always unhappy about the bugs they find, its that they are happy and not happy enough to recognize how they can be more happy.
If finding a bug doesn't make you happy, how sad a tester you are!
Don't ask me the secret of being a "Happy Tester". I am sure a lot of Sad testers won't believe my reply, "Rapid Software Testing, Exploratory Testing and Context Driven Testing that provides me freedom to think, experiment and find a lot of bugs".
In every Skilled Exploratory Testing corporate workshop I do, I challenge testers on testing their product for 90 minutes and find a lot of bugs within that short span. One of the recent experience was in SAS, Pune, where testers started clapping after the 90th minute witnessing the Happy Testing I did. It felt like a Hero to receive clapping from testers for demonstrating Rapid Software Testing.
What I heard from Rajesh K , one of the managers who nominated his team and attended the workshop in SAS over an e-mail was "The bug count has certainly gone up". Happy Testing Rajesh, Vikram, Manoj Nair and their teams.
Jerry Weinberg revamped his website recently and a sentence in his revamped website made me think what Jerry was trying to say and here is the sentence for you "Dedicated to Helping Smart People be Happy". It made me wonder if smart people can be unhappy. I then thought about all those talented testers in India ( and probably other countries, too ) who have been forced to write and execute test cases and follow best practices that might have worked for someone else. That reminds me to say, I get happy when I read Jerry Weinberg's books because reading his work makes smart and I made myself a lot happier when I bought Jerry's books for about 300 US dollars during my trip to Canada.
Speaking of all that, let me ask a question to myself: Am I never unhappy?
Oh yes, whenever I come across people who are unhappy and they don't want to listen to stuff that can make them realize they are smart and can be happy enough, too.
Am I talking as though I am Mr Perfect?
Oh well, I forgot to share with you that an important lesson I learned is that - Humans are fallible and so are their ideas. Everything is a heuristic and it is a choice of heuristics in a suitable context that makes a person smart. If you think Perfect Software and Software Testing is possible, read Jerry Weinberg's Perfect Software and other Illusions about Testing.
Unfortunately, I want so see those sad testers I come across as happy as me. Unfortunately, they aren't happy that I am claiming to be happy by doing things that they think is impractical or hard.
Fortunately there are some sad testers who are smart and want to be more happy. I live to help them. Oh! Did I forget to tell you that I get happy when I help testers find more bugs.
-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Friday, August 08, 2008
Testing to avoid being sued or embarrased
North American, European Union and some Asian countries laws and law enforcements are much stricter than those in India. Most testers in India who have not known much about it, tend to ignore that fact while they test. I too didn't know till I actively kept reading. The value of reading is - I know to focus on problems that matter to my clients in countries like North America.
You might want to know that the cost of being sued is heavy on the organizations. All investment on a product and estimated profits go for a toss on that single day of verdict. Organizations like Microsoft, Apple, Sony and a lot of them have suffered losses by being sued or have withdrew released products on the fear of being sued.
As usual, I was browsing through Orkut today [ 8th August 2008 at 14:00 hours ] and recognized a problem. Please go through the following screen shot carefully before you read further.
One of the newly introduced feature from Orkut is an option to let friends know what I am updating in my profile and vice versa. A friend of mine hadn't added his gender in his orkut profile since he created the profile . He probably decided to do it yesterday.
What I ( and you ) see as an update is "XYZ updated gender". Well, he just added his gender and nothing else.
An update of Gender column could mean, "I have changed my gender" - which sounds absurd or might offend people to know that their friends are thinking that he changed his gender or their friends are thinking that she changed her gender.
This might invite trouble to Orkut or Google if there exists a law in any country that organizations should not mislead or misinform about one's gender. If there exists such a law in any country then they might be susceptible to be sued for showing something that mentally hurt one's gender status amongst their friends and other society members to whom the profile is visible. It also causes a lot of embarassment to someone whose friends make a joke of him/her updating "gender".
Orkut has been in news all over India after people claimed it responsible for being a channel for pranksters who even ended up killing for money and a lot of other cases. What if someone attempts a suicide attempt and blames it on Orkut saying "I could not withstand the embarrassment of my friends asking about my gender".
Do you think its worth Google's time to fight such cases?
Do you also think such problems can be found by test case base approach?
Minutes before I was planning to post this, I thought of getting this post tested from Ben Simo and Jonathan Kohl. Ben Simo shared with me that Facebook came out with some gender issues recently.
A lot of other testers might have seen the problem that I highlighted in this post while they were browsing Orkut, why didn't they find it?
Answer Ben's question to find out the answer to above question : Will you recognize a problem if you see it?
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
You might want to know that the cost of being sued is heavy on the organizations. All investment on a product and estimated profits go for a toss on that single day of verdict. Organizations like Microsoft, Apple, Sony and a lot of them have suffered losses by being sued or have withdrew released products on the fear of being sued.
As usual, I was browsing through Orkut today [ 8th August 2008 at 14:00 hours ] and recognized a problem. Please go through the following screen shot carefully before you read further.
One of the newly introduced feature from Orkut is an option to let friends know what I am updating in my profile and vice versa. A friend of mine hadn't added his gender in his orkut profile since he created the profile . He probably decided to do it yesterday.
What I ( and you ) see as an update is "XYZ updated gender". Well, he just added his gender and nothing else.
An update of Gender column could mean, "I have changed my gender" - which sounds absurd or might offend people to know that their friends are thinking that he changed his gender or their friends are thinking that she changed her gender.
This might invite trouble to Orkut or Google if there exists a law in any country that organizations should not mislead or misinform about one's gender. If there exists such a law in any country then they might be susceptible to be sued for showing something that mentally hurt one's gender status amongst their friends and other society members to whom the profile is visible. It also causes a lot of embarassment to someone whose friends make a joke of him/her updating "gender".
Orkut has been in news all over India after people claimed it responsible for being a channel for pranksters who even ended up killing for money and a lot of other cases. What if someone attempts a suicide attempt and blames it on Orkut saying "I could not withstand the embarrassment of my friends asking about my gender".
Do you think its worth Google's time to fight such cases?
Do you also think such problems can be found by test case base approach?
Minutes before I was planning to post this, I thought of getting this post tested from Ben Simo and Jonathan Kohl. Ben Simo shared with me that Facebook came out with some gender issues recently.
A lot of other testers might have seen the problem that I highlighted in this post while they were browsing Orkut, why didn't they find it?
Answer Ben's question to find out the answer to above question : Will you recognize a problem if you see it?
--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
Wednesday, July 30, 2008
Most test cases fail
- Most testers who write test cases carry a hope that most test cases they write would help in finding most bugs.
- Most testers who execute them carry a hope that most test cases they execute would help in finding most bugs.
- Most testers who add more test cases carry a hope that they will help in finding most bugs.
- Most managers who ask testers to write test cases carry a hope that their testers will catch most bugs with the help of test cases.
- Most managers who asked them to write test cases ask a question: How many test cases passed? when considering to ship.
- Most testers in those contexts reply, "Most".
"If a test case didn't help in finding a bug then the purpose of that test case failed. Products are shipped when most test cases fail their purpose of existence."
------------------------------------------------------------------------------------------------------------------------------------------
- Test cases fail to help most testers who write and execute them realize that testing is not a monotonous job as they are doing.
- Test cases fail to help most testers realize that there are other ways to achieve what they are trying to achieve or there are other ways to do much better testing.
- Test cases fail to help most managers understand that the number of test cases passed is a misleading metric because most test cases means few documented tests from a possible hundred billion tests.
- Test cases fail to help most customers understand that there are ways to gain more value for the cost they are paying.
- Test cases fail to help testers earn more money because in scripted testers view - an expert tester is one who writes a lot of test cases and executes a lot of test cases.
Test cases are human ideas and all human ideas can fail. Most testers who write and execute it fail to realize it.
If human ideas to test can't fail so will the software, not fail. Don't test.
If human ideas to test can't fail so will the software, not fail. Don't test.
------------------------------------------------------------------------------------------------------------------------------------------
To participate ( or just enjoy ) in a discussion involving exploratory and scripted testers on test cases , click here .
-- Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Update: The comments posted as Angel and Devil are by me ( Pradeep Soundararajan ). Wondering why I posted comments on the post I wrote - you would know the answer if you go through those comments.
The Angels and Devils concept in Testing is inspired by a presentation from Michael Bolton and Jonathan Kohl . Here is an excerpt of the presentation and also illustrates Elizabeth Kohl's design skills.
Monday, July 21, 2008
Heroes of software testing - Do you know about them and their work?
My initial plan was to board the flight to Toronto on July 11th from Mumbai to attend CAST 08 software testing conference. Sangeeta, a tester from Mumbai influenced me to change my plan and organized a lecture in Ness Technologies in Mumbai.
I liked the offer and accepted it, as I knew I would enjoy the experience. It's fun to meet new testers, talk to them and learn new things. I am in constant search for budding testers like Sharath Byregowda , Ajay Balamurugadas , Sathish Kumar Chinappa , Sangeeta, Sandesh , Girija, Bhargavi and Nattu ( who accompanied me to CAST 08 ) and those who are passionate and do things about seeing a better tester community like Mohan Panguluri, COO of Edista Testing ... I never know where I can find similar ones. One of the ways that has been of great help to me, to find such testers and thinkers is, while at lectures and my workshops. I offer whatever help I am capable of doing to expose them to good stuff and I do it when they ask me for it.
Reading good stuff as a bad practice and reading bad stuff as the best practice
The above state is what most Indian testers to my knowledge are stuck in. This state resulted because most of testers don't have the habit of reading as though its a bad practice. Most of those who read, are the ones who search in Google and land up at bad sites ( that includes my blog at times ). Most of those who Google search are the ones who are not interested at better thought process but at ready made best practice answers.
There are good and great stuff from internet. I once searched for expert testers and found James Bach. That google search changed my life and the way I test. There were a lot of junk stuff written to attract such search results that I had to pass through before finding James Bach's website. I was sure that I needed help about thought process from an expert tester and not ready made solutions from time to time.
The road to Toronto
I had to face a lot of trouble to get to Toronto. My Visa was rejected ( at the first attempt ), I didn't have sufficient funds ( as per the VISA officials and my bank statement ) , I was denied permission to board the plane as my ticket was booked via United States and I didn't have a US Visa. The Canadian customs had a discussion that bothered me if they would let me in after I landed at Toronto and then my baggage went missing and finally landed as the last one on the belt. Amidst all this was my hope to meet the heroes in real space who have inspired me and to meet those whom I can draw more inspiration from. I believe in God and also believe that God takes a human form to help humans. A great human who lives in 61, Ashburnham in Toronto and the one who lived in my house ( my dad ) helped me and cleared traps so that I reached Toronto and was able to attend CAST 08.
Chasing dreams
Finally I got a chance to meet the heroes who have been inspiring me to test and think better. My super hero James Bach wasn't present at the conference and hence I couldn't meet him. I would be meeting him in November at a conference where we both are invited to speak on testing at a developers conference in Malmo Sweden.
By meeting my heroes, I learned some important things that I could not have learned, had I not met them. I got an insight into the way they live, some ways they think and they communicate. It was several dreams of several years that came true, all in one place - CAST 08.
Not all learning is fun.
If you have understood what I have written in this post so far, then my English writing skill has improved a lot since I wrote my first blog post. It pained a lot to know that I was writing terribly bad English but had I not learned it, I wouldn't have been able to make sense to you in this post.
One of the important things I learned in CAST is the cost of making a few mistakes and the cost of wasting someone's time. I did some mistakes a few months ago but experienced the cost of making those mistakes when I could not stand in front of those people whose time I had wasted. I felt too ashamed of myself but I know I would not feel ashamed for a life time as I am recovering from those mistakes.
Learning is fun as long as you don't learn about yourself and how bad you are in somethings that you want to be really good at. The more I learn that I am bad, I am inspired to work hard enough to get over it. Being a student of James Bach means I keep searching for the answer, "How do I know what I know?" and often I discover what I don't know when I meet and discuss with great minds like the ones I met in CAST 08. I make it a practice to learn what I don't know or what more I'd want to learn on what I want to be good at.
Not all pain is bad
I went all way from Bengaluru ( Bangalore ) to Toronto to know more about myself and how bad I am in things I want to be good at. It hurts but there is no short cut to learning. For those who might disagree that learning could hurt - as testers we provide information about the quality of the product to the management. If we find a lot of bugs, it might hurt the management when they know it especially when it upsets their shipping schedules. It hurts them because they have learned something about their own product. This pain is then converted to measures to not get into a similar situation again. That is what I think some great man said, "No pain, No gain". The things that I learned from my heroes at the conference and other discussions would be reflected in my blog for years to come, so unfortunately keep reading them.
Person dependent software projects and organization
India depended on Sachin Tendulkar for winning a lot of matches. Cricket ( a team sport ) became a person dependent one. Is it a problem for Sachin?
Not really. Actually it demanded him to be at a very good knock and he loved that challenge as it challenged its consistency. It could also be stated that - he faced tough situations and cleared balls beyond boundaries which is why he is admired by millions worldwide.
Similarly, if your organization doesn't want to make their testing a person dependent thing it is because they don't like you to be Sachin Tendulkar . They can't stop from you becoming a Sachin of software testing because it depends on your learning as a tester and not they giving a learning opportunity to you.
You will be interested to read what James Bach wrote about the need for super heroes for software projects long back.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
I liked the offer and accepted it, as I knew I would enjoy the experience. It's fun to meet new testers, talk to them and learn new things. I am in constant search for budding testers like Sharath Byregowda , Ajay Balamurugadas , Sathish Kumar Chinappa , Sangeeta, Sandesh , Girija, Bhargavi and Nattu ( who accompanied me to CAST 08 ) and those who are passionate and do things about seeing a better tester community like Mohan Panguluri, COO of Edista Testing ... I never know where I can find similar ones. One of the ways that has been of great help to me, to find such testers and thinkers is, while at lectures and my workshops. I offer whatever help I am capable of doing to expose them to good stuff and I do it when they ask me for it.
Here is an excerpt from the talk:
"We are in India, the land where cricket is considered close to being a religion. It has super heroes like Sachin Tendulkar and Dhoni who have inspired a lot of people in India to play better cricket. They could inspire because a lot of people spent their time watching them play great cricket at tough situations. As Indians, you might have seen a lot of people who enjoy playing cricket the way Sachin and Dhoni play. As testers, have you seen anyone doing testing that has inspired you to test better?"
Response: Silence
and then a question comes up from a corner, "Who did Sir Don Bradman see as an inspiration for he is the best known cricketer of centuries?" and all other testers burst to laughter thinking that I was cornered with that question.
I took a while for the audience to settle down and replied,
"I know who inspired Sir Don Bradman" and then a pause to add, "It was himself."
"How many of you are inspired by your own testing?"
Response: Silence
End of excerpt
"We are in India, the land where cricket is considered close to being a religion. It has super heroes like Sachin Tendulkar and Dhoni who have inspired a lot of people in India to play better cricket. They could inspire because a lot of people spent their time watching them play great cricket at tough situations. As Indians, you might have seen a lot of people who enjoy playing cricket the way Sachin and Dhoni play. As testers, have you seen anyone doing testing that has inspired you to test better?"
Response: Silence
and then a question comes up from a corner, "Who did Sir Don Bradman see as an inspiration for he is the best known cricketer of centuries?" and all other testers burst to laughter thinking that I was cornered with that question.
I took a while for the audience to settle down and replied,
"I know who inspired Sir Don Bradman" and then a pause to add, "It was himself."
"How many of you are inspired by your own testing?"
Response: Silence
End of excerpt
Reading good stuff as a bad practice and reading bad stuff as the best practice
The above state is what most Indian testers to my knowledge are stuck in. This state resulted because most of testers don't have the habit of reading as though its a bad practice. Most of those who read, are the ones who search in Google and land up at bad sites ( that includes my blog at times ). Most of those who Google search are the ones who are not interested at better thought process but at ready made best practice answers.
There are good and great stuff from internet. I once searched for expert testers and found James Bach. That google search changed my life and the way I test. There were a lot of junk stuff written to attract such search results that I had to pass through before finding James Bach's website. I was sure that I needed help about thought process from an expert tester and not ready made solutions from time to time.
The road to Toronto
I had to face a lot of trouble to get to Toronto. My Visa was rejected ( at the first attempt ), I didn't have sufficient funds ( as per the VISA officials and my bank statement ) , I was denied permission to board the plane as my ticket was booked via United States and I didn't have a US Visa. The Canadian customs had a discussion that bothered me if they would let me in after I landed at Toronto and then my baggage went missing and finally landed as the last one on the belt. Amidst all this was my hope to meet the heroes in real space who have inspired me and to meet those whom I can draw more inspiration from. I believe in God and also believe that God takes a human form to help humans. A great human who lives in 61, Ashburnham in Toronto and the one who lived in my house ( my dad ) helped me and cleared traps so that I reached Toronto and was able to attend CAST 08.
Chasing dreams
Finally I got a chance to meet the heroes who have been inspiring me to test and think better. My super hero James Bach wasn't present at the conference and hence I couldn't meet him. I would be meeting him in November at a conference where we both are invited to speak on testing at a developers conference in Malmo Sweden.
By meeting my heroes, I learned some important things that I could not have learned, had I not met them. I got an insight into the way they live, some ways they think and they communicate. It was several dreams of several years that came true, all in one place - CAST 08.
Not all learning is fun.
If you have understood what I have written in this post so far, then my English writing skill has improved a lot since I wrote my first blog post. It pained a lot to know that I was writing terribly bad English but had I not learned it, I wouldn't have been able to make sense to you in this post.
One of the important things I learned in CAST is the cost of making a few mistakes and the cost of wasting someone's time. I did some mistakes a few months ago but experienced the cost of making those mistakes when I could not stand in front of those people whose time I had wasted. I felt too ashamed of myself but I know I would not feel ashamed for a life time as I am recovering from those mistakes.
Learning is fun as long as you don't learn about yourself and how bad you are in somethings that you want to be really good at. The more I learn that I am bad, I am inspired to work hard enough to get over it. Being a student of James Bach means I keep searching for the answer, "How do I know what I know?" and often I discover what I don't know when I meet and discuss with great minds like the ones I met in CAST 08. I make it a practice to learn what I don't know or what more I'd want to learn on what I want to be good at.
Not all pain is bad
I went all way from Bengaluru ( Bangalore ) to Toronto to know more about myself and how bad I am in things I want to be good at. It hurts but there is no short cut to learning. For those who might disagree that learning could hurt - as testers we provide information about the quality of the product to the management. If we find a lot of bugs, it might hurt the management when they know it especially when it upsets their shipping schedules. It hurts them because they have learned something about their own product. This pain is then converted to measures to not get into a similar situation again. That is what I think some great man said, "No pain, No gain". The things that I learned from my heroes at the conference and other discussions would be reflected in my blog for years to come, so unfortunately keep reading them.
Person dependent software projects and organization
India depended on Sachin Tendulkar for winning a lot of matches. Cricket ( a team sport ) became a person dependent one. Is it a problem for Sachin?
Not really. Actually it demanded him to be at a very good knock and he loved that challenge as it challenged its consistency. It could also be stated that - he faced tough situations and cleared balls beyond boundaries which is why he is admired by millions worldwide.
Similarly, if your organization doesn't want to make their testing a person dependent thing it is because they don't like you to be Sachin Tendulkar . They can't stop from you becoming a Sachin of software testing because it depends on your learning as a tester and not they giving a learning opportunity to you.
You will be interested to read what James Bach wrote about the need for super heroes for software projects long back.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Monday, June 30, 2008
It's a "tester" who finds a bug, even with the robust Google Search
The image you might see above is a screen shot of what I saw after hitting the 12th page for the search results of "tester". I am not sure if you can reproduce that because I haven't investigated on it but I did plan to capture it to demonstrate:
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it."
Be it with the robust systems like Google Search Engine or with weak systems that we might be using, it is always a HUMAN who finds a bug. A lot of testers I know think of "test case" finding a bug.
There is a test case document that consists of 9856985956895869698569956985698459698 test cases and no tester executing it wouldn't find bugs by itself. There is a test case document with 3 documented tests and a tester takes the help of that to find bugs when he executes, observes the result and recognizes a bug.
A test case is an extension of a test idea. What matters to a tester is a test idea and not the test case. Skilled (exploratory testers ( humans ) use tons of ideas ( heuristics and oracles ) to find and recognize bugs. That's why they can find more bugs that matter than those running thousands of test cases over and over again.
Honestly, 99% of testers I have come across didn't say - "I read each test case each time I have to execute it after I have done it once. Also, I religiously follow what is written in the test case".
What happens when they deviate from the documented test case is, they are exploring and running different test. Maybe they don't like to call it that way because their management who pays wouldn't like to know that they are not executing the "test cases".
That's how customers are fooled by management saying "yes, we are running test cases" and yet benefited by the testing community by running more tests.
A test idea can be executed in hundreds of different ways. Check out the deep analysis made by James Bach and Michael Bolton on What Do Scripts Tell Us?
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Tuesday, June 17, 2008
Friday, June 06, 2008
Letter to myself
Dear Pradeep,
Greetings!
I have been with you all this while and shall continue to be with you. I am enjoying each moment you enjoy as a tester and also each moment that you don't enjoy as a tester. I am sorry for enjoying those moments which you think you haven't been enjoying. I am sure you'd be interested to know why and how I have been enjoying those moments that you haven't been.
First, let me list the thing that you haven't been enjoying:
I also remember that you had made a note in your Moleskine of Saurav Ganguly's television interview where he was asked: How were you able to make a great comeback to the world cup cricket squad after being axed for poor performance? His answer: I didn't worry about things that are not under my control ( media critique, jokes on bad batting performance, e-mail forwards about my performance, people gossiping about it ) but focussed on things that are under my control ( Practice, skill enhancement, consistent batting record in Ranji trophy)
Similarly, you don't have a control over the thoughts of people thinking whatever they think after reading whatever you have written. You have a control over what you write and you have a control over the way you write it.
Your testing has been influenced by a lot of experts but not all have similar influence. They haven't seen great testing to appreciate the things you are sharing and I doubt if all those who witness it would be influenced by it because it's hard work and high skill demanding.
Here are three questions ( like the Monty Python and the Holy grail bridge of death piece you enjoy )
1.Whom are you serving through your writing?
I am sure your ongoing struggle is in understanding that. Let me help you with what I think about whom you are serving - You are serving those testers who look for better thought process and those who enjoy the better thought process and those who think you have a better thought process.
2. Who asked you to serve them?
I asked you to do that!
3. Why haven't you been enjoying some moments that I have been?
You want all testers to do great testing although you know its not possible. Some people question your idea of "great testing" because they already have an idea of "great testing" and it conflicts with the idea you have. You are able to demonstrate that their idea of "great testing" lacks critical thought as your idea of "great testing".
By the way, your idea of "great testing", is not yours but of those people who have influenced you. You have just subscribe to those ideas and are contributing to it in different forms. I have occasionally witnessed you doing bad testing and I am sure I would see that in future, too. Do not forget that you are a human and your ideas are fallible. I know bad testing and bad thought process irritates you, even if you are the one who is doing it.
I would love to see you doing things that are under your control - learning, reading, writing, bettering your skills, helping those testers who enjoy the thought process that you enjoy, speaking, coaching and mentoring.
Your power to influence testers is limited. Limited to the ones who don't want to limit themselves. So do unlimited things under limited time that you and I will be here in this world.
I will be with you forever, enjoying everything you do from great things to not so great things. Anything you do is great to me.
I will write to you whenever I feel a need for it. This letter is personal, just between you and me.
Yours truly,
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Greetings!
I have been with you all this while and shall continue to be with you. I am enjoying each moment you enjoy as a tester and also each moment that you don't enjoy as a tester. I am sorry for enjoying those moments which you think you haven't been enjoying. I am sure you'd be interested to know why and how I have been enjoying those moments that you haven't been.
First, let me list the thing that you haven't been enjoying:
- Whenever and wherever you write about test automation, a handful of readers tend to think that you reject the idea of test automation and they write harsh e-mails to you.
- Whenever and wherever you write about certification, the certified tester community attacks you over e-mails/phone calls that you and I are spoiling the craft.
- Whenever and wherever you write about programming skill for testers, another handful readers think that you are suggesting testers not to learn programming.
- Whenever and wherever you write about tools, most testers think you are referring to test automation tool.
- Whenever and wherever you write about yourself or your experience, a set of people think you lack humility.
- Whenever and wherever you write about exploratory testing as a skilled activity, a set of people think "no tester would do be able to do that".
- Whenever and wherever you write about ideas to solve a problem than giving a one line answer, a set of people think you don't know how to solve it and is faking what you know.
- Whenever and wherever you are writing about testing being an excellent thinking job, a few people think you are trying to paint a picture that does not exist.
- Whenever and wherever you - do bad testing, fail in testing course like BBST, you feel intimidated by more skilled people than you, you feel bad about not having learned or practiced those things that helps you become a better tester, you fail to give enough respect to expert testers time, etc...
I also remember that you had made a note in your Moleskine of Saurav Ganguly's television interview where he was asked: How were you able to make a great comeback to the world cup cricket squad after being axed for poor performance? His answer: I didn't worry about things that are not under my control ( media critique, jokes on bad batting performance, e-mail forwards about my performance, people gossiping about it ) but focussed on things that are under my control ( Practice, skill enhancement, consistent batting record in Ranji trophy)
Similarly, you don't have a control over the thoughts of people thinking whatever they think after reading whatever you have written. You have a control over what you write and you have a control over the way you write it.
Your testing has been influenced by a lot of experts but not all have similar influence. They haven't seen great testing to appreciate the things you are sharing and I doubt if all those who witness it would be influenced by it because it's hard work and high skill demanding.
- Not all testers want to do great testing
- Not all testers know they are doing bad testing.
- Not all testers want to know they are doing bad testing.
- Not all testers want to know more about testing.
- Not all testers know what skills to gain and practice.
- Not all testers agree to be context driven.
Here are three questions ( like the Monty Python and the Holy grail bridge of death piece you enjoy )
1.Whom are you serving through your writing?
I am sure your ongoing struggle is in understanding that. Let me help you with what I think about whom you are serving - You are serving those testers who look for better thought process and those who enjoy the better thought process and those who think you have a better thought process.
2. Who asked you to serve them?
I asked you to do that!
3. Why haven't you been enjoying some moments that I have been?
You want all testers to do great testing although you know its not possible. Some people question your idea of "great testing" because they already have an idea of "great testing" and it conflicts with the idea you have. You are able to demonstrate that their idea of "great testing" lacks critical thought as your idea of "great testing".
By the way, your idea of "great testing", is not yours but of those people who have influenced you. You have just subscribe to those ideas and are contributing to it in different forms. I have occasionally witnessed you doing bad testing and I am sure I would see that in future, too. Do not forget that you are a human and your ideas are fallible. I know bad testing and bad thought process irritates you, even if you are the one who is doing it.
I would love to see you doing things that are under your control - learning, reading, writing, bettering your skills, helping those testers who enjoy the thought process that you enjoy, speaking, coaching and mentoring.
Your power to influence testers is limited. Limited to the ones who don't want to limit themselves. So do unlimited things under limited time that you and I will be here in this world.
I will be with you forever, enjoying everything you do from great things to not so great things. Anything you do is great to me.
I will write to you whenever I feel a need for it. This letter is personal, just between you and me.
"Here is a way to test if your mission on earth is complete - if you are alive, it isn't" -- Richard Bach
Yours truly,
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Wednesday, May 28, 2008
Reduce Reuse Recycle
I toured Singapore for about 10 days this month. I don't know why my eyes kept catching signboards that suggested people to Reduce, Reuse and Recycle plastics. I think it was because I am from India.
As a part of my tour, I had been to a Zoo and there, I heard the animal show host explaining the need to reduce, reuse and recycle as it impacts the environment and animals.
I felt great about Singaporeans, for stressing the point in many places. No wonder their country ( most parts ) is so clean and tidy that I felt I am in a foreign land. I thought for a while that people in India don't talk about Reduce, Reuse and Recycle as much as Singaporeans do.
16 days after the tour ended, I realized I was wrong. People in India talk as much as Singaporeans do about reducing, reusing and recycling. Here is how they ask:
How can I reduce testing and thereby decrease the cost of the project and increase my profitability?
If testing is a job of providing quality related information to the stake holders to help them take better informed what could reducing testing mean?
I think the people who talk about reducing testing to decrease costs want to know if there is
I think people who talk about reusing / recycling test cases, scripts and testers for future projects want to know:
Well, if all those who ask the questions knew how to ask it more elaborate or deeper questions, we'd be living in a different world. Thankfully, we still are in the same world.
How did I reuse, recycle and reduce?
If working on reducing, reusing and recycling test cases aren't working, you might want to reduce your intention of reducing, reusing and recycling /and/ think of reusing those ideas in a different context /or /recycle those ideas at a later date when you think the context has changed to suit it.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
As a part of my tour, I had been to a Zoo and there, I heard the animal show host explaining the need to reduce, reuse and recycle as it impacts the environment and animals.
I felt great about Singaporeans, for stressing the point in many places. No wonder their country ( most parts ) is so clean and tidy that I felt I am in a foreign land. I thought for a while that people in India don't talk about Reduce, Reuse and Recycle as much as Singaporeans do.
16 days after the tour ended, I realized I was wrong. People in India talk as much as Singaporeans do about reducing, reusing and recycling. Here is how they ask:
How can I reduce testing and thereby decrease the cost of the project and increase my profitability?
If testing is a job of providing quality related information to the stake holders to help them take better informed what could reducing testing mean?
I think the people who talk about reducing testing to decrease costs want to know if there is
- a way (or more than one) to make testers more efficient, skilled and competent.
- a way (or more than one) to get information more faster.
- a way (or more than one) to know if they can save costs by not purchasing unnecessary tools.
- a way (or more than one) to know if they can avoid hiring bad candidates.
- a way (or more than one) to know if they could spend less on managing attrition.
- a way (or more than one) to know if they are not doing things that would not add value.
- a way (or more than one) to know if they are meeting the mission.
- a way (or more than one) to know if they can fall into fewer traps.
- a way (or more than one) to know if they can recover much faster from the traps.
- a way (or more than one) to know if they can be more successful in getting more projects.
I think people who talk about reusing / recycling test cases, scripts and testers for future projects want to know:
- many ways (or at least one) to know if they can cut costs by not needing to train testers on a similar domain, technology or product.
- many ways (or at least one) to know if they can cut costs by not having to spend time on writing test cases and test scripts for a new project by modifying existing test cases and test scripts.
- many ways (or at least one) to know if they can cut costs by not having to write code that tests code by modifying existing code.
- many ways (or at least one) to know if they can cut costs by not needing to spend much of a time on learning the new product and finding bugs faster.
- many ways (or at least one) to know if by thinking of reusing and recycling, they are definitely saving costs without sacrificing the value they want to add.
Well, if all those who ask the questions knew how to ask it more elaborate or deeper questions, we'd be living in a different world. Thankfully, we still are in the same world.
How did I reuse, recycle and reduce?
- I don't recommend writing test cases and executing tests with the help of that. Although I am not someone who'd like looking at test cases, there was a context in which I looked in to test case document that someone else had written, to gather ideas for my exploratory testing. That's how I reused a test case. It definitely reduced the cost because I took help of an already existing database of ideas. ( That doesn't mean test cases can be handy for ideas to test. A check list, cheat sheet, mnemonics, heuristics... might do it as well more cost effectively).
- In one of the several product development organizations I worked for, I identified that a tester was not performing fair enough. I probed for his history of performance within the organization in past projects and recommended him to be fired ( of course, I had the authority to recommend ) He was being paid a lot for he had over 7 years experience. The management feared firing him could send wrong signals to other team members but I asked, "What more right a signal can people get?". On firing him we had money to afford hiring 4 junior testers for 1/5th of what he was paid and got more than what he was delivering. That's how I reduced the cost of the product.
If working on reducing, reusing and recycling test cases aren't working, you might want to reduce your intention of reducing, reusing and recycling /and/ think of reusing those ideas in a different context /or /recycle those ideas at a later date when you think the context has changed to suit it.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Tuesday, April 08, 2008
Using "testing" || Abusing "testing"
As you have started to read this post, before you continue further, I'd like you to listen to a podcast - The Word Test . If you are skipping the podcast, it's OK but it might be a good idea to not skip.
Also, don't read this post while you are mid way listening to the podcast, it's more bad than not listening to it.
One of my student who works for a leading IT services provider from India, asked a question to testers in the organization he works for - "Is it good to stop testing after a couple of years of experience or after promoted to a lead or a manager?" [ My intention of this post is not to answer this question that my student asked but ... ]
There were responses like:
One need not do hands-on testing all through his/her career. When you test all by yourself, you are adding a value of say 'X' to your project. When you manage say 5 Testers, you are letting your skills, knowledge and experience on Testing percolate to 5 other members and you would be adding a value of 5 times X to your project.
and
Whatever you have said is ok for a resource working in a team which uses tools for testing. For someone who is into manual testing where is the career growth? For those wouldn't management be a blessing to be grabbed with both hands?
and
You cant be a tester for all your life. Same is the case with development. You need to manage things at one point in time. But when and where, you need to decide yourself......
and
more such.
All of these people ( including my student ) and maybe testers who are sitting nearby your cubicles while you read this mean "testing" as test execution. [ You wouldn't be surprised at this, had you listened to the podcast ]
Here is my question to those people: If testing means test execution, under what category does - test planning, test data collection, finding bugs, reporting bugs, triaging bugs, test set up, test bed creation, test documentation, thinking of test techniques, exploring, investigating bugs, reviewing test results, test reporting, modeling, diversifying test approaches, etc... fit in?
Well, when the word testing could mean so many things, why are most of us thinking only about test execution when someone uses the word "testing". This makes me question, how many people who claim to be testers really know little about testing that is enough to communicate with people without such ambiguity?
A lot of testers' only thinking is -- every thing in this field is defined pretty well and no need to think beyond it. A definition, in my opinion, should be viewed as a help for a human to think further on it and not in stopping to think beyond what it states.
In another context, if you ask them what "testing" means, they'd love to share their own impractical definitions like:
"Testing is a process of making a product bug free" OR "Testing is a systematic approach towards delivering a quality product" OR "Testing is about following quality processes to ensure bugs don't leak to customer"
and more such stupid stuff !
That's an evidence that the word "testing" itself is context sensitively used by the whole world out of which most of them might disagree with the context driven testing community about their approach. Funny world!
Ben Simo, in a recent conversation, helped me become conscious of the fact that the word "test" is both a noun and a verb; and that one feeds the other.
If one doesn't know what "testing" means, how will they ever know when they are stopping to do it?
If you think you have benefited by this post, here is a "test" you might want to take:
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Also, don't read this post while you are mid way listening to the podcast, it's more bad than not listening to it.
One of my student who works for a leading IT services provider from India, asked a question to testers in the organization he works for - "Is it good to stop testing after a couple of years of experience or after promoted to a lead or a manager?" [ My intention of this post is not to answer this question that my student asked but ... ]
There were responses like:
One need not do hands-on testing all through his/her career. When you test all by yourself, you are adding a value of say 'X' to your project. When you manage say 5 Testers, you are letting your skills, knowledge and experience on Testing percolate to 5 other members and you would be adding a value of 5 times X to your project.
and
Whatever you have said is ok for a resource working in a team which uses tools for testing. For someone who is into manual testing where is the career growth? For those wouldn't management be a blessing to be grabbed with both hands?
and
You cant be a tester for all your life. Same is the case with development. You need to manage things at one point in time. But when and where, you need to decide yourself......
and
more such.
All of these people ( including my student ) and maybe testers who are sitting nearby your cubicles while you read this mean "testing" as test execution. [ You wouldn't be surprised at this, had you listened to the podcast ]
Here is my question to those people: If testing means test execution, under what category does - test planning, test data collection, finding bugs, reporting bugs, triaging bugs, test set up, test bed creation, test documentation, thinking of test techniques, exploring, investigating bugs, reviewing test results, test reporting, modeling, diversifying test approaches, etc... fit in?
Well, when the word testing could mean so many things, why are most of us thinking only about test execution when someone uses the word "testing". This makes me question, how many people who claim to be testers really know little about testing that is enough to communicate with people without such ambiguity?
A lot of testers' only thinking is -- every thing in this field is defined pretty well and no need to think beyond it. A definition, in my opinion, should be viewed as a help for a human to think further on it and not in stopping to think beyond what it states.
In another context, if you ask them what "testing" means, they'd love to share their own impractical definitions like:
"Testing is a process of making a product bug free" OR "Testing is a systematic approach towards delivering a quality product" OR "Testing is about following quality processes to ensure bugs don't leak to customer"
and more such stupid stuff !
That's an evidence that the word "testing" itself is context sensitively used by the whole world out of which most of them might disagree with the context driven testing community about their approach. Funny world!
Ben Simo, in a recent conversation, helped me become conscious of the fact that the word "test" is both a noun and a verb; and that one feeds the other.
If one doesn't know what "testing" means, how will they ever know when they are stopping to do it?
If you think you have benefited by this post, here is a "test" you might want to take:
- What would you say, when you want to communicate that you are doing test execution?
- What would you say, when you want to communicate that you are stopping to execute tests?
- What word would you use instead of "testing" to communicate any specific activity that you do as a part of testing?
- When someone uses the word "testing", what would you want to ask them?
- When someone says "test", would you be curious to know if it is a verb or a noun?
- What would you want to know if someone said, "I want to do testing"?
- Would you be interested to pass this learning to someone with whom you have been communicating on "testing"?
- Read it once again ;-)
- Listen to the podcast, if you have missed it ;-)
- And then exit. It's just not worth one more glance, for today.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Sunday, March 30, 2008
Graphology, Music, Chess, Cooking, Drawing and becoming a better tester
I am observing that every passionate learning tester have tried doing things that are not testing and try to bring in some learnings to testing from doing them, to test better. For instance, Michael Bolton has tried cooking and theater and has benefited by it in testing, you could read his interview in which he mentions that. Michael Hunter (a.k.a The Braidy Tester) has recently blogged about Drawing and how it has been helping him to test and think about testing. Jonathon Kohl's Exploratory Testing-Music of Investigation is no exception. Bach brothers ( James Bach and Jon Bach) are active in Chess.com and you could read how playing Chess helps to test better by reading Jon Bach's post.
CAST 08 has its theme "Beyond the Boundaries: Interdisciplinary Approaches to Software Testing". Interdisciplinary approaches drawn from diversified branches of learning or practice, such that insights can be drawn upon and synthesized to influence a particular craft.
Anuj Magazine, an experienced tester from India has been working for quite sometime on Graphology and Software Testing.
I wanted aid his research and agreed to share my handwriting because his research is to identify the traits in successful testers. You can read the analysis of my handwriting by clicking on this link. I hope you'd enjoy reading the analysis and might be of help to Anuj if he approaches you.
I am sure there a lot of untold stories of what else other than testing has helped them to test better. I hope a lot of other testers come out with those stories that might benefit the community.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
CAST 08 has its theme "Beyond the Boundaries: Interdisciplinary Approaches to Software Testing". Interdisciplinary approaches drawn from diversified branches of learning or practice, such that insights can be drawn upon and synthesized to influence a particular craft.
Anuj Magazine, an experienced tester from India has been working for quite sometime on Graphology and Software Testing.
I wanted aid his research and agreed to share my handwriting because his research is to identify the traits in successful testers. You can read the analysis of my handwriting by clicking on this link. I hope you'd enjoy reading the analysis and might be of help to Anuj if he approaches you.
I am sure there a lot of untold stories of what else other than testing has helped them to test better. I hope a lot of other testers come out with those stories that might benefit the community.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Tuesday, March 18, 2008
Automation replaces humans - The truth about what kind of humans it replaces
I often meet people who claim to be "automation testers" and I love and hate to argue with them about test automation. The more I meet such people, I am getting confident about making a conjecture that more than many people who claim to be doing test automating in India and other APAC country testers I visited, are doing it without knowing it or without wanting to know it.
Here are the things they say:
Test automation replaces manual testing and Test automation is the future
There are two planes standing there. One of them was tested by a trillion scripts that flew the plane for 400 hours and the other was tested by those 4 humans, flying 100 hours on it. If you were to board one of them, which one would you board and why?
None of those great human beings who said, "Test automation replaces manual testing" said, "I shall board the plane which was tested by a trillion scripts" and instead said, "I will take the plane that humans tested because I feel safer throughout the flight"
So did test automation replace manual testing even for those automating tests?
Test automation is the future of what?
Test automation is faster than manual testing & Test automation is cheaper than manual testing
Here are some statements I make. You can pick ones that makes most sense to you:
A. My Toshiba laptop runs faster than a McLaren Mercedes Formula1 car.
B. My Toshiba laptop runs faster than my old Acer laptop.
C. My Toshiba laptop runs faster on an XP than on Vista.
D. My Toshiba laptop runs faster than that horse which has a broken leg.
E. When someone needs help and I need to run to them, I use a test automation tool because that's faster.
F. QTP license is cheaper than hiring a 5 year experienced skilled tester.
G. QTP is expensive than Winrunner.
I. QTP is cheaper than a Ferrari engine.
J. Winrunner is cheaper than a business class ticket from Bangalore-Toronto.
Option C and G makes most sense. Why? Why not A , D, E, F, G, I, and J?
"Well, option C and G makes most sense to me because you are comparing the same Toshiba laptop when Vista and XP runs on it AND you are comparing two testing tools in G"
Is test automation [ using a software or a computer ] faster than humans to those who say that?
Is test automation cheaper than those human brains who claim to create it?
Test Automation is more reliable than manual testing
I first heard this eye opener, insightful idea from Michael Bolton , "People write code to find bugs on another code. They write code that is buggy which claims to find bugs on another code" and "Writing test scripts is another development project . Most of them don't realize that they are running two development projects in parallel and hence their main development project suffers."
So is test automation reliable than manual testing to those who write code that test another code?
We humans are fallible and all things we create are likely to be fallible, too. Test automation scripts are not excused by God to feel "All test scripts don't have bugs".
That doesn't mean human testing is more reliable. It means, anything done by humans is fallible. When you depend on a fallible thing that a fallible human created, you might not want to talk about reliability there.
Test automation removes the monotonous work of a manual tester
Why is testing monotonous? The unspoken truth!
Test automation is better than manual testing as a career option
Well, people think by becoming an automation tester they aren't doing manual testing. Here is what I have discovered - There are more people who claim themselves as automation testers in India than those who claim to be manual testers. So, let me know which is not manual testing? Can manual be termed as anything that does involve humans?
Lack of skilled testers at least has helped me grow phenomenal in my career. I am not THE skilled but I claim to be one of them and there might be very few like me and not many in India.
Test automation can find a lot more bugs than a manual tester & Test automation provides confidence that the product works
Yes, test automation is very intelligent. If a script is programmed to catch a bug when a value exceeds 5, it can also notice that the value did not exceed 5 and pass the test but the screen went blue for a while. If after all scripts have run and there aren't any bugs found, the test automation suite will alter its values and run once again to catch bugs. As the scripts run over and over again, the scripts learn the product well and if the management needs to take a decision to ship the product they would come near the script and say,
"Oh magical script, I want to ship this product. If you think it's a bad idea to ship now, speak out else remain silent"and then they listen to the script and decide whether to ship or not.
Yes, we are dumb. We can see a numerical value not exceeding 5 and pass the test case but also we can't spot the screen becoming blue for a while and do not report that as an issue while running a test that we claim to pass.
Test automation is about converting manual test cases to automation
Oh wow! So, I can't write automation scripts without writing cases and executing it at least once by my own? And if I write, many might cry "Foul".
Well, test automation scripts are just another program as the program it tests. Did developers have something they did manually before they wrote the code that you test with your manually converted automation test scripts?
Test automation tools provides Return On Investment.
Ok, so here is the industry standard of returns on a tool: For every $500 invested on a tool, a good ROI is about the tool helping the team find 50 bugs.
So, a good ROI for $9000 tool = ( 9000 / 500 ) * 50 = 900 bugs
If that doesn't make sense to you then what is the Return on Investment for a tool?
Bugs? Customer appreciation? The number of testers who get fired? The cost savings of hiring 10 skilled testers v/s the cost of buying the tool? The value that the tool brings to the testing effort? Tons of documentation produced? The added cost of training? The added cost of support? The added confusion that prevails? The number of fake experienced testers who apply to the job of a toolsmith?
Test automation is better than manual testing as a career option
Test automation, at least in India, is more chaotic than the so called manual testing. Hardly any tester knows why they are automating things. You may join any forum where testers are active discussing tools and automation, you'd see questions that the help file of the tool has answers and you would discover replies to such questions that is sometimes horribly wrong and yet the person who asked the question says, "Thanks" and the chaos continues.
There is more chaos among so called automation testers than so called manual testers because the so called automation testers form a huge community than the so called manual testers. More people with lack of insightful ideas means more chaos.
Want to know some useful information about automation or want ideas to think more about it?
Cem Kaner prefers to call most of the automation activities that most testers do as Computer Assisted Testing and I couldn't find evidence to refute the idea. I think it is a great idea to call it "Computer Assisted Testing" and you would know why it is a great idea if you go through this article by Cem or the PDF version of it.
So, all of us irrespective of calling us manual or automation testers do Computer Assisted Testing.
I know of an article written by James Bach that made many people who claims a lot of things about test automation feel guilty and I am sure if you have reached this place reading this post of mine, you'd be interested at reading James Bach's Test Automation Snake Oil - the paper or the slides .
I also know of a blog post from James - Manual tests cannot be automated and Shrini's post on A Mystery called automated testing.
Here is why we do different things in testing such as writing test code, executing test ideas, investigating, diversify approaches, think of different techniques, use heuristics, use oracles, Explore... - because each of them add different kind of value and each of them helps in finding different kinds of information or helps us ask new questions AND NOT because something is better than the other.
There is lot of thinking, reading and writing, I and you require, to know about test automation or about testing. The lazy ones don't get to learn and continue to spoil our craft because they don't stop writing, just like me. [ It's intentional ]
I know I haven't answered a question that you might have in mind but that's intentional again.
Update: Here is Jonathan Kohl's awesome interview about the same topic. Thanks to Konstantin for pointing it out through a comment.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Here are the things they say:
- Test automation replaces manual testing.
- Test automation is faster than manual testing.
- Test automation is cheaper than manual testing.
- Test automation is more reliable than manual testing.
- Test automation removes the monotonous work of a manual tester.
- Test automation provides confidence that the product works.
- Test automation is the future.
- Test automation is better than manual testing as a career option.
- Test automation tools provides Return On Investment.
- Test automation can find a lot more bugs than a manual tester.
- Test automation provides more job opportunities.
- Test automation is a way to go about testing.
- Test automation is about converting manual test cases to automation.
Test automation replaces manual testing and Test automation is the future
There are two planes standing there. One of them was tested by a trillion scripts that flew the plane for 400 hours and the other was tested by those 4 humans, flying 100 hours on it. If you were to board one of them, which one would you board and why?
None of those great human beings who said, "Test automation replaces manual testing" said, "I shall board the plane which was tested by a trillion scripts" and instead said, "I will take the plane that humans tested because I feel safer throughout the flight"
So did test automation replace manual testing even for those automating tests?
Test automation is the future of what?
Test automation is faster than manual testing & Test automation is cheaper than manual testing
Here are some statements I make. You can pick ones that makes most sense to you:
A. My Toshiba laptop runs faster than a McLaren Mercedes Formula1 car.
B. My Toshiba laptop runs faster than my old Acer laptop.
C. My Toshiba laptop runs faster on an XP than on Vista.
D. My Toshiba laptop runs faster than that horse which has a broken leg.
E. When someone needs help and I need to run to them, I use a test automation tool because that's faster.
F. QTP license is cheaper than hiring a 5 year experienced skilled tester.
G. QTP is expensive than Winrunner.
I. QTP is cheaper than a Ferrari engine.
J. Winrunner is cheaper than a business class ticket from Bangalore-Toronto.
Option C and G makes most sense. Why? Why not A , D, E, F, G, I, and J?
"Well, option C and G makes most sense to me because you are comparing the same Toshiba laptop when Vista and XP runs on it AND you are comparing two testing tools in G"
Is test automation [ using a software or a computer ] faster than humans to those who say that?
Is test automation cheaper than those human brains who claim to create it?
Test Automation is more reliable than manual testing
I first heard this eye opener, insightful idea from Michael Bolton , "People write code to find bugs on another code. They write code that is buggy which claims to find bugs on another code" and "Writing test scripts is another development project . Most of them don't realize that they are running two development projects in parallel and hence their main development project suffers."
So is test automation reliable than manual testing to those who write code that test another code?
We humans are fallible and all things we create are likely to be fallible, too. Test automation scripts are not excused by God to feel "All test scripts don't have bugs".
That doesn't mean human testing is more reliable. It means, anything done by humans is fallible. When you depend on a fallible thing that a fallible human created, you might not want to talk about reliability there.
Test automation removes the monotonous work of a manual tester
Why is testing monotonous? The unspoken truth!
Test automation is better than manual testing as a career option
Well, people think by becoming an automation tester they aren't doing manual testing. Here is what I have discovered - There are more people who claim themselves as automation testers in India than those who claim to be manual testers. So, let me know which is not manual testing? Can manual be termed as anything that does involve humans?
Lack of skilled testers at least has helped me grow phenomenal in my career. I am not THE skilled but I claim to be one of them and there might be very few like me and not many in India.
Test automation can find a lot more bugs than a manual tester & Test automation provides confidence that the product works
Yes, test automation is very intelligent. If a script is programmed to catch a bug when a value exceeds 5, it can also notice that the value did not exceed 5 and pass the test but the screen went blue for a while. If after all scripts have run and there aren't any bugs found, the test automation suite will alter its values and run once again to catch bugs. As the scripts run over and over again, the scripts learn the product well and if the management needs to take a decision to ship the product they would come near the script and say,
"Oh magical script, I want to ship this product. If you think it's a bad idea to ship now, speak out else remain silent"and then they listen to the script and decide whether to ship or not.
Yes, we are dumb. We can see a numerical value not exceeding 5 and pass the test case but also we can't spot the screen becoming blue for a while and do not report that as an issue while running a test that we claim to pass.
Test automation is about converting manual test cases to automation
Oh wow! So, I can't write automation scripts without writing cases and executing it at least once by my own? And if I write, many might cry "Foul".
Well, test automation scripts are just another program as the program it tests. Did developers have something they did manually before they wrote the code that you test with your manually converted automation test scripts?
Test automation tools provides Return On Investment.
Ok, so here is the industry standard of returns on a tool: For every $500 invested on a tool, a good ROI is about the tool helping the team find 50 bugs.
So, a good ROI for $9000 tool = ( 9000 / 500 ) * 50 = 900 bugs
If that doesn't make sense to you then what is the Return on Investment for a tool?
Bugs? Customer appreciation? The number of testers who get fired? The cost savings of hiring 10 skilled testers v/s the cost of buying the tool? The value that the tool brings to the testing effort? Tons of documentation produced? The added cost of training? The added cost of support? The added confusion that prevails? The number of fake experienced testers who apply to the job of a toolsmith?
Test automation is better than manual testing as a career option
Test automation, at least in India, is more chaotic than the so called manual testing. Hardly any tester knows why they are automating things. You may join any forum where testers are active discussing tools and automation, you'd see questions that the help file of the tool has answers and you would discover replies to such questions that is sometimes horribly wrong and yet the person who asked the question says, "Thanks" and the chaos continues.
There is more chaos among so called automation testers than so called manual testers because the so called automation testers form a huge community than the so called manual testers. More people with lack of insightful ideas means more chaos.
Want to know some useful information about automation or want ideas to think more about it?
Cem Kaner prefers to call most of the automation activities that most testers do as Computer Assisted Testing and I couldn't find evidence to refute the idea. I think it is a great idea to call it "Computer Assisted Testing" and you would know why it is a great idea if you go through this article by Cem or the PDF version of it.
So, all of us irrespective of calling us manual or automation testers do Computer Assisted Testing.
I know of an article written by James Bach that made many people who claims a lot of things about test automation feel guilty and I am sure if you have reached this place reading this post of mine, you'd be interested at reading James Bach's Test Automation Snake Oil - the paper or the slides .
I also know of a blog post from James - Manual tests cannot be automated and Shrini's post on A Mystery called automated testing.
Here is why we do different things in testing such as writing test code, executing test ideas, investigating, diversify approaches, think of different techniques, use heuristics, use oracles, Explore... - because each of them add different kind of value and each of them helps in finding different kinds of information or helps us ask new questions AND NOT because something is better than the other.
There is lot of thinking, reading and writing, I and you require, to know about test automation or about testing. The lazy ones don't get to learn and continue to spoil our craft because they don't stop writing, just like me. [ It's intentional ]
I know I haven't answered a question that you might have in mind but that's intentional again.
Update: Here is Jonathan Kohl's awesome interview about the same topic. Thanks to Konstantin for pointing it out through a comment.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Wednesday, February 20, 2008
The Bangalore Roti Curry! Now comes with free* testing lessons
Yesterday, I presented a guest lecture on "Questioning Specification and Requirements as a skilled tester" at Edista Testing's newly launched, really unique career program, which I think, can be titled as "Keep learning software testing and we'll keep supporting you to do that" as they intend to keep track of the progress of one's testing career, knowledge and skills even after the completion of the program or after the placement.
One hour after the guest lecture, I hit the treadmill at a gym nearby my home. I love pace walking over the treadmill. Once I get my rhythm , I start thinking about something that doesn't make me focus on how tired I get or how long I have been on a treadmill. Yesterday's treadmill thoughts were about questioning specification, requirement, process, policies, rules, guidelines, best practices...
The common thing about the above list is, "Someone says it, others follow it". Topic I chose to think on the treadmill was interesting enough to help me burn a lot more. I guess for people like me, thinking about following a requirement/rules/policies/process/best practices without questioning burns more calories than running heavy on a treadmill for 30 minutes.
After the workout, I dropped in at a nearby popular and reputed food joint for dinner at Bannerghatta Road, Bangalore.
I bought a token for 1 plate of Roti and curry ( Indian bread ) Cost: 16 rupees. After I got the token in hand, I realized it would not be sufficient and asked if I could get an extra Roti. I had to pay 10 rupees to buy an extra Roti token.
While eating one of the two Rotis, I realized that the curry isn't sufficient for the next Roti and I asked for more curry, I was asked to pay 10 rupees more for extra curry that is of same quantity provided to me the first time.
So here is the interesting thing:
One plate Roti and curry is: 16 rupees : [ 1 Roti + 1 set Curry ]
If I buy an extra Roti and Extra curry: that's 10 + 10 = 20 rupees
Total: 36 rupees [ 2 * 1 Roti + 2 * 1 set Curry ]
If I buy two plate Roti and curry: 16 * 2 = 32 rupees [ 2 * 1 Roti + 2 * 1 set Curry ]
I went to the hotel supervisor and said, "I have never come across any hotel other than yours which has two different prices for the same quantity and item served if purchased in a different order. Is that intended?" and then explained to him the sequence in which I bought Roti and Curry. You might not believe this: He nor the other staff knew that they had been charging customers different rate for the same quantity they buy an item based on the order in which they buy things.
This certainly surprised their staff and I could witness an argument over the issue among other staff members. There was excitement over the issue and I enjoyed the argument between them.
Supervisor of the hotel then assured me that they would fix this issue after consulting the hotel manager and thanked me with a smile for bringing this up.
I was happy. As a tester, I questioned things and provided them information about their service with evidence. I handled it politely and helped them understand how illogical it would sound to people and how their credibility is at stake if they continue to do so.
If I hadn't questioned at the logic, or their menu, or their process, or their specification, I wouldn't have helped them understand something was seriously wrong - because they seem to be concerned to not do anything wrong as it could affect their reputation. They are into this business for at least 5 years and they hadn't realized what rule they set could conflict with their own menu rates.
When customers say "We want A,B, C, D, E, F", they are less aware that A could conflict with E unless we ( testers ) help them understand and that's our job. If we (great testers) don't question requirement/specification/process/rule/guideline/best practices, and the customer by himself finds B conflicts with D at a later stage, he might think of us as fools and maybe get our butts fired.
One hour after the guest lecture, I hit the treadmill at a gym nearby my home. I love pace walking over the treadmill. Once I get my rhythm , I start thinking about something that doesn't make me focus on how tired I get or how long I have been on a treadmill. Yesterday's treadmill thoughts were about questioning specification, requirement, process, policies, rules, guidelines, best practices...
The common thing about the above list is, "Someone says it, others follow it". Topic I chose to think on the treadmill was interesting enough to help me burn a lot more. I guess for people like me, thinking about following a requirement/rules/policies/process/best practices without questioning burns more calories than running heavy on a treadmill for 30 minutes.
After the workout, I dropped in at a nearby popular and reputed food joint for dinner at Bannerghatta Road, Bangalore.
I bought a token for 1 plate of Roti and curry ( Indian bread ) Cost: 16 rupees. After I got the token in hand, I realized it would not be sufficient and asked if I could get an extra Roti. I had to pay 10 rupees to buy an extra Roti token.
While eating one of the two Rotis, I realized that the curry isn't sufficient for the next Roti and I asked for more curry, I was asked to pay 10 rupees more for extra curry that is of same quantity provided to me the first time.
So here is the interesting thing:
One plate Roti and curry is: 16 rupees : [ 1 Roti + 1 set Curry ]
If I buy an extra Roti and Extra curry: that's 10 + 10 = 20 rupees
Total: 36 rupees [ 2 * 1 Roti + 2 * 1 set Curry ]
If I buy two plate Roti and curry: 16 * 2 = 32 rupees [ 2 * 1 Roti + 2 * 1 set Curry ]
I went to the hotel supervisor and said, "I have never come across any hotel other than yours which has two different prices for the same quantity and item served if purchased in a different order. Is that intended?" and then explained to him the sequence in which I bought Roti and Curry. You might not believe this: He nor the other staff knew that they had been charging customers different rate for the same quantity they buy an item based on the order in which they buy things.
This certainly surprised their staff and I could witness an argument over the issue among other staff members. There was excitement over the issue and I enjoyed the argument between them.
Supervisor of the hotel then assured me that they would fix this issue after consulting the hotel manager and thanked me with a smile for bringing this up.
I was happy. As a tester, I questioned things and provided them information about their service with evidence. I handled it politely and helped them understand how illogical it would sound to people and how their credibility is at stake if they continue to do so.
If I hadn't questioned at the logic, or their menu, or their process, or their specification, I wouldn't have helped them understand something was seriously wrong - because they seem to be concerned to not do anything wrong as it could affect their reputation. They are into this business for at least 5 years and they hadn't realized what rule they set could conflict with their own menu rates.
When customers say "We want A,B, C, D, E, F", they are less aware that A could conflict with E unless we ( testers ) help them understand and that's our job. If we (great testers) don't question requirement/specification/process/rule/guideline/best practices, and the customer by himself finds B conflicts with D at a later stage, he might think of us as fools and maybe get our butts fired.
"If testers aren't questioning, they aren't testing. If testers aren't testing then they aren't testers anymore".
In the Roti Curry story above, I think there is another lesson hiding. As a customer of the hotel, I realized I wanted more, only when I got some bit of it - which means I didn't know how much I needed to buy till I saw some sample. If someone were to have collected my requirement/need, I would have stated something and changed it because I am not sure what I want.
There are a lot of customers who are not sure what they want and if we ( great testers ) are to follow a document that was prepared so early in the project before the customer could see something, we would never be doing what we should be doing. Testing is *not* satisfying a customer but providing information to help the management or customer take informed decisions. The information we provide could be bugs, assessment of risks, questions we have, answers we found...
Little does the hotel know that testing lessons are for free when people buy Roti and Curry.
Laugh (if you want): Research Doctors at National Health Research Institute are recommending to take a minute of every day, especially for those in software testing, to laugh out loud at those testers who don't question the specification/requirements/test scripts/test cases/process/best practices.
It is found through their research that the health condition and testing has had a significant improvement by doing so. Researchers also encourage laughing at yourself, if you are one among the testers who don't question.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
In the Roti Curry story above, I think there is another lesson hiding. As a customer of the hotel, I realized I wanted more, only when I got some bit of it - which means I didn't know how much I needed to buy till I saw some sample. If someone were to have collected my requirement/need, I would have stated something and changed it because I am not sure what I want.
There are a lot of customers who are not sure what they want and if we ( great testers ) are to follow a document that was prepared so early in the project before the customer could see something, we would never be doing what we should be doing. Testing is *not* satisfying a customer but providing information to help the management or customer take informed decisions. The information we provide could be bugs, assessment of risks, questions we have, answers we found...
Little does the hotel know that testing lessons are for free when people buy Roti and Curry.
Laugh (if you want): Research Doctors at National Health Research Institute are recommending to take a minute of every day, especially for those in software testing, to laugh out loud at those testers who don't question the specification/requirements/test scripts/test cases/process/best practices.
It is found through their research that the health condition and testing has had a significant improvement by doing so. Researchers also encourage laughing at yourself, if you are one among the testers who don't question.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Subscribe to:
Posts (Atom)