I am observing that every passionate learning tester have tried doing things that are not testing and try to bring in some learnings to testing from doing them, to test better. For instance, Michael Bolton has tried cooking and theater and has benefited by it in testing, you could read his interview in which he mentions that. Michael Hunter (a.k.a The Braidy Tester) has recently blogged about Drawing and how it has been helping him to test and think about testing. Jonathon Kohl's Exploratory Testing-Music of Investigation is no exception. Bach brothers ( James Bach and Jon Bach) are active in Chess.com and you could read how playing Chess helps to test better by reading Jon Bach's post.
CAST 08 has its theme "Beyond the Boundaries: Interdisciplinary Approaches to Software Testing". Interdisciplinary approaches drawn from diversified branches of learning or practice, such that insights can be drawn upon and synthesized to influence a particular craft.
Anuj Magazine, an experienced tester from India has been working for quite sometime on Graphology and Software Testing.
I wanted aid his research and agreed to share my handwriting because his research is to identify the traits in successful testers. You can read the analysis of my handwriting by clicking on this link. I hope you'd enjoy reading the analysis and might be of help to Anuj if he approaches you.
I am sure there a lot of untold stories of what else other than testing has helped them to test better. I hope a lot of other testers come out with those stories that might benefit the community.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Sunday, March 30, 2008
Tuesday, March 18, 2008
Automation replaces humans - The truth about what kind of humans it replaces
I often meet people who claim to be "automation testers" and I love and hate to argue with them about test automation. The more I meet such people, I am getting confident about making a conjecture that more than many people who claim to be doing test automating in India and other APAC country testers I visited, are doing it without knowing it or without wanting to know it.
Here are the things they say:
Test automation replaces manual testing and Test automation is the future
There are two planes standing there. One of them was tested by a trillion scripts that flew the plane for 400 hours and the other was tested by those 4 humans, flying 100 hours on it. If you were to board one of them, which one would you board and why?
None of those great human beings who said, "Test automation replaces manual testing" said, "I shall board the plane which was tested by a trillion scripts" and instead said, "I will take the plane that humans tested because I feel safer throughout the flight"
So did test automation replace manual testing even for those automating tests?
Test automation is the future of what?
Test automation is faster than manual testing & Test automation is cheaper than manual testing
Here are some statements I make. You can pick ones that makes most sense to you:
A. My Toshiba laptop runs faster than a McLaren Mercedes Formula1 car.
B. My Toshiba laptop runs faster than my old Acer laptop.
C. My Toshiba laptop runs faster on an XP than on Vista.
D. My Toshiba laptop runs faster than that horse which has a broken leg.
E. When someone needs help and I need to run to them, I use a test automation tool because that's faster.
F. QTP license is cheaper than hiring a 5 year experienced skilled tester.
G. QTP is expensive than Winrunner.
I. QTP is cheaper than a Ferrari engine.
J. Winrunner is cheaper than a business class ticket from Bangalore-Toronto.
Option C and G makes most sense. Why? Why not A , D, E, F, G, I, and J?
"Well, option C and G makes most sense to me because you are comparing the same Toshiba laptop when Vista and XP runs on it AND you are comparing two testing tools in G"
Is test automation [ using a software or a computer ] faster than humans to those who say that?
Is test automation cheaper than those human brains who claim to create it?
Test Automation is more reliable than manual testing
I first heard this eye opener, insightful idea from Michael Bolton , "People write code to find bugs on another code. They write code that is buggy which claims to find bugs on another code" and "Writing test scripts is another development project . Most of them don't realize that they are running two development projects in parallel and hence their main development project suffers."
So is test automation reliable than manual testing to those who write code that test another code?
We humans are fallible and all things we create are likely to be fallible, too. Test automation scripts are not excused by God to feel "All test scripts don't have bugs".
That doesn't mean human testing is more reliable. It means, anything done by humans is fallible. When you depend on a fallible thing that a fallible human created, you might not want to talk about reliability there.
Test automation removes the monotonous work of a manual tester
Why is testing monotonous? The unspoken truth!
Test automation is better than manual testing as a career option
Well, people think by becoming an automation tester they aren't doing manual testing. Here is what I have discovered - There are more people who claim themselves as automation testers in India than those who claim to be manual testers. So, let me know which is not manual testing? Can manual be termed as anything that does involve humans?
Lack of skilled testers at least has helped me grow phenomenal in my career. I am not THE skilled but I claim to be one of them and there might be very few like me and not many in India.
Test automation can find a lot more bugs than a manual tester & Test automation provides confidence that the product works
Yes, test automation is very intelligent. If a script is programmed to catch a bug when a value exceeds 5, it can also notice that the value did not exceed 5 and pass the test but the screen went blue for a while. If after all scripts have run and there aren't any bugs found, the test automation suite will alter its values and run once again to catch bugs. As the scripts run over and over again, the scripts learn the product well and if the management needs to take a decision to ship the product they would come near the script and say,
"Oh magical script, I want to ship this product. If you think it's a bad idea to ship now, speak out else remain silent"and then they listen to the script and decide whether to ship or not.
Yes, we are dumb. We can see a numerical value not exceeding 5 and pass the test case but also we can't spot the screen becoming blue for a while and do not report that as an issue while running a test that we claim to pass.
Test automation is about converting manual test cases to automation
Oh wow! So, I can't write automation scripts without writing cases and executing it at least once by my own? And if I write, many might cry "Foul".
Well, test automation scripts are just another program as the program it tests. Did developers have something they did manually before they wrote the code that you test with your manually converted automation test scripts?
Test automation tools provides Return On Investment.
Ok, so here is the industry standard of returns on a tool: For every $500 invested on a tool, a good ROI is about the tool helping the team find 50 bugs.
So, a good ROI for $9000 tool = ( 9000 / 500 ) * 50 = 900 bugs
If that doesn't make sense to you then what is the Return on Investment for a tool?
Bugs? Customer appreciation? The number of testers who get fired? The cost savings of hiring 10 skilled testers v/s the cost of buying the tool? The value that the tool brings to the testing effort? Tons of documentation produced? The added cost of training? The added cost of support? The added confusion that prevails? The number of fake experienced testers who apply to the job of a toolsmith?
Test automation is better than manual testing as a career option
Test automation, at least in India, is more chaotic than the so called manual testing. Hardly any tester knows why they are automating things. You may join any forum where testers are active discussing tools and automation, you'd see questions that the help file of the tool has answers and you would discover replies to such questions that is sometimes horribly wrong and yet the person who asked the question says, "Thanks" and the chaos continues.
There is more chaos among so called automation testers than so called manual testers because the so called automation testers form a huge community than the so called manual testers. More people with lack of insightful ideas means more chaos.
Want to know some useful information about automation or want ideas to think more about it?
Cem Kaner prefers to call most of the automation activities that most testers do as Computer Assisted Testing and I couldn't find evidence to refute the idea. I think it is a great idea to call it "Computer Assisted Testing" and you would know why it is a great idea if you go through this article by Cem or the PDF version of it.
So, all of us irrespective of calling us manual or automation testers do Computer Assisted Testing.
I know of an article written by James Bach that made many people who claims a lot of things about test automation feel guilty and I am sure if you have reached this place reading this post of mine, you'd be interested at reading James Bach's Test Automation Snake Oil - the paper or the slides .
I also know of a blog post from James - Manual tests cannot be automated and Shrini's post on A Mystery called automated testing.
Here is why we do different things in testing such as writing test code, executing test ideas, investigating, diversify approaches, think of different techniques, use heuristics, use oracles, Explore... - because each of them add different kind of value and each of them helps in finding different kinds of information or helps us ask new questions AND NOT because something is better than the other.
There is lot of thinking, reading and writing, I and you require, to know about test automation or about testing. The lazy ones don't get to learn and continue to spoil our craft because they don't stop writing, just like me. [ It's intentional ]
I know I haven't answered a question that you might have in mind but that's intentional again.
Update: Here is Jonathan Kohl's awesome interview about the same topic. Thanks to Konstantin for pointing it out through a comment.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Here are the things they say:
- Test automation replaces manual testing.
- Test automation is faster than manual testing.
- Test automation is cheaper than manual testing.
- Test automation is more reliable than manual testing.
- Test automation removes the monotonous work of a manual tester.
- Test automation provides confidence that the product works.
- Test automation is the future.
- Test automation is better than manual testing as a career option.
- Test automation tools provides Return On Investment.
- Test automation can find a lot more bugs than a manual tester.
- Test automation provides more job opportunities.
- Test automation is a way to go about testing.
- Test automation is about converting manual test cases to automation.
Test automation replaces manual testing and Test automation is the future
There are two planes standing there. One of them was tested by a trillion scripts that flew the plane for 400 hours and the other was tested by those 4 humans, flying 100 hours on it. If you were to board one of them, which one would you board and why?
None of those great human beings who said, "Test automation replaces manual testing" said, "I shall board the plane which was tested by a trillion scripts" and instead said, "I will take the plane that humans tested because I feel safer throughout the flight"
So did test automation replace manual testing even for those automating tests?
Test automation is the future of what?
Test automation is faster than manual testing & Test automation is cheaper than manual testing
Here are some statements I make. You can pick ones that makes most sense to you:
A. My Toshiba laptop runs faster than a McLaren Mercedes Formula1 car.
B. My Toshiba laptop runs faster than my old Acer laptop.
C. My Toshiba laptop runs faster on an XP than on Vista.
D. My Toshiba laptop runs faster than that horse which has a broken leg.
E. When someone needs help and I need to run to them, I use a test automation tool because that's faster.
F. QTP license is cheaper than hiring a 5 year experienced skilled tester.
G. QTP is expensive than Winrunner.
I. QTP is cheaper than a Ferrari engine.
J. Winrunner is cheaper than a business class ticket from Bangalore-Toronto.
Option C and G makes most sense. Why? Why not A , D, E, F, G, I, and J?
"Well, option C and G makes most sense to me because you are comparing the same Toshiba laptop when Vista and XP runs on it AND you are comparing two testing tools in G"
Is test automation [ using a software or a computer ] faster than humans to those who say that?
Is test automation cheaper than those human brains who claim to create it?
Test Automation is more reliable than manual testing
I first heard this eye opener, insightful idea from Michael Bolton , "People write code to find bugs on another code. They write code that is buggy which claims to find bugs on another code" and "Writing test scripts is another development project . Most of them don't realize that they are running two development projects in parallel and hence their main development project suffers."
So is test automation reliable than manual testing to those who write code that test another code?
We humans are fallible and all things we create are likely to be fallible, too. Test automation scripts are not excused by God to feel "All test scripts don't have bugs".
That doesn't mean human testing is more reliable. It means, anything done by humans is fallible. When you depend on a fallible thing that a fallible human created, you might not want to talk about reliability there.
Test automation removes the monotonous work of a manual tester
Why is testing monotonous? The unspoken truth!
Test automation is better than manual testing as a career option
Well, people think by becoming an automation tester they aren't doing manual testing. Here is what I have discovered - There are more people who claim themselves as automation testers in India than those who claim to be manual testers. So, let me know which is not manual testing? Can manual be termed as anything that does involve humans?
Lack of skilled testers at least has helped me grow phenomenal in my career. I am not THE skilled but I claim to be one of them and there might be very few like me and not many in India.
Test automation can find a lot more bugs than a manual tester & Test automation provides confidence that the product works
Yes, test automation is very intelligent. If a script is programmed to catch a bug when a value exceeds 5, it can also notice that the value did not exceed 5 and pass the test but the screen went blue for a while. If after all scripts have run and there aren't any bugs found, the test automation suite will alter its values and run once again to catch bugs. As the scripts run over and over again, the scripts learn the product well and if the management needs to take a decision to ship the product they would come near the script and say,
"Oh magical script, I want to ship this product. If you think it's a bad idea to ship now, speak out else remain silent"and then they listen to the script and decide whether to ship or not.
Yes, we are dumb. We can see a numerical value not exceeding 5 and pass the test case but also we can't spot the screen becoming blue for a while and do not report that as an issue while running a test that we claim to pass.
Test automation is about converting manual test cases to automation
Oh wow! So, I can't write automation scripts without writing cases and executing it at least once by my own? And if I write, many might cry "Foul".
Well, test automation scripts are just another program as the program it tests. Did developers have something they did manually before they wrote the code that you test with your manually converted automation test scripts?
Test automation tools provides Return On Investment.
Ok, so here is the industry standard of returns on a tool: For every $500 invested on a tool, a good ROI is about the tool helping the team find 50 bugs.
So, a good ROI for $9000 tool = ( 9000 / 500 ) * 50 = 900 bugs
If that doesn't make sense to you then what is the Return on Investment for a tool?
Bugs? Customer appreciation? The number of testers who get fired? The cost savings of hiring 10 skilled testers v/s the cost of buying the tool? The value that the tool brings to the testing effort? Tons of documentation produced? The added cost of training? The added cost of support? The added confusion that prevails? The number of fake experienced testers who apply to the job of a toolsmith?
Test automation is better than manual testing as a career option
Test automation, at least in India, is more chaotic than the so called manual testing. Hardly any tester knows why they are automating things. You may join any forum where testers are active discussing tools and automation, you'd see questions that the help file of the tool has answers and you would discover replies to such questions that is sometimes horribly wrong and yet the person who asked the question says, "Thanks" and the chaos continues.
There is more chaos among so called automation testers than so called manual testers because the so called automation testers form a huge community than the so called manual testers. More people with lack of insightful ideas means more chaos.
Want to know some useful information about automation or want ideas to think more about it?
Cem Kaner prefers to call most of the automation activities that most testers do as Computer Assisted Testing and I couldn't find evidence to refute the idea. I think it is a great idea to call it "Computer Assisted Testing" and you would know why it is a great idea if you go through this article by Cem or the PDF version of it.
So, all of us irrespective of calling us manual or automation testers do Computer Assisted Testing.
I know of an article written by James Bach that made many people who claims a lot of things about test automation feel guilty and I am sure if you have reached this place reading this post of mine, you'd be interested at reading James Bach's Test Automation Snake Oil - the paper or the slides .
I also know of a blog post from James - Manual tests cannot be automated and Shrini's post on A Mystery called automated testing.
Here is why we do different things in testing such as writing test code, executing test ideas, investigating, diversify approaches, think of different techniques, use heuristics, use oracles, Explore... - because each of them add different kind of value and each of them helps in finding different kinds of information or helps us ask new questions AND NOT because something is better than the other.
There is lot of thinking, reading and writing, I and you require, to know about test automation or about testing. The lazy ones don't get to learn and continue to spoil our craft because they don't stop writing, just like me. [ It's intentional ]
I know I haven't answered a question that you might have in mind but that's intentional again.
Update: Here is Jonathan Kohl's awesome interview about the same topic. Thanks to Konstantin for pointing it out through a comment.
--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com
"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --
Subscribe to:
Posts (Atom)