The negatives of "negative testing"

Before you proceed reading this post, please take a piece of paper or a notepad and write down what you mean by "negative testing".

Are you done with it?

I spent time investigating what many Indian testers say about "negative testing" by browsing online forums, google search and I also sought help from testers who were online in my buddy list. What you read below till you see a "Meep" are views of testers who responded to me over IM when I asked them - What do you mean by "negative testing"? or When someone says "negative testing", what occurs to you? or What do you think about "negative testing"?

"I infer that some of the people are trying hit on negative tests (the probality of user acting the same is very less) o begin with"

"Testing carried out to test expected behaviour from product by providing wrong (negative) inputs "

"A negative test would be the program not delivering an error when it should or delivering an error when it should not. Negative Testing = (Showing error when not supposed to) + (Not showing error when supposed to)"

"there is nothing called as negative testing.. but there is something called negative test case or test idea.."

"Negative testing is that testing which attempts to show that the application does not do anything that it is not supposed to do"

" input which do not lie in valid domain"

"any testing carried out by passing non reccomded values with an aim of breaking down the application is called negative testing"

"if a developer designed a edit box to accept only numerics and length of 10 nos and if it accepts alphabets which we enter if it accepts either than numerics is negative testing"

"negative testing is checking for scenarios which are not smooth path.."

"Testing aimed at showing software does not work. Also known as "test to fail".in simpl ewords"

"negative testing is something you know tht the values wont support but u need to test it.."

"negative testing for me will be done after validating the postive test cases first"

Look this is an interesting one - "as far i am concerned..i belive there is noting called -ve testing."

Meep!

I am happy that Debasis and Venkat Reddy asked about contexts before they came out with their answers.

I found a lot more definitions by individuals in online discussion forums and orkut communities about what people had to say about negative testing and it's a similar experience to what you have read above. If you are interested Google it up and have fun!

Thinking critical, I conjecture that testers to my knowledge are using the term "negative testing" as a guide word heuristic. They are happy of using it because they found bugs with it, be it whatever they did assuming that they were doing "negative testing".

So doing something that could find bugs is more important than what meaning one has for the technique he is using to test ( in the context of this discussion) BUT testing in a skilled way is more important than just doing it. The problem comes when you have to describe what you did to others, especially when they question the techniques you used. A "stress" test to you might be a "load" test to someone - How would you communicate yourself effectively being in a land of confused terms?

After I wrote the above sentence, I leaned back on my chair for a while and thought the following -

"Negative testing" might have different meanings in different different contexts and the following list are my assumptions of the scenarios where I might have done or might do negative testing -

1. If I find a bug but did not report it, which later turned out to be a critical issue, I did negative testing.
2. If I fail to observe a bug that passed away the screen, which could have been caught if I had practiced my observatory skills, I did negative testing.
3. If I do not learn the product or technology during test execution, I did negative testing.
4. If I do not run cost v/s value in mind and end up doing an expensive test away from the mission, I did negative testing.
5. If I failed to ask questions before accepting the mission or jumping to test, I did negative testing.
6. If I blindly follow the specification document, believing it's a bible, I did negative testing.
7. If I am into automation testing just because my friends said "Manual testing has no future" without realizing that automation testing also involves manual activity, I did negative testing.
8. If I manipulate the metrics, without realizing the impact of it on reputation of the company and myself, I did negative testing.
9. If I freeze my test plan and never bother to change it despite the daily dynamics, I did negative testing.
10. If I mess up my relationships with developers, without realizing the fact that developers are great resource of information for testing, I did negative testing.
11. If I stopped learning or practicing testing skills, I did negative testing.
12. If I spend most of the time in orkut and online groups asking questions about how I could solve my day to day problems with subject "Urgent - Need help", I did negative testing.
13. If I list everything that I have in mind about this topic giving less room for you as a tester to think about this, I did negative testing.

Some testers expressed shock when I said to them that I don't know what negative testing actually means. It's very important to say "I don't know", in my opinion when we really don't know something being asked. How long has it been since you said, "I don't know"?

I recently faced a question from someone who claimed to be my friend but isn't happy of my growth in testing field, "Are you a humble person, Pradeep?".. I said, "I don't know" and he replied, "Well, this answer itself says, you are not". I had to laugh at three things - the situation, myself and then my dear friend.

Not knowing a good enough definition that suits your context while applying it, is a negative thing to happen to the project you are working or for people who are tasking you. When you do negative testing with already a negative happening, you might be finding bugs because the resultant is always positive, mathematically. Ah! That's my imagination and I could be wrong.

I am open to more ideas on this but if you put the comments in a negative way, I might have to make the resultant positive ;-)

Testing in my opinion, should not be classified as positive and negative, it isn't electricity but still someone has done the damage by misunderstanding someone else. I don't know what I did with this post!

-- Pradeep Soundararajan - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

FAQ's to my rescue

"Hi pradeep I want to add 2 years of fake experience in software testing field to get job. I am fearing about getting caught. What would you reccomend"

I have a podcast on fake experience and Indian testers. You might want to listen to the podcast to know the truth about faking experience in software testing or any other field for that matter and here is the link "Fake experience and Indian testers".

Everyone who adds a fake experience fears about being caught or feel guilty throughout their career and some unlucky ones are jailed. If you plan to add a fake experience, all you need to do is to decide whether you want to travel through your life with guilt and fear of being jailed or paddle smoothly as others who work hard do. Those who add a fake experience are those who indirectly admit that they dont have the brains that work and I pity their parents who think their son/daughter has brains that work. I ask one question to all those who plan to add a fake experience - Would you be happy if you come to know your father also did the same? (unless your father is a politician) You are the role model for your children and if this is what you do, don't expect your children to be any better than you!

If you still haven't listened to the audio link, you are missing something important.

"Pradeep, I am from India and want to know if there is a future in software testing. I am confused as some of my friends are saying software testing has no future"

I usually say, "There is a great future for software testing but not sure if there is a future to people who ask questions about its future"

Here is when software testing has no future:

• When all of us decide to live with bugs that causes us not to perform some important tasks.
• When all of us are ready to pay a huge amount to buy a product that has some issues.
• When all of us are not bothered if the ATM delivers a 100 rupee note as compared to 1000 being deducted for the withdrawal.
• When all of us decide to board a plane whose components have not undergone any testing.
• When all of us are happy to see a online banking system take away all our heard earned savings because we hit the "delete" button by mistake.
• When all of us as customers love bugs than the products in which it exists.
• When all of us decide to pay for an anti virus software that doesn't catch any viruses but multiplies them.
• When all of us are happy and ready to settle a bill that a software error generated and making our monthly rental for a mobile phone connection as 4594858 rupees.
• When all of us decide that we don't buy or use software products in India.
• When all of us decide to walk 100 miles a day and stand in a queue that has thousands of people waiting to book a ticket in a kiosk whose system crashed and is not recovering.
• Maybe a zillion more things.

Its time for you to *think* and decide whether you have a future. Your friends do belong to the "all of us" category and you must be asking such questions to them. If you are passionate, you can create new business opportunities in this field even if it doesn't exist in your time. All it needs it critical thinking. If you aren't a good thinker then certainly there is no future for you in any of the field you work in. Time to look for a real good friendship!

"Pradeep i really have a doubt.....i had been in manual testing for the past 1.5 years...still not gone into Automated testing...I think u know the domain in which m in. People r really scaring me that u will never have future in manual testing...I really got pissed out ....If i like to move into automated testing....whats the stepping stone for it."

Hmm! What do you mean by "still not gone into automated testing"? There is no thumb rule that a tester is mandated to follow to shape his career. It's never like this:
First year of career - Manual testing
Next 3 years - Automation Tester - QTP
Next 3 years - Lead
Next 4 years - Manager
Next 2 years - Senior Manager
Next 2 years - Assistant Vice President

I infer that many people to my knowledge in North America do what they enjoy doing and many people to my knowledge in India do what others feel they would enjoy. I enjoy blogging and hence many testers do read, appreciate, learn and come back. I am not sure people would have enjoyed reading my blog had I wrote something with an assumption that if I didn't write about QTP, some testers might not come back or enjoy reading it, I would have been exposed as a fool.

If you enjoy what you do, you are set to be happy despite the failure you might face. If you do what you don't enjoy doing, you might be unhappy despite the success that you think has come to you.

"...recently i did software testing course from Infics Solutions.now i am trying to get job in the same field.i am attaching my resume with this mail. u plz go thru it...n plz suggest me, whether i can get job in testing field with the the technical skills n the background i have ????? coz in few companies...only B.E. candidates r preferred."

First, if you want to write to someone and expect them to respond to your e-mail, I recommend you to not use a chat style of writing. They might not take you serious and maybe you are repeating this because none of them pointed it out to you.

By going through your resume, I might not be able to say whether you can get a job in software testing but I might be able to say if I have discussions with you on testing if I am looking to hire someone. Your resume or profile is a set of claims that you make about your technical skills and knowledge. By going through the claims your technical skills or the knowledge you have in testing, cannot be quantified, in my opinion.

Also, you have undergone training from a training center that claims to teach testing and yet it appears to me that you aren't confident or skeptic about your chances of getting job in this field. Perhaps, you must help your juniors realize that such a kind of a training hardly helps you develop confidence or build your passion nor it helps you to get a job in software testing.

A good test team needs to be diversified with skills, background of a tester, knowledge... If a company is insisting on having only BE degree holders as candidates then probably they might re-consider their decision when they realize or discover the need to have candidates from different degree and science backgrounds. It is good to have someone who has worked on banking applications with a banking or finance degree in a testing team instead of all members of the team from computer science background.

" ... my boss asked me about automation testing, he wants to automate his projects, just coz he believes that lots of bugs can be found only by automation. can you help me?".

You want to automate things just because your boss feels automation could find a lot of bugs? Well, I don't know about your context, maybe your boss wanted to mean "We could find those bugs that can't be caught by manual computer assisted testing by automating those tests that can be automated".

If I were in your situation, I would point him out to this article written by James Bach to help him get much clear view on what idea I subscribe to about automation, and help him understand what it could do and what it can't.

I came to know from your e-mail that you do a web application testing and you might want to look into tools like WATIR / SAHI (developed by an Indian in Thought Works, Bangalore) and choose the one that suits you. Some commercially available tools like Winrunner or QTP might be less helpful for your project but many to my knowledge in India don't realize it because many people we find around us aren't testers but toolsmith.

"...Before I joined this company there wasn’t any employee specialized for testing, this job was done by developer itself. And even now some times the developers here underestimate testing sometimes...”.

As testers we shouldn't spend time looking who is respecting our profession but instead look for bugs that could build reputation for us within the organization. I happened to work with developers who were from premier universities like IISc and IIT who wondered if I who had studied from a not so premier university could find bugs in their code. Within a month their impression on me changed and they started calling me "crash specialist" and they started valuing testing as an important activity to better their development skills.

We talked about the crashes stories when we met recently and it was great fun on both sides. Probably, instead of waiting for the software testing craft to gain respect, people like you and me can help the craft get better respect by doing good testing.

"Is it necessary that the testers should have the knowledge of coding to do testing? I am really confused sometimes as I really feel I am in wrong profession. Can you advice?"

The necessity is based on the product and context you are working. Diversity is a very important aspect in software testing and I love to give an analogy of Jurassic park movie where we see raptors doing a coordinated attack for a healthy test team. When raptors need to hunt their prey, each of them takes a role - to corner the prey, to distract the prey, to launch a surprise attack...

Now, a healthy test team might comprise of a technology man, domain man, business behind the product - fellow, script wizard, explorer, bug hungry man, lateral thinker, logic analyzer ( man ), general system thinker, plan man, cooler ( man ), Sherlock Holmes and Watson, the inventor, the discoverer.

It's tough to look for all these qualities in one man and also tough to make such people to get together as a team or maybe there is a scarcity. As testers, we need to have skills of at least 3 of them to start off and as we practice testing, we might want to put more men into us.

For instance, you might already be a Bug Hungry Man but you should start growing yourself to a Bug Hungry Man + Sherlock Holmes + General Systems Thinker ... over a few years of experience.

You are also free to discard one or two of them if you feel you are strong with other men inside you. If your team already has a script wizard, you better develop yourself as other personalities. If you are in a team where there is a need of a script wizard despite all other personalities being present in the team, you develop the skill of scripting.

I feel safe most of the times, as I can always find someone who knows scripting but I did scripting in Perl for a couple of months when there was a need and I volunteered to be the script wizard for a while.

"problem that i am facing is that i don't get time to test as i am always busy preparing the metrics, reports, meetings which gives me goosbumps - i am not testing."

If you run Cost v/s Value in your mind and if you are training yourself to be a situationally aware person, you might not face such a problem. Instead of having a 2 hour meeting on what could be done for a customer who is expecting a report at the end of the day, it is better to do a few tests and find bugs that might help you in deciding what you could do by the end of the day.

Also, if you spend too much time calculating or preparing a metric sheet which otherwise could be spent in finding more important problems and have a report that is crisp and yet conveys the important information you might not run into problems or might clear such traps in testing.

Pradeep Soundararajan - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

Exercises that exercise a testers mind - Workshop

Updated: 7th August 2008

Do you drive a car/bike?
Did you learn it by sitting in a class?
Did you become better at it without practicing it?

Now you know what this class is about.

History of the class

This is not an experimental class or a pilot program. James Bach and Michael Bolton, two great testers and philosophers of North America with other expert testers they collaborate, invented and have been inventing different ways to educate testers and help them get better at it. This gave rise to a philosophy, approach, heuristics, oracles, techniques, exercises, mnemonics, methodology, skills … combined to call Rapid Software Testing.

Rapid Software Testing is a brand name of Expert Testing. James Bach and Michael Bolton have visited India often in the past teaching Rapid Software Testing to organizations a couple of top IT organizations in India.

While the certification course make a tester to sit in a class where he never even gets to see even a video of an expert driving a car ( an expert doing testing ), Rapid Software Testing ( not a certification program ) a skill development and nurture program, gets a tester on the driving seat, makes him to drive, explore, identify mistakes, suggests corrections, makes him drive again and keep doing it for a life time. RST has been self realization program for some testers in the world.

Why would your competitor love this program?

• More or less your organizations and theirs hires a bunch of certified or non certified testers who have ( or would be) been certified not because they know to drive ( test ) better but because they were able to crack 50 multiple choice questions out of 983294839483294893489328493849348 billion possible challenging questions in software testing that you want them to solve.

• With increasing complexity of products, increasing need to demonstrate human resource skills and attract customers, your competitors have something to brag about that you might not have.

• They would be happy that their testers know more approaches to testing that might result in cost effective value addition to potential customers. ( Ask your testers if they know the value of each test they are executing )

• What more a competitor wants than to know that they have more competent resources than that of your organization?

There are only two possible choices you can make:

1. Get this training to your staff and put yourself in much competent position.
2. Watch your customer grow competent and wonder how they got there. ( because you wouldn’t know if your competitor took this program )

We think the second choice would be very expensive but we wouldn’t mind that happening if you are OK with that.

About the driver

Pradeep Soundararajan, is an independent test consultant. He coaches, consults, speaks, writes, tests, manages, thinks on software testing and problem solving. He is an international invited speaker. His testing has had an influence from James Bach Bach, Michael Bolton, Jerry Weinberg, Cem Kaner and Ben Simo.

He has tested over 100+ products in the last 6 years. Some of the products he tested ranges: Desktop Applications, Bluetooth communication systems, Wireless and Embedded Multimedia Systems, Pocket PC and Mobile Phone applications, Auction systems, E-learning products, Online Management product, Outsource Relationship Management solutions, Telephony systems, Online applications for Finance and Banking products, Anti Virus and Anti Spam product, Project Management Solutions, Call centre applications, Billing solutions etc…

Some testers he coached have surprised their management with hundreds of bugs that weren’t found earlier.

-- Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton