A good tester clears traps. Who are you?

There is no standard definition of a "good tester" to my knowledge but I am sure there are ways to say if I am doing a good testing. I self certify as a "good tester" and I *think* I am respected because my standards are higher and growing!

Years back, I fell into a lot of testing traps and I fall into fewer testing traps today. The traps that I fall in decides how bad my testing is. It certainly takes time to recover from the traps and hence I lose time which otherwise could have spent adding value to the project.

Customers aren't happy to pay for the time we spend to recover from the traps we fell into. There are times where we ourselves set traps (and jump into it) and there are also times where others or situations sets a trap. Whatever is the source of the trap, we fall into it and the biggest and dangerous of all traps is this: To not realize that we have fallen into a trap or to be reluctant of having fallen in a trap.

Here is an exercise that I worked on, which I think, helps in demonstrating the art of clearing traps in testing. I would call it : 100 questions in 20 minutes - The Art of clearing traps in testing [ PDF file : 104 kilobytes ]

Wishing you a good learning experience!

Do you think I had a specification or requirement document when I took up the exercise?

-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

How does Testing look from the eyes of Tester Tested?

Michael Bolton, gave me a writing exercise a month back ( to rewrite an article that I wrote without changing the meaning but changing the examples ) when I showed him a piece of my work that was published by Mahantesh Ashok Pattan, a fundoo test professional from Microsoft. Mahantesh currently heads a testing excellence team in Microsoft's Hyderabad campus. Mahantesh honored me when he asked if I could write an article for his upcoming software testing website. So here goes my exercise result ...

How does Testing look from the eyes of Tester Tested? Version 2.0

Testing - from my eyes looks different, each day. Everyday, I ensure that I learn something new, observe something more carefully, listen to something more carefully, think deep on a topic that I choose to think, ask questions, read what others write about testing, teach testing to someone, learn a skill from someone, test, self critique my own testing, use Wikipedia/Google, practice the skills that I have developed yesterday, day before yesterday and what you might call as “past” and think about the future.

That doesn’t mean I am a workaholic! How can that be?

3 years back, I went to meet a friend in Jayanagar 4th block, Bangalore. While we we were having a walk the talk session, I noticed 2 child rag pickers running towards a food packet that was lying in a dust bin. A dog was running from the opposite direction, trying to grab the food packet before the 2 rag pickers could get it. The dog lost battle and the 2 rag pickers put their hands into the food packet. I was shocked witnessing this scene (without knowing that a bigger shock was waiting for me)

Today I can say, "The two rag pickers taught me a wonderful lesson that I can apply to testing, to better my testing"

Here is what happened: While the two rag pickers put their hands on the food packet, I shouted, "Drop that, I shall get you good food in a hotel" and their body language suggested to me that they would come with me to the hotel.

I started walking towards a hotel nearby to get them good hot food. Minutes before we reached the hotel, I turned back to see if the 2 guys are following and I noticed that one guy still held the food that he could hold picked from the food packet while the dog had a thankful look at me as the guys had dropped the food packet.

I turned to him and said, "Hey, I told I would get you good food but why aren't you throwing that junk food?"

He replied, "Well, what if the hotel is closed? what if you change your mind? what if you played a prank on us? I still have something to eat as compared to the other guy who doesn't have anything."

Poverty might have made him say that but I was shocked by his thinking capabilities. I am sure he has not had any formal education and everything that he learned is through his experiences in life and is also having the same challenge that we face, "survival of the fittest".

When this incident happened, I must admit that I didn't think of a testing lesson that I could learn from it but an year later when I sat relaxed in my chair after a good heavy meal and I thought about the rag picker incident, something that is related to testing struck me.

We testers lose control of things many times that might have been of great help to do a better testing. For instance, I have worked with testers who waited for the requirement documents to arrive although the product ( in some shape ) was in hand.

The boy who had a little food in his hand still had control of the situation and thought of possible situations in which he might have lost everything ( the food packet and the good hot food that I promised ) had he not done that act. Taking control of the situations ( no matter how hard it is ) is a skill that testers need to conciously practice.

An oracle ( not the database ) is a principle or mechanism in which we identify problems. If you learn and start using oracles consciously, I guess you might be in a position to take control of a situation which demands you to test without specifications or with less or poorly documented specifications.

Case 1: I hand over you my mobile phone and ask you to send an SMS and you see the application crashing for that operation; would you call it a bug/issue? (Although you don’t know what the requirement document for the mobile phone that I possess say)

Case 2: I hand over my mobile phone to you and ask you to open each application and exit it. In all applications you use the Right Soft Key to exit whereas one application demands you to exit using the Left Soft Key. Would you have something to say or ask?

In Case 1 and 2, we are using “oracles” of consistency and heuristics (fallible method of solving a problem) to find or discuss about an issue. Doesn’t that mean having diversified set of oracles, heuristics, and test design techniques can fetch great value for the testing you do?

We have a control over the software, platforms, our thoughts, ideas, questions we can ask, skills that we can put to practice or use when such a demanding situation arises, knowledge that we can bring in, use different heuristics and oracles and try reaching closer to the solution than to stay far away by saying, “Where is the spec, I can’t test without it”. You might want to make a note or mention a risk that a specific document that wasn’t available to you could have helped you go more close to what others might call as “great testing”.

Thats how I learned an important lesson from a rag picker!

Now, you could observe that I was actually supposed to have fun while being with my friend and not think about testing. I did have fun and at the same time I was conscious to see if I have missed an important learning that helped me better my testing. I usually don’t miss the fun nor the learning and hence I hardly get tired.

Here is another story, which I shall connect to the above one, in an interesting way.

A tester reporting to me introduced me to her husband by saying, “This is Pradeep, a passionate tester.” I love to be introduced that way since I am aware that I am one such.

Her husband immediately shot a question: “My friends who are into testing say that bringing testing to their life has caused some serious damages to their relationships, how has it been with you?”

Read my reply to that question, as careful as possible: “Thanks for asking me this question. I am probably one of the most passionate testers and I am sure that those who really know, what testing is, wouldn’t mess up his/her life with it. It appears to me that your friends who claim to know testing, don’t know that because had they known, they wouldn’t have got it into their personal lives”

I have enjoyed some great relationships with people. If you claim to be a good tester, it means testing has taught you not to get testing into relationships although testing happens between relationships and lives in a sub conscious level.

“Testing is questioning a product in order to evaluate it” – James Bach

Now the time for connecting two stories: The first story says, “There is always a thread about testing that I run in my mind while or after having fun or watching TV or chatting with friends” and the second story says, “If you really know what testing means, you wouldn’t bring it into your personal life”.

Connecting both of them I want to mean, I try to look for opportunities to learn from things that are happening around that I can apply for my testing activity for which I am paid and *not* to use testing as an approach to build/spoil relationships or systems that others are wanting to use after my usage. For instance, after withdrawing cash from an ATM, I do not intend to perform some quick tests, because it might stop the system and someone in emergency might have to go looking for another ATM.

You have been seeing testing from my eyes, time for you to see it from your own eyes, and I hope you see it in a better way than mine, teach me something and help me to see something different from your eyes. E-mail me at pradeep.srajan@gmail.com

---

If you have liked this article, you might want to read a different version of the same and here is the link. If you haven't liked this article, maybe you want to read it again :)

Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

The story of a monk who sold bugs!

In January 2006, I wrote an interesting post in my blog that became one of the favorite e-mail forward among testers in India. To my disappointment, when I got the forward e-mail from a couple of people, I could see different names as authors in it and I never got the credit. I bet no one can make the kind of creative grammatical mistakes that I do and its always easy to spot my writing. Also as Michael Bolton pointed out in software-testing list, "Pradeep writes from heart".

Fearing those idiots who did not owe me credits I didn't want to write such stories that could make another interesting forward with a claim that they are the authors. I am no coward and most important of all, I don't want my blog readers miss such stories that I have in mind. So here is a story that I think that has gone weeks of thinking:

The story of a monk who sold bugs written by Pradeep Soundararajan [Idiots and fools can change it to their name if wanting to forward the story]

Once upon a time, not so long ago, a monk set out looking for a job in India that could help him earn enough to feed his family. Monk knocked the door of a few software companies and by God's grace, a company did consider him for an opening in testing that they had.

The monk was new to the software industry and had calculated his monthly salary by dividing the CTC ( or cost to company per annum ) by 12 and felt happy about it but when he received his first pay check, he did come to know that maths in software industry, especially when it comes to salary is different. It was not possible to run his family with the monthly pay check he received and hence had to find a way to earn more.

Looking out for another job means, taking time off the work, which also implies that he earns lesser for that month and might end up not getting an offer. So what is the way to work in the same company and yet earn more?

In conversation with other testers in the company during lunch, the monk heard a couple of testers lament about the development teams not fixing the bugs found by them. One tester said, "I bet a 100 rupees on each of these bugs" and other testers too started making similar statements. The monk who was listening to this got an idea and asked all these testers, "did you say 100 rupees to convince developers to fix a bug?" and then testers replied, "Oh yes!"

The monk went back to the monastry in search of his Guru to seek help to win the bet that was placed on each bug. The Guru had left for a world tour but had left a note to all the monks who came in search of him and the note said, "Your problems will be solved even if I am not here to help you and the one who can help you this time is yourself". After reading this the monk didn't get disappointed but thanked his Guru for leaving such a wonderful note.

The monk boarded a bus to head back to office and at the bus he handed over a 10 rupee note to purchase a ticket. The conductor said, "Sorry, I cant give you the ticket, the low battery condition in this ticket generator will make the ticket get stuck during the print out and its a very painful job to remove the paper out and fix the roll in the position and hence we are losing money and business because of this" and the monk said, "Thank you! I got the answer for a question that was bothering me".

The monk, on reaching office called for a meet with other testers and asked them to bring their bug report print outs. Everyone gave their bunch of reports and the monk started his homework.

He picked a bug that was logged with a low battery operation of an embedded product where the developers had commented "Users are not supposed to use the product in low battery". The monk went to the developer and narrated the incident that happened in bus and also added that, "I guess the government is planning to sue the company that provided that machine. As ours is a huge company where different products are developed, are you sure its not our company thats going to be sued?"

That question to a developer was enough for the monk to make his first 100 rupees. As days passed, the monk narrated a lot of real time stories to developers who had offered reluctance to fixing a lot of bugs assigned to them and the monk made a lot of money. As the stories from the monk became popular among the developers in the company, developers loved listening to monk' stories and the monk had a very high credibility among all teams.

The testers conducted a meet and discussed about the monk who made a lot of money through them and decided to cancel their bets thereafter and some testers said, "Ah! I can say better stories, what big deal? Let me convince the developers hereafter and I am not going to pay that already rich monk anymore"

The monk had not only caused a revolution within himself, he had also created an evolution, the way other testers wanted to work.

The monk decided to go to a new place looking for more and different kind of problems. He made a lot of money solving such problems and eventually became one of the richest tester of the world!

Lessons to learn from the story of a monk who sold bugs:

• Everyone needs something convincing to take up a task, be it fixing a bug or executing scripts.
• Developers need some real time scenarios, facts, customer information and information about customers knowledge, models in which the customer uses a product in a packaged form - story - to get convinced to fix bugs.
• One who constantly thinks of solving a problem, will find the solution anywhere.
• The most reliable person for help, is yourself.
• Testers need to develop and practice a skill of story telling to get better at selling bugs.
• Testing is not an activity of improving quality, its an activity of finding information and it is the product/development teams who fix those issues for improving the quality. In case you still want to think that testing is an activity of improving the quality, you might want to become a best bug seller to continue thinking of it.
• Testing is not necessarily a computer science subject and its life science, social science, ... all science and all non science, too.
• There is a traditional way of making money and as a tester if you take the traditional way, you make less with it.
• Some bugs are born with self explanatory story but some aren't and testers need to build them.
• More your sales figure ( selling bugs, I mean) higher is your credibility and as testers if you don't have credibility, you might not be able to boost your sales figures further.
• Pradeep Soundararajan will keep writing such stories without fearing the people who try putting their name as authors and exposing themselves as idiots, fool.

-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

The negatives of "negative testing"

Before you proceed reading this post, please take a piece of paper or a notepad and write down what you mean by "negative testing".

Are you done with it?

I spent time investigating what many Indian testers say about "negative testing" by browsing online forums, google search and I also sought help from testers who were online in my buddy list. What you read below till you see a "Meep" are views of testers who responded to me over IM when I asked them - What do you mean by "negative testing"? or When someone says "negative testing", what occurs to you? or What do you think about "negative testing"?

"I infer that some of the people are trying hit on negative tests (the probality of user acting the same is very less) o begin with"

"Testing carried out to test expected behaviour from product by providing wrong (negative) inputs "

"A negative test would be the program not delivering an error when it should or delivering an error when it should not. Negative Testing = (Showing error when not supposed to) + (Not showing error when supposed to)"

"there is nothing called as negative testing.. but there is something called negative test case or test idea.."

"Negative testing is that testing which attempts to show that the application does not do anything that it is not supposed to do"

" input which do not lie in valid domain"

"any testing carried out by passing non reccomded values with an aim of breaking down the application is called negative testing"

"if a developer designed a edit box to accept only numerics and length of 10 nos and if it accepts alphabets which we enter if it accepts either than numerics is negative testing"

"negative testing is checking for scenarios which are not smooth path.."

"Testing aimed at showing software does not work. Also known as "test to fail".in simpl ewords"

"negative testing is something you know tht the values wont support but u need to test it.."

"negative testing for me will be done after validating the postive test cases first"

Look this is an interesting one - "as far i am concerned..i belive there is noting called -ve testing."

Meep!

I am happy that Debasis and Venkat Reddy asked about contexts before they came out with their answers.

I found a lot more definitions by individuals in online discussion forums and orkut communities about what people had to say about negative testing and it's a similar experience to what you have read above. If you are interested Google it up and have fun!

Thinking critical, I conjecture that testers to my knowledge are using the term "negative testing" as a guide word heuristic. They are happy of using it because they found bugs with it, be it whatever they did assuming that they were doing "negative testing".

So doing something that could find bugs is more important than what meaning one has for the technique he is using to test ( in the context of this discussion) BUT testing in a skilled way is more important than just doing it. The problem comes when you have to describe what you did to others, especially when they question the techniques you used. A "stress" test to you might be a "load" test to someone - How would you communicate yourself effectively being in a land of confused terms?

After I wrote the above sentence, I leaned back on my chair for a while and thought the following -

"Negative testing" might have different meanings in different different contexts and the following list are my assumptions of the scenarios where I might have done or might do negative testing -

1. If I find a bug but did not report it, which later turned out to be a critical issue, I did negative testing.
2. If I fail to observe a bug that passed away the screen, which could have been caught if I had practiced my observatory skills, I did negative testing.
3. If I do not learn the product or technology during test execution, I did negative testing.
4. If I do not run cost v/s value in mind and end up doing an expensive test away from the mission, I did negative testing.
5. If I failed to ask questions before accepting the mission or jumping to test, I did negative testing.
6. If I blindly follow the specification document, believing it's a bible, I did negative testing.
7. If I am into automation testing just because my friends said "Manual testing has no future" without realizing that automation testing also involves manual activity, I did negative testing.
8. If I manipulate the metrics, without realizing the impact of it on reputation of the company and myself, I did negative testing.
9. If I freeze my test plan and never bother to change it despite the daily dynamics, I did negative testing.
10. If I mess up my relationships with developers, without realizing the fact that developers are great resource of information for testing, I did negative testing.
11. If I stopped learning or practicing testing skills, I did negative testing.
12. If I spend most of the time in orkut and online groups asking questions about how I could solve my day to day problems with subject "Urgent - Need help", I did negative testing.
13. If I list everything that I have in mind about this topic giving less room for you as a tester to think about this, I did negative testing.

Some testers expressed shock when I said to them that I don't know what negative testing actually means. It's very important to say "I don't know", in my opinion when we really don't know something being asked. How long has it been since you said, "I don't know"?

I recently faced a question from someone who claimed to be my friend but isn't happy of my growth in testing field, "Are you a humble person, Pradeep?".. I said, "I don't know" and he replied, "Well, this answer itself says, you are not". I had to laugh at three things - the situation, myself and then my dear friend.

Not knowing a good enough definition that suits your context while applying it, is a negative thing to happen to the project you are working or for people who are tasking you. When you do negative testing with already a negative happening, you might be finding bugs because the resultant is always positive, mathematically. Ah! That's my imagination and I could be wrong.

I am open to more ideas on this but if you put the comments in a negative way, I might have to make the resultant positive ;-)

Testing in my opinion, should not be classified as positive and negative, it isn't electricity but still someone has done the damage by misunderstanding someone else. I don't know what I did with this post!

-- Pradeep Soundararajan - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

Every tester should become a good developer

Hi Reader,

I do understand that you are surprized and you would want to ask me:

"Pradeep, have you gone crazy? It is you who said in your previous posts that testers should develop passion towards testing and it is not good always for all testers to think of jumping into development".

I still remain the same, just that I did want you to read this entire post to make you understand why a tester should become a good developer.

_ Every tester should become a good developer _

Do you belong to the category of thinking that development means coding?

Testers do development, how many of you are aware of it?
Yes, as testers, you and me have done development.

"Pradeep, do you mean test automation?"
No but yes, inclusive of that.

Well, let me break the ice, it is late, I am aware.

Testers should become good developer of test ideas !

Ah, now I am sure you would have got a better reason for going through this post.

One of the testers who is in touch with me posed me a question on chat, I am sure you would be interested to go through the whole discussion:

Anonymous: Hi Pradeep
Anonymous: How u doing?
prad_intel S: Hi
prad_intel S: I am doing great
prad_intel S: wats up
Anonymous: Nice to hear this
Anonymous: are you busy now?
Anonymous: I have a doubt
prad_intel S: tell me
Anonymous: write 10 test cases for the triangle which takes input values as length of side of the triangle from a data file & displays the message whether triangle is scalene, isosceles or equilateral triangle.
Anonymous: Data file i didnt get it..
prad_intel S: what for is this?
Anonymous: This is a Question which was asked in Adobe Interview
Anonymous: I just saw this Question in Geek Interviews
prad_intel S: well could you not think of it ? ( I would want everyone of you to think before you pose a question to me or someone else whom you would want to ask a question,sometimes, you yourself could have an answer and may end up making others feel great for answering your questions)
Anonymous: Yes
Anonymous: But i just want suggestion from ur side
prad_intel S: ok
prad_intel S: let me start off with it
Anonymous: Ok
prad_intel S: my understanding : there is a program which takes the values from a file for the 3 sides of the triangles and throws out a message of what type of triangle is it
Anonymous: ok
Anonymous: Yes right
Anonymous: So input will be taken from the file itself
prad_intel S: my understanding 2 - all three sides value will be taken by the file and the value is taken either sequential or random
Anonymous: ok
prad_intel S: the above was an assumption and not understanding
Anonymous: yes
prad_intel S: test cases could be ...
prad_intel S: 1) Input values 5, 5.0 and 5.00 to check for equilateral triangle
Anonymous: ok
prad_intel S: 2) (5, 5, 5),(5,3,2) and ( 5,5,4) to check whether the message shown maps to the type of triangle
Anonymous: ok
prad_intel S: 3) 0 0 and 0 - to check for error conditions
Anonymous: ok
prad_intel S: 4) 10000 , 1 and 1 - which cannot make a triangle ( am i right)
Anonymous: ok
Anonymous: yes
Anonymous: Perfect Test Cases
Anonymous: Good Suggestion
prad_intel S: 5) a mix of positive and negative values/fractions
Anonymous: Yes
Anonymous: Thanks for giving suggestion Pradeep
prad_intel S: 6) values as co ordinates like - 2,3 and 5,4 and 6,8
Anonymous: ok
prad_intel S: 7) values in words like - nine , six and four
Anonymous: ok
prad_intel S: 8) the file is a program that depends on some other file for its output value
Anonymous: ok
prad_intel S: 9) numbers in ASCII/UNICODE ...
Anonymous: oh nice
Anonymous: this i was not aware off
prad_intel S: 10) Last but not least, the program trying to open locked files
Anonymous: yes
prad_intel S: Extra cases
Anonymous: perfect
prad_intel S: 11) Reading from different file formats
prad_intel S: 12) Reading values from charts( example - excel charts)
Anonymous: ok
prad_intel S: I can probably think of 100 cases on this,all useful ones, of course.
Anonymous: Hahaha !

If I tell you, what are useless test cases, will you be able to write smart test cases?

1. A test case is useless if it does not have the probability of catching a bug.
2. A test case is useless if there could have been a better case of the same type in its place.
3. A test case is useless if it is not written properly and the person executing it is unable to understand it. ( chuck out those reasons where the person executing it is not interested to learn and improve in life...)
4. A test case is useless if it does not get reviewed.
5. A test case is useless if it is not the outcome of use case document.
6. A test case is useless if there exists similar one but with different words.
7. A test case is useless if the person writing it was not in a mood to write it.
8. A test case is useless if the person reviewing it did not go through it or give a thought about it. ( of course, it is better to have a reviewer who is better or competent to us)
9. A test case is useless if you write it just for the sake of writing one.
10. A test case document is useless if more bugs are found by the customer.
11. A test case document is useless if it is so large in number that makes the tester feel "Oh my God".
12. A test case document is useless if it has not been updated after every release or bugs reported outside test team.

<<>>

_ end of _ Every tester should become a good developer _

"The birth of a test case should be the death of a bug"

Regards,

Pradeep Soundararajan

pradeep.srajan@gmail.com

Disclaimer: There are lot many factors that can brand a test case as useless but with little knowledge, I have tried putting up something that can excite your grey area. People who are in touch with me through mails or phone or face to face know that, I never force someone to take my points but would want to think and validate my points, if they want to go ahead with it. Let me know, if by any chance helped you in improving the way you write test cases. You can call your smart cases Tester Tested ! :P

UPDATE: There was an original name instead of "Anonymous" till Sep7 2006 and the person sent me an offline shouting at me as its insulting him/her, since he/she searched his/her name in google and this post topped the search result. I am puzzled to as what made someone think a learning experience as an insulting one and hence have changed his/her name to "Anonymous". Long live such attitudes.