Weinberg on more than writing - the Fieldstone method

11th March, 2012 in Bangalore was a beautiful Sunday. Now that it is almost over, I can use the past tense "was". Just the weather making it beautiful is one part but what about the weather within me? I decided to make it beautiful within. Everybody has a pile of books that they bought but never paid its due attention. Weinberg on Writing ( WoW as I want to call now) - The Fieldstone Method was one of the books that hadn't got my time yet. So, I read the first chapter at home, closed the book, went to my wife and asked her if she had any plans for the afternoon. As I had taken her out yesterday, she excused herself out of any plans. Whenever a husband asks "What's your plan?" to his wife, it means, "I have a plan and want to be sure if your plan doesn't disturb mine" :). I took my backpack and scooted to a coffee shop nearby. Coffee, book reading and relaxing was my plan.

About 2 PM, I was all set. I wanted to start with Camomile Tea. I had told my wife that I'd be back only when I am finished with the book, so I was hoping there would be a couple of beverages. I realize now that there is nothing like "finishing the book" because The Fieldstone Method provides an experiential learning to those who want to work with it. Not just that.

I have had the great opportunity to meet with Jerry and spend a couple of days, attending his workshop, being a part of a peer conference that had Jerry, a conference and much more in 2008. I have been a reviewer of Jerry Weinberg's book "The Perfect Software and Other Illusions About Testing". I have read a couple of other Jerry's books that have changed me from time to time. With this book, I felt Jerry sitting next to me and coaching me on the Fieldstone Method. This is no hallucination. I am sure the emotions he had while turning the stones that he used to build this book has rubbed on me and has caused some very good emotions within me. Jerry intends that happens with the experiencers of the book. / OK there is no such word as experiencers in any dictionary /

I haven't published a book yet. If you are thinking I haven't written one yet, you are mistaken. I just haven't published it. I see the light now that will lead me to publishing. That's what I seem to be getting out of The Fieldstone Method because I am going to build my books not write it.

So, why am I telling you all this? To get you buy the book and experience it? To write a review of the book?    You will discover why I am writing about it, if you were to continue reading.

It is all about me. I recognized that I had almost stopped note taking. In other words, I had stopped collecting fieldstones. I guess I was just processing whatever my imperfect memory could store. Isn't it amazing that I stopped doing something that I advocate to other testers - note taking. When I realized this while experiencing the book, I picked my backpack and searched for my Indian version Moleskine and a pen. I must have missed observing a trillion stones but thankfully I have now saved myself from my ignorance that would have led to missing trillion power trillion stones in the future.

Here are some of the stones I collected today that I am pulling from my notebook (and typing it for you):

• Thought: When you are alone in a coffee shop, you hear people and their conversation you usually wouldn't bother to hear if you were not alone.
• Retrospective as I read the book: I was probably smart all this while but not happy because I always wanted to be more smart but never wanted to be happy.
• I shook the coffee table accidentally and a glass of water placed on it started to get its vibration and then I wondered: Pradeep, when was the last time you observed water settle down after being disturbed with a vibration. Your 9 month old kid now would watch it with curiosity. Relation to testing: The first time an information looks interesting and everybody pays attention to it. Once they learn how it happens, they lose interest to observe things that they were once curious about and if there was a different behavior? They would continue to assume they know something. Software is volatile, just like the water. 
• I was asking myself a question: Does it matter if something takes a long time to learn? What determines "long"?
• I saw cold coffee on some table and decided to order it. I wrote a note : How did my choice alter after seeing something that caused my brain to demand it.
• Project Gutenberg quoted in Jerry's book - Awesome

  A friend of mine, Nandan Pujar wanted to meet with me for a consultation and I felt, "What a great time for a good friend to come in. Some of the exercises I want to practice requires such an occasion and a trust worthy friend". I took notes as I was consulting for him:

• I was reminded of Warren Buffet interview of why he didn't come and invest in India prior to last year. His response was, "Nobody invited me to do so". 1.2 billion people didn't think of knocking that door that way. I felt sad for myself.
• My friend Nandan said, "Being with mediocre people helps you identify your delta with them and being with intellects helps you fix the delta". I thought it was a cool thing.
• Made note of words he used "Feudal" and "Artificial scarcity"

 I came back home and made other note of stones I saw, heard, observed, felt, experienced...

• My 9 month old daughter had to poo. After cleaning the poo, I wrote in my note: As a kid I must have not known this is called "poo" and why people clean it off. Now that I know, I clean the mess I do. I am wondering if testers who recognize the "poo" of their work will ever clean the mess they do. Some of them are stuck with it and they seem to think as though it is a newly grown part of their system. Some who come new to the system see people stuck with the "poo" of their work and assume that, to be experienced, they also need to be stuck to "poo"

It is fantastic. I love it.  Thank you Jerry. You helped me recognize the "poo" I was carrying all this while and I am not going to be one of those who will not clean it. I will clean it. I will not just clean the "poo" I have been carrying for a while but identify with the help of trillions of stones I can see, to not carry any type of "poo" of my life. I also recognize the idea of heuristics and they are fallible.

Till yesterday, I was thinking, "Now that I have accomplished all this in life, how do I change myself?", Weinberg on Writing (or WOW) - The Fieldstone Method is a change catalyst. 11th March, 2012 was a beautiful Sunday in Bangalore. So can everyday be if I were to collect the stones.

Test effort estimation: Getting it wrong before getting it right

While the world thinks test estimation is a very tough thing to do, I think, everyone gets test estimation right. The point at which they realize their estimates were wrong is the time they get the estimation right. I wish they corrected themselves. Mostly, it remains unfulfilled.

So, the deal is to communicate the number when you know you are getting it right and buy time to get it right. By "right", I mean, more meaningful than you would have when you provided an estimate even without looking at the product.

I don't know what's wrong in going wrong the first time. Most of us learned to ride a bicycle, someone did it in a day and others took a month. How would you have answered a question, how long would you take to learn to ride a bicycle? Some people might need more time to get out of the fear of falling from it and others need time to get the balance right. It varies and hence cannot be determined before even someone got on to a bicycle?

Testers & Blocks Consulting - Puzzle 2

You haven't yet read Testers & Blocks Consulting - Puzzle 1? You know Testerlock, right?


"Are you Testerlock of Testers & Blocks Consulting?" screamed a voice on the road just when Testerlock was buying some vegetables. Before Testerlock could respond, the young man filled with excitement said, "I have watched you test at a webinar and I loved the way you brought in so many heuristics to your testing approach. I even wrote an e-mail to you a couple of months back".

"And you are..."

"Oh, I am Philip. A senior tester at PepLabs and I really wish I could learn more from you. In fact my whole team would love to learn from you"

"I am glad you are interested at my testing as much as I will be interested to watch your team test"

"We have a problem though. Our manager somehow wouldn't get convinced that we need your training?"

"Really? Do you have some time, let me pay for these vegetables and we can sit around for a coffee"

"Oh, sure"


A few minutes later, Testerlock and Philip were at a Cup-O, a coffee bar in Portland, Oregon, discussing about the problem that Philip raised.



After the first sip of a coffee, "So, you think your manager is a barrier to bring me in, to coach your team?"

Checkmate heuristic :: A security testing attack

It has been five times over the last six months that someone considered hiring me before making a release decision or after getting skeptical about scripted tests.

Amidst recessionary times, someone considered outsourcing some testing work to me from United States. An advanced version of the product was slated for a release in the next couple of days. The United States company had outsourced development and testing work to a Structured Fancy Name Process Following Disciples company who had ran through thousands of tests over it and had achieved >98% test case pass, a week before today.

Someone in the United States company thought they'd like some Exploratory Testing and I got the opportunity to lay my hands on it to perform a Rapid Testing on it. The charter for me was to report any security related threats and usability problems.

I found about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems:

• I could reset the password of any account by tweaking the variables that the client was using to interact with the server.

• I could stop auto e-mailers reaching any registered mail account in a similar manner as above.
  

I then tried to reset the password of the dummy account I was using. No e-mail reached me. I then thought, "How about reset of admin account?" and then did the same.

The password was reset and no e-mail reached the admin, as auto e-mailers were stopped. So, I asked the admin of the UnitedStates company to login from his credentials and the response was a pleasent, "What did you do and How did you do that?"

Subsequently, all other users were blocked. Only the admin could release the lock but the admin could not log in to the system. 2 hours of outage till someone got into the database to recover the admin account.

You write a lengthy email, hit the submit button and the application prompts for your Username and Password. You enter them and it says, "Incorrect Username or Password". You attempt to reset your own password but the email does not reach you.

Checkmate!

Change in hiring and interviewing process in India for software testing and software testers

I claim to be one of the most experienced and most affected tester in the context of interview in India and here ( in this link ) is more information about it. After going through that link or this video you would know that I have been struggling to not see another Pradeep Soundararajan in the job market who is frustrated with this industry's idea of interviewing. Edista Testing Institute ( my client and partner ) is constantly pushing towards seeing a better testing community and that's why I chose to work with them.

I consider the following as one of my biggest contribution to the change the industry needs. Most training institutes in India (even the so called highly reputed ones) have people who dont know to test, teaching testing by running a thousand slides. I think they have so many slides that if you run 25 slides a second to watch it like a movie, it still runs for about 2 hours. Such slides have always caused an avalanche slide of many victims career, knowledge and skills.

Edista started to redefine things by hiring me and then allowing me to hire Manoj and then Sharath and then now more people. The last I heard from Manoj is that testers who were interacting with him are now excited about what Manoj is doing and are enquiring what it takes to be able to get skilled in testing.

You might also discover that Manoj has started to publish his practice sessions on testing in his blog.

We sit to gether, test, learn from each other, create exercises, practice pair testing, discover new tools, debate on ideas, think about more heuristics, brainstorm test ideas, discuss about bugs, run a test club ( like the movie Fight Club ), teach people how to learn and then how to test. We are never away from testing and we are never away from anything about testing.

We aren't skeptical about the fact that there could be more people like us and who knows they might be interested to join us. So here goes the job posting for the same: ( and I have posted this in LinkedIn and other communities - please feel free to share this job posting to all other Indian testers )

Consultants in Software Testing and Software Test Education :: Bangalore

Profile:

Edista Testing Institute ( www.edistatesting.com ) has a couple of openings for Consultants in Software Testing and Test Education.

It wouldn't be wrong if I say, they are looking for people who have the urge to be heroes in software testing. This role demands you to train ( yourself and others ), collaborate with the on going research activities, test products, learn and innovate.

This role also demands you to grow to an extent to be able to contribute valuable things to the testing community and work for its betterment.

About your co-workers:

You would work with skilled testers and brains who constantly engage in learning activities, blogging, discussions, teaching, mentoring, challenge and argue online in testing forums ( like www.testrepublic.com ), offline and reinvent the art of reinventing things in testing.

If you think you wanted to be a hero ( or hero-in ) in software testing and never got the opportunity, here it is.

Eligibility:

We'd be glad if you have worked as a tester for a while ( at least 2 - 5 years ) and also be glad if you are willing to travel within India (or abroad ) on short term assignments.

We would prefer you have a degree in Engineering or Science however if you dont have them but have a demonstrated ability of good thinking, we'd be fine.

Interview Process:

Our interview process is cut above all other organizations that neighbor us. We put you in the testers seat, give you time to test a product and have a discussion of your testing based on the report you produce.

We aren't too bothered if you dont know the difference between Sanity testing and Smoke testing because we believe, knowing the difference ( even if it exists ) doesn't make a huge difference.

We aren't bothered if you have a certification in Software Testing or not as long as you are passionate, skilled in testing, and have the fire and fuel to take you a long way. In simpler words, it *doesn't matter* if you dont have ISTQB, ISEB or CSTE certifications.

About Edista:

www.edistatesting.com

Time to join:

Immediate is preferred. A little delay is fine if you are stuck somewhere.

Contact:

Send your profile to : resume@edistatesting.com

If you can crunch your profile in one page, we'd silently thank you for that.

The results so far:

• We invited about 7 candidates so far who claimed to have energy, passion and demonstrable testing skills.
  
• We are seeing great benefits of this approach.
  
• It makes us spend 3 minutes ( after a person has finished the test and generated a report ) to know about the claims a tester has made in his profile are fale and a little bit about the organization that said, "Yeah, he can test".
• We spoke only to one person in depth as his report was quite interesting.
• We know that we can filter more candidates with lesser time we have and get better ones to work with.
• It would be dangerous to get a person who cant test and report credibly into any organization that wants to hire testers.
• Those who fake testing experience fear to even apply or even if they do and by our overlooking we invite them for the testing session, we dont need to spend time beyond 3 minutes post their test.
  

Testers have to be tested on their testing skills and not on memorization skill. The best test ( based on the current situation ) you could give a tester during an interview is to make him sit on a computer and asking him to test a piece of software by giving a meaningful mission and time to do it.

Side note: Are you planning to be in the supporters list?

Excerpts from participants work - Practical Hands on Software Testing Training

• Are you curious to look at excerpts of the participants work that we have been consistent in achieving from Hands on Testing Training that I talked about in 2008?
• Are you aware of the power of young generation?
• Are you aware of the power of coaching them with better education in software testing?
• Are you aware of the impact of providing better education in software testing to young generation today is going to be one of the key factors of deciding the future state of software testing?
• Are you aware you can read more about this?

Read more... and write to us ( isupport [//at//] etifinishingschool.com ) know if you'd like to support this initiative.

Learning to test better by teaching testing

Make a list of some of the teaching staff that considered bad at school or college. You might be able to notice a pattern - most of them were not willing to learn while they were teaching. Here are some of my stories and attempts that I made during my experience as a coach in software testing.


Story 1

I was teaching at Datamatics 2 weeks before. When it was time to officially end the workshop, a participant asked me, "What did you learn in these 2 days of workshop?". I was so happy to have got that question. I thanked her for the question and listed some of the learning I had in those 2 days. 

I also mentioned to them that I make a note of the learning I have in one or the other way. I also mentioned that my blog acts as my note taking tool at times. As you might have seen my Progress Report, the list of events and experiences that I had in every year is well documented.

The question, "What did you learn by teaching me?" is a powerful question to ask someone who is claiming to teach you because if the person has nothing then you know you might not want to sit in that class anymore or being in that class is a waste of your time. 

In some places I have an answer, "I learned that you can't be taught at least by me" and thankfully those people don't ask me that question. Learning and teaching needs an open mind. I think I must have a close mind to say, "Not everyone have an open mind".

I have learned some great deal of stuff about how to test better by teaching testers how to test. I understand that my brain is not capable of asking all the questions that I'd love to ask myself and hence I need other brains to ask me those questions. Where do I find those brains if I am stuck in one place and I keep interacting with just the same team for over years?

An year and a half back Ben Simo mentioned to me the idea of "Apply inputs that force all the error messages to occur" from James Whittaker's book while we were discussing about error messages. I liked the idea and it then got included in my armory of heuristics to test a product and helps me in model the product in more ways than what I have been doing.

Recently when I was teaching the Hands on Testing Training for Freshers for Edista Testing, Muhammed Kothari, pointed out a tool to me that helped in displaying all error messages programmed in most Windows applications. At every workshop, I ask the participants to challenge me in demonstrating the kind of testing I talk about. At Datamatics the challenge was to demonstrate testing on Microsoft Freecell game. On using the Process Explorer tool, I demonstrated how I could learn how many error messages the product is programmed with and designed my tests to all those error messages ( in other words, all printable strings from the program ). That's one way of demonstrating coverage.

Some curious testers were excited about it and came up and said, "I'd like to use the tool. It helps me find out areas that I might have been missing while testing the product" -- which is cool. [ some? well, one. :-) ]

That's just one of the many ways in which Process Explorer can be of great help to a tester. Note that Process Explorer is a tool. A tool that a tester can use to augment his testing effort and do a better testing. Tools are not just QTP, Winrunner, Loadrunner, Silk, Cotton, Wool etc...

Story 2

Lunch tables are a great learning ground. At Datamatics, I was joined by a group of testers at the lunch table. I ordered for a Dal Kichdi that some other people had ordered too. As I ordered late, others had started off with what they were served. When I got my Dal Kichdi and rowed the spoon over it for grabbing the first bite, I saw a hair (hopefully human one) in it and then found more. I picked a hair and showed it to testers and asked, "Did we order for this?" (meaning, no one order for bugs to be served with the software they buy) and then the brilliant Saurabh Sahoo, Test Lead at Datamatics asked, "Do you know why you found the hair while we didn't?", and the curious me said, "Can you explain, please?" for which he said, "You were looking at the plate while we were so much engrossed in the discussion and were watching each other's face".

That's a nice observation. It helped me explain to the people over table - that's what happens when you run scripts or test cases that have an expected result, "You are too much focused looking at the expected result that makes you blind to see other things happening in the environment. Having multiple oracles in your mind helps you see more than just the expected result and hence you'd be able to spot more problems than what you have been doing". 

I also added to it and said, "I wish we could say to our customers, why are you looking at the screen?" when they say they found a bug with our software.

I think that at least made Saurabh Sahoo convince that its worth spending time to learn things that I was trying to teach (and learn) and he spend an additional two and half hours after the workshop with me and he is determined to do a much better job. I learned a great deal answering the questions he asked me on clearing traps.

Answering questions like that definitely helps me in negotiating with management and clients.

Story 3

I was teaching at a Fortune 100 organization in Pune a couple of months ago. Within just 20 minutes of starting my workshop, the Test Lead announces to the whole class "This is class is a waste of time. Lets go back to work".

• How do you deal with such a situation? 
• How do you teach someone who doesn't want to learn from you?
• How do you put them in a receptive mode? 
• How do you know what to speak to them? 
• How do you know if its worth speaking to them anymore? 
• How do you know if they are more right than you?

What if you are in front of your customer and your customer wants to walk out of a meeting because they didn't like your idea? You might not be able to afford to lose all customers especially if recession is around.

I made an announcement, "All of you have the freedom to walk out of the class but I'd be happy if you answer a question: Have you ever walked out of a movie just because the titles of the movie suck?
 
If your answer is No, why would you want to do it here?"

Plus

I did something that I learned from the book Turning Numbers Into Knowledge that again cites Bruce Lee's master. 
I took a tea cup and poured water into it and kept pouring despite the cup overflowing and quoted Bruxe Lee's master, "Either your cup is too small or it is already full of opinions".

The result: I am teaching in the same organization again next month. The legs that rose up to walk out sat back and enjoyed the ride. Some even wrote to their management to have the class for other testers in their group.

Story 4

A common problem that most testers talk about - is being caught for missing a bug when a customer finds it. A tester in one of the previous workshop said, "It is my responsibility if I miss a bug and hence I have to be more careful with the tests I do". I was reminded of what Michael Bolton said, "I am a character in the story of how a bug got missed". It indicates that, we testers are a part of the entire story and we do not play the hero role in a bug that missed our hands.

That tester was adamant that testers should be made completely responsible for it and not anyone else because it was their fault and they shouldn't have missed it. I tried asking him, "How about making the person who put the bug into the product more responsible for it than you?" and yet that didn't help.

I made a pretty bold move, "I am going to slap you now. Would you blame yourself for standing in a place where I was swinging my hands or blame me for doing something I shoudn't have done?"

That tester remained adamant but other testers got the message that a tester is a character in the story of how a bug got missed and not the hero in the same story. Later that night when I was self retrospecting, I realized that I should have used a non violent example to explain it. Teaching testing to experienced testers helps in correcting your communication and thinking aspects.

Some important points to consider

• It is easy to teach dull minds that don't oppose and appear to nod for every idea that you share with them. Teaching bright minds are tough and demanding. They teach you a lot about how to test better and what more I want than that.

• Most people who teach testing in India at several training centers haven't done testing at all or haven't done enough testing. So, those who have remained a hands on tester and teaches testing gains a competitive edge.

• If you have that competitive edge then there is good money as well.
  

• Helps in meeting a lot of different kinds of testers and understanding different contexts, problems and different solutions to problems. It helps in avoiding traps much better and faster and translates to better testing.
  

• It is a challenge to get to know more about ourselves, get tested , refine communication, thinking, learning, teaching and testing skills.

• Make a list of some of the great testers you know ( in my case Jerry, Cem, Ben, James, Michael, Scott, Kohl, Elizabeth, Vipul ... ), you'd find that they teach extensively. Oh, you have been doing that as well but probably you have been teaching yourself all this time. Think about why do they do that?
•  
•  Go, at least teach yourselves. 
•  
• Teaching, teaches the teacher about teaching
  

If you are a tester and is interested to join a small test group that specailzes in coaching, consulting, and testing services, write to me. You'd work with some of the upcoming very good minds who care and live for benefitting the software testing community. Write to me: pradeep.srajan@gmail.com

Happy testing and Sad Testing

I don't know from where I heard "Happy Testing", the first time and fear to think if it echoed within me. I also don't know how I caught those words to sign all my e-mails to testers I communicate, writing "Happy Testing" at the end of the e-mail. I noticed that a lot of other testers to whom I communicated also started doing that in their e-mail communication to me and in their blog posts. I didn't know what "Happy Testing" meant when I started using it long back but I think this post explains what I mean by Happy Testing.

For the moment, forget about people ( that includes me ) asking you to do good testing, better testing, great testing, pleasing customers, pleasing managers, and getting great hike. Think about doing happy testing.

As a tester what makes me happy is when I find 'a' bug. What makes me more happy is when I find more than a bug. What makes me the happiest is when I find more and more bugs in every product I test.

I come across a lot of Sad testers and observe a pattern of the bugs they find. Most Sad testers that I come across are the ones who follow test scripts or cases to find bugs. You might observe that those Sad testers say "These test cases found those bugs". I am sure if test cases have life, they would be happy set of organisms because a lot of Sad testers owe credit to the test case for whatever bugs they ( humans ) find.

Its not that those Sad testers are always unhappy about the bugs they find, its that they are happy and not happy enough to recognize how they can be more happy.

If finding a bug doesn't make you happy, how sad a tester you are!

Don't ask me the secret of being a "Happy Tester". I am sure a lot of Sad testers won't believe my reply, "Rapid Software Testing, Exploratory Testing and Context Driven Testing that provides me freedom to think, experiment and find a lot of bugs".

In every Skilled Exploratory Testing corporate workshop I do, I challenge testers on testing their product for 90 minutes and find a lot of bugs within that short span. One of the recent experience was in SAS, Pune, where testers started clapping after the 90th minute witnessing the Happy Testing I did. It felt like a Hero to receive clapping from testers for demonstrating Rapid Software Testing.

What I heard from Rajesh K , one of the managers who nominated his team and attended the workshop in SAS over an e-mail was "The bug count has certainly gone up". Happy Testing Rajesh, Vikram, Manoj Nair and their teams.

Jerry Weinberg revamped his website recently and a sentence in his revamped website made me think what Jerry was trying to say and here is the sentence for you "Dedicated to Helping Smart People be Happy". It made me wonder if smart people can be unhappy. I then thought about all those talented testers in India ( and probably other countries, too ) who have been forced to write and execute test cases and follow best practices that might have worked for someone else. That reminds me to say, I get happy when I read Jerry Weinberg's books because reading his work makes smart and I made myself a lot happier when I bought Jerry's books for about 300 US dollars during my trip to Canada.

Speaking of all that, let me ask a question to myself: Am I never unhappy?

Oh yes, whenever I come across people who are unhappy and they don't want to listen to stuff that can make them realize they are smart and can be happy enough, too.

Am I talking as though I am Mr Perfect?

Oh well, I forgot to share with you that an important lesson I learned is that - Humans are fallible and so are their ideas. Everything is a heuristic and it is a choice of heuristics in a suitable context that makes a person smart. If you think Perfect Software and Software Testing is possible, read Jerry Weinberg's Perfect Software and other Illusions about Testing.

Unfortunately, I want so see those sad testers I come across as happy as me. Unfortunately, they aren't happy that I am claiming to be happy by doing things that they think is impractical or hard.

Fortunately there are some sad testers who are smart and want to be more happy. I live to help them. Oh! Did I forget to tell you that I get happy when I help testers find more bugs.


-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --

It's a "tester" who finds a bug, even with the robust Google Search

The image you might see above is a screen shot of what I saw after hitting the 12th page for the search results of "tester". I am not sure if you can reproduce that because I haven't investigated on it but I did plan to capture it to demonstrate:

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it."

Be it with the robust systems like Google Search Engine or with weak systems that we might be using, it is always a HUMAN who finds a bug. A lot of testers I know think of "test case" finding a bug.

There is a test case document that consists of 9856985956895869698569956985698459698 test cases and no tester executing it wouldn't find bugs by itself. There is a test case document with 3 documented tests and a tester takes the help of that to find bugs when he executes, observes the result and recognizes a bug.

A test case is an extension of a test idea. What matters to a tester is a test idea and not the test case. Skilled (exploratory testers ( humans ) use tons of ideas ( heuristics and oracles ) to find and recognize bugs. That's why they can find more bugs that matter than those running thousands of test cases over and over again.

Honestly, 99% of testers I have come across didn't say - "I read each test case each time I have to execute it after I have done it once. Also, I religiously follow what is written in the test case".

What happens when they deviate from the documented test case is, they are exploring and running different test. Maybe they don't like to call it that way because their management who pays wouldn't like to know that they are not executing the "test cases".

That's how customers are fooled by management saying "yes, we are running test cases" and yet benefited by the testing community by running more tests.

A test idea can be executed in hundreds of different ways. Check out the deep analysis made by James Bach and Michael Bolton on What Do Scripts Tell Us?


--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --

Reduce Reuse Recycle

I toured Singapore for about 10 days this month. I don't know why my eyes kept catching signboards that suggested people to Reduce, Reuse and Recycle plastics. I think it was because I am from India.

As a part of my tour, I had been to a Zoo and there, I heard the animal show host explaining the need to reduce, reuse and recycle as it impacts the environment and animals.

I felt great about Singaporeans, for stressing the point in many places. No wonder their country ( most parts ) is so clean and tidy that I felt I am in a foreign land. I thought for a while that people in India don't talk about Reduce, Reuse and Recycle as much as Singaporeans do.

16 days after the tour ended, I realized I was wrong. People in India talk as much as Singaporeans do about reducing, reusing and recycling. Here is how they ask:

How can I reduce testing and thereby decrease the cost of the project and increase my profitability?

If testing is a job of providing quality related information to the stake holders to help them take better informed what could reducing testing mean?

I think the people who talk about reducing testing to decrease costs want to know if there is

• a way (or more than one) to make testers more efficient, skilled and competent.
• a way (or more than one) to get information more faster.
• a way (or more than one) to know if they can save costs by not purchasing unnecessary tools.
• a way (or more than one) to know if they can avoid hiring bad candidates.
• a way (or more than one) to know if they could spend less on managing attrition.
• a way (or more than one) to know if they are not doing things that would not add value.
• a way (or more than one) to know if they are meeting the mission.
• a way (or more than one) to know if they can fall into fewer traps.
• a way (or more than one) to know if they can recover much faster from the traps.
• a way (or more than one) to know if they can be more successful in getting more projects.

and : How can I reuse and recycle my test cases, test scripts and testers for future projects thereby reducing the cost of my future projects to increase my profitability?

I think people who talk about reusing / recycling test cases, scripts and testers for future projects want to know:

• many ways (or at least one) to know if they can cut costs by not needing to train testers on a similar domain, technology or product.
  
• many ways (or at least one) to know if they can cut costs by not having to spend time on writing test cases and test scripts for a new project by modifying existing test cases and test scripts.
• many ways (or at least one) to know if they can cut costs by not having to write code that tests code by modifying existing code.
• many ways (or at least one) to know if they can cut costs by not needing to spend much of a time on learning the new product and finding bugs faster.
• many ways (or at least one) to know if by thinking of reusing and recycling, they are definitely saving costs without sacrificing the value they want to add.

Well, if all those who ask the questions knew how to ask it more elaborate or deeper questions, we'd be living in a different world. Thankfully, we still are in the same world.

How did I reuse, recycle and reduce?

• I don't recommend writing test cases and executing tests with the help of that. Although I am not someone who'd like looking at test cases, there was a context in which I looked in to test case document that someone else had written, to gather ideas for my exploratory testing. That's how I reused a test case. It definitely reduced the cost because I took help of an already existing database of ideas. ( That doesn't mean test cases can be handy for ideas to test. A check list, cheat sheet, mnemonics, heuristics... might do it as well more cost effectively).
  

• In one of the several product development organizations I worked for, I identified that a tester was not performing fair enough. I probed for his history of performance within the organization in past projects and recommended him to be fired ( of course, I had the authority to recommend ) He was being paid a lot for he had over 7 years experience. The management feared firing him could send wrong signals to other team members but I asked, "What more right a signal can people get?". On firing him we had money to afford hiring 4 junior testers for 1/5th of what he was paid and got more than what he was delivering. That's how I reduced the cost of the product.

There might be a lot of ways to solve a problem and there might be a lot more ways to not solve them. Unfortunately the ways to not solve a problem appear like the ways to solve problem. Humans are stuck!

If working on reducing, reusing and recycling test cases aren't working, you might want to reduce your intention of reducing, reusing and recycling /and/ think of reusing those ideas in a different context /or /recycle those ideas at a later date when you think the context has changed to suit it.

--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --

Automation replaces humans - The truth about what kind of humans it replaces

I often meet people who claim to be "automation testers" and I love and hate to argue with them about test automation. The more I meet such people, I am getting confident about making a conjecture that more than many people who claim to be doing test automating in India and other APAC country testers I visited, are doing it without knowing it or without wanting to know it.

Here are the things they say:

• Test automation replaces manual testing.
• Test automation is faster than manual testing.
• Test automation is cheaper than manual testing.
• Test automation is more reliable than manual testing.
• Test automation removes the monotonous work of a manual tester.
• Test automation provides confidence that the product works.
  
• Test automation is the future.
• Test automation is better than manual testing as a career option.
• Test automation tools provides Return On Investment.
• Test automation can find a lot more bugs than a manual tester.
• Test automation provides more job opportunities.
• Test automation is a way to go about testing.
• Test automation is about converting manual test cases to automation.

Lets look at some of them based on the conversation I have had with such people saying one or more of the above or the questions I'd like to ask you:

Test automation replaces manual testing and Test automation is the future

There are two planes standing there. One of them was tested by a trillion scripts that flew the plane for 400 hours and the other was tested by those 4 humans, flying 100 hours on it. If you were to board one of them, which one would you board and why?

None of those great human beings who said, "Test automation replaces manual testing" said, "I shall board the plane which was tested by a trillion scripts" and instead said, "I will take the plane that humans tested because I feel safer throughout the flight"

So did test automation replace manual testing even for those automating tests?
Test automation is the future of what?

Test automation is faster than manual testing & Test automation is cheaper than manual testing

Here are some statements I make. You can pick ones that makes most sense to you:

A. My Toshiba laptop runs faster than a McLaren Mercedes Formula1 car.
B. My Toshiba laptop runs faster than my old Acer laptop.
C. My Toshiba laptop runs faster on an XP than on Vista.
D. My Toshiba laptop runs faster than that horse which has a broken leg.
E. When someone needs help and I need to run to them, I use a test automation tool because that's faster.
F. QTP license is cheaper than hiring a 5 year experienced skilled tester.
G. QTP is expensive than Winrunner.
I. QTP is cheaper than a Ferrari engine.
J. Winrunner is cheaper than a business class ticket from Bangalore-Toronto.

Option C and G makes most sense. Why? Why not A , D, E, F, G, I, and J?

"Well, option C and G makes most sense to me because you are comparing the same Toshiba laptop when Vista and XP runs on it AND you are comparing two testing tools in G"

Is test automation [ using a software or a computer ] faster than humans to those who say that?
Is test automation cheaper than those human brains who claim to create it?

Test Automation is more reliable than manual testing

I first heard this eye opener, insightful idea from Michael Bolton , "People write code to find bugs on another code. They write code that is buggy which claims to find bugs on another code" and "Writing test scripts is another development project . Most of them don't realize that they are running two development projects in parallel and hence their main development project suffers."

So is test automation reliable than manual testing to those who write code that test another code?
We humans are fallible and all things we create are likely to be fallible, too. Test automation scripts are not excused by God to feel "All test scripts don't have bugs".

That doesn't mean human testing is more reliable. It means, anything done by humans is fallible. When you depend on a fallible thing that a fallible human created, you might not want to talk about reliability there.

Test automation removes the monotonous work of a manual tester

Why is testing monotonous? The unspoken truth!

Test automation is better than manual testing as a career option

Well, people think by becoming an automation tester they aren't doing manual testing. Here is what I have discovered - There are more people who claim themselves as automation testers in India than those who claim to be manual testers. So, let me know which is not manual testing? Can manual be termed as anything that does involve humans?

Lack of skilled testers at least has helped me grow phenomenal in my career. I am not THE skilled but I claim to be one of them and there might be very few like me and not many in India.

Test automation can find a lot more bugs than a manual tester & Test automation provides confidence that the product works

Yes, test automation is very intelligent. If a script is programmed to catch a bug when a value exceeds 5, it can also notice that the value did not exceed 5 and pass the test but the screen went blue for a while. If after all scripts have run and there aren't any bugs found, the test automation suite will alter its values and run once again to catch bugs. As the scripts run over and over again, the scripts learn the product well and if the management needs to take a decision to ship the product they would come near the script and say,

"Oh magical script, I want to ship this product. If you think it's a bad idea to ship now, speak out else remain silent"and then they listen to the script and decide whether to ship or not.

Yes, we are dumb. We can see a numerical value not exceeding 5 and pass the test case but also we can't spot the screen becoming blue for a while and do not report that as an issue while running a test that we claim to pass.


Test automation is about converting manual test cases to automation

Oh wow! So, I can't write automation scripts without writing cases and executing it at least once by my own? And if I write, many might cry "Foul".

Well, test automation scripts are just another program as the program it tests. Did developers have something they did manually before they wrote the code that you test with your manually converted automation test scripts?

Test automation tools provides Return On Investment.

Ok, so here is the industry standard of returns on a tool: For every $500 invested on a tool, a good ROI is about the tool helping the team find 50 bugs.

So, a good ROI for $9000 tool = ( 9000 / 500 ) * 50 = 900 bugs

If that doesn't make sense to you then what is the Return on Investment for a tool?

Bugs? Customer appreciation? The number of testers who get fired? The cost savings of hiring 10 skilled testers v/s the cost of buying the tool? The value that the tool brings to the testing effort? Tons of documentation produced? The added cost of training? The added cost of support? The added confusion that prevails? The number of fake experienced testers who apply to the job of a toolsmith?

Test automation is better than manual testing as a career option

Test automation, at least in India, is more chaotic than the so called manual testing. Hardly any tester knows why they are automating things. You may join any forum where testers are active discussing tools and automation, you'd see questions that the help file of the tool has answers and you would discover replies to such questions that is sometimes horribly wrong and yet the person who asked the question says, "Thanks" and the chaos continues.

There is more chaos among so called automation testers than so called manual testers because the so called automation testers form a huge community than the so called manual testers. More people with lack of insightful ideas means more chaos.

Want to know some useful information about automation or want ideas to think more about it?

Cem Kaner prefers to call most of the automation activities that most testers do as Computer Assisted Testing and I couldn't find evidence to refute the idea. I think it is a great idea to call it "Computer Assisted Testing" and you would know why it is a great idea if you go through this article by Cem or the PDF version of it.

So, all of us irrespective of calling us manual or automation testers do Computer Assisted Testing.

I know of an article written by James Bach that made many people who claims a lot of things about test automation feel guilty and I am sure if you have reached this place reading this post of mine, you'd be interested at reading James Bach's Test Automation Snake Oil - the paper or the slides .

I also know of a blog post from James - Manual tests cannot be automated and Shrini's post on A Mystery called automated testing.

Here is why we do different things in testing such as writing test code, executing test ideas, investigating, diversify approaches, think of different techniques, use heuristics, use oracles, Explore... - because each of them add different kind of value and each of them helps in finding different kinds of information or helps us ask new questions AND NOT because something is better than the other.

There is lot of thinking, reading and writing, I and you require, to know about test automation or about testing. The lazy ones don't get to learn and continue to spoil our craft because they don't stop writing, just like me. [ It's intentional ]

I know I haven't answered a question that you might have in mind but that's intentional again.

Update: Here is Jonathan Kohl's awesome interview about the same topic. Thanks to Konstantin for pointing it out through a comment.

--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --

Educating customers on testing

Everyone has heard about testing and unfortunately everyone has heard about it from different sources that say different and contradicting things.

• One of the customer I worked for thought - testing is about delivering bug free product - which, in my opinion and many other experts with whom I interact is an impractical idea about testing.
• Another customer I worked for thought - testing is about delivering quality - which, again, is a bad idea.
• Another customer thought - testing is about executing test cases and based on the test case pass/fail ratio decide to release - even if the test cases isn't helping testers catch bugs.
• Another customer thought - testing is a job of repeating tests to compare build quality and take a decision to release a specific build to his/her customer.
• Another customer thought - testing is meeting specification.
• Another customer thought - testing is satisfying the customer.

Working with multiple customers is a huge challenge for test managers or testers like me and you. I have so far been repeating the word "customer" and whom do you think I am referring to?

I am referring to the manager we report to and the manager we report to is also our first customer. When testers quit their job and join another one, they never ask their customer (manager) what he/she thinks about testing. That's why some people feel they were performing good in their previous company or they are able to perform better in their recent organization - reason being - their idea of testing matched with that of their manager's idea.

If our idea of testing doesn't match with that of your customer's ( manager ), you and I seem to face a lot of trouble in meeting our customer's expectation and are unable to satisfy them.

Everyone, including you and me might be thinking that the idea we have towards testing is the right one. I have been realizing and teaching that there isn't THE RIGHT WAY to do something. Rapid Software Testing taught me cost v/s value and I took it very serious and I am practicing it.

All customers, irrespective of what definition they have towards testing look for cost v/s value. I am striving to deliver a fabulous value to my customers and I am facing a challenge since I realize what value means to me is different from what value means to my customer (manager).

That's why I was looking as a bad tester (despite finding a lot of bugs) to some of my managers in the past. They think the value they wanted to get out of me is - to execute thousands of scripts or test cases.

Unfortunately I wanted to add more value to the testing efforts which wasn't liked and was fired for attempting to do that. They weren't wrong - they just cleared a trap that they dug in when they hired me.

I learned - what I think as value might be a threat to someone else's idea of value and I value this learning I had about value.

Once I (and my team) found a lot of bugs, so much that there weren't as many bugs that the test case document had helped the team catch in the past. The questions from our customer started to focus on why we had a bad test case document and *not* "Why can't we do more of what you guys did?"

A definition, customer has towards "value" doesn't appear to change despite showing enough evidence. Its hard to continue working for such a customer but there is another dimension to think - money.

Some of those who attend my Exercises for a testers mind - A Rapid Software Testing Approach workshop say, "Well, this is great. I am excited to do testing this way but I suspect the problem is - I don't think my manager would be interested on any of these things I do"

I have had different replies to the above:

• Stop thinking that your manager would not be interested. That's the first step.
• Convince your manager to take this workshop.
• Educate your manager on this.
• Just do it and make your manager ask, "How are you doing these things?"

Here is a list that I suggest you to use, if you realize facing similar challenges: (I practice the same, too)

• Educating your customer, starts with you.
  
• Stop thinking that your first customer wouldn't be willing to see more value.
• Provide evidence for him/her and ask her to critique the evidence you provide.
• Request for a discussion about - What you think of value v/s What your first customer thinks of value.
  
• If it fails the first time, provide repeated evidence and suddenly do things the way your first customer wants to do - see if your first customer comes back asking why suddenly things appear not so good.
• If you have been successful educating your first customer, then motivate your first customer to educate his/her first customer.

BTW, what do you plan to educate your first customer on?

Here is a list of heuristics about educating your first customers:

• Educate yourself before you educate your first customer.
  
• Testing is questioning a product in order to evaluate it AND testers provide quality related information to stakeholders/management to help them take better informed decisions -- James Bach & Cem Kaner
  
• 100 test cases might not help a tester catch 1000 bugs and if you want more bugs, exploratory testing ( with a mission/charter) is an important approach to be incorporated. Adding another 1000 test cases is not a solution.
• Specification document is just one or the oracle that helps a tester find bugs. End users or your customers never use specification oracle to spot bugs and hence going beyond specification can help in finding more bugs.
• Session Based Test Management is a way where Exploratory Testing can be tracked and managed more efficiently.
• Spending less time on documentation that's going to end up wasteful is a nice way to get more time executing tests to find more bugs.
• "If you are too bothered about repeatability of tests - then it is good to have a document of test ideas and *not* test cases that tends to have more ambiguity."
• "Diversification of approaches and techniques helps testers find more and different bugs that otherwise might be caught by a customer."
• Software testing books are not the *only* books that helps tester find bugs and hence the library should stock a lot of books on thinking, ideas, pattern, philosophy, psychology, humans, communication...
• We didn't appear to find a bug because of one or all of these:
  
• We did not have the skill.
• Someone did not let us know when they put it.
• A stakeholder asked us not to report it although we found it.
• We didn't intend to find that.
• We thought it is OK to not to find that, if it existed.
• We were finding other important bugs thinking opportunity cost.
• We didn't have the right oracle.
  
• { You might want to try a lot more, provided some of the above works for you. Sometimes I am aware that I am going to fall into a trap but that's a way I choose to avoid bigger traps }
  

There might be a trap you may fall into. Some of your first customer's might not deserve/ might be reluctant / be egoistic - to such an education, sense it out smartly and stop educating him/her to get more time to educate yourself. An indication that you fell into a trap is when your first customer fires you.

+ A heuristic is a fallible method of solving a problem
++ An oracle is a principle or mechanism by which we (humans) identify problems.

--
Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." --

Notes from spying test experts

This is a top secret post and be very careful with the information. I am not sure if I can expose my notes to my dear blog readers but I am taking a risk of doing it.

There are some secrets of test experts that not many know. I had a mission sometime back to spy about how test experts actually find many important problems, quickly.

Many people think ( that included me, too) that Test Experts do lot more tests and have more fantastic test ideas and that's why they find many important problems, quickly.

I was hoping that there is more information hiding than what is visible about test experts and spying helped to uncover those hidden secrets.

This post is all about the secrets that I found spying test experts and the secrets of how they find many bugs?

You can trust me about the information to an extent that I and many other testers have been practicing the secrets that has resulted to our success of finding many important problems, quickly.


They just do one test to find many bugs!

Now, I am sure you don't want to believe me because my finding says that they do just one test to find many bugs. I am sure you have seen testers finding one bug per tests that find a bug and that makes you not believe this information but... you must understand that the secrets are always unbelievable.

You can feel safe after knowing a fact that I too didn't trust the secret of one test finding many bugs but my spying mission was to find out more granular details about it.

Looking more carefully into it, you might discover that it is not a test that finds a bug but it is a human that finds a bug and a test plays a role in helping the human find it. So there is something with humans that is helping to find bugs and not completely with the tests.

When I collected this much of information over months of spying, I was invited to be a part of the round table conference of experts as I had posed myself as a test expert from India (I knew I wasn't a test expert and even today I am too conscious that I am not one such) but you know... I am a spy and had to pose that way.

Over the round table conference, test experts started discussion about ORACLES* and I didn't know their terminology and just took notes. That evening, when I returned to my hotel, I was trying to connect ORACLES and the spying mission I had in hand.

* Oracles - principle or mechanism by which we (humans) identify problems

I was perhaps lucky that James Bach and Michael Bolton were staying in the same hotel that I was put up and we caught up at the dinner table.

I had an idea that I hope it worked. I was hoping that James Bach is a kind of person who would spill off the secrets in a context to prove me wrong and help me learn the secrets if I talked nonsense about ORACLES, because his passion to the craft of software testing is something that I am trying to match. Michael Bolton, the Birbal of North American software testing was a person whom I had to handle very carefully as he is more likely to identify me as a spy. At the end, I assumed that I was able to outsmart Michael in not letting him know that I am spy.

Going back to the hotel room, I started writing my discoveries about the secrets of test experts and the granular details of how they find important problems,quickly.

Before I publish a report to end the spying mission, I was sure that I had to practice the details I collected. I did practice and was excited that those ideas really worked and I found more bugs by executing one test.

Why many testers might not be able to practice the secrets that I am unlocking here is because they have a fixed expected result that is derived from a specification document that the tester thinks is The Holy Bible for the project.

Specification Document - is one single oracle and hence you might find one bug per test you execute.

The important thing is experts define a bug as "anything that threatens the value of the product" (-- James Bach)

Here are a list of Oracles that experts use with every test they do:

Consistency with the Image: When I execute a test, I try comparing and contrasting it with the image that my company or stake holders have been projecting about the product or the company.

For instance, if my company has an image in the market as people who produce software whose performance is good, and I execute a test whose documented expected result is "This functionality should work", I ask question, "Well it works but didn't it take too long to perform the operation?" and then I find a bug and say, "Although this operation goes through the time it took to complete the operation exceeds beyond the image the market has about us"

Consistency with Similar Product(s): When I execute a test the functionality might work as expected but I ask a question, "Well it works but doesn't it seem to perform in a way that users who are used to similar products might expect it to happen?" and then I find an important problem, quickly.

Consistency within the Product: When I execute a test that appears to have passed, I ask a question, "Well it works but doesn't it work in a different way than other areas of the same product?" and then I find an important problem, quickly.

Consistency with Claims: Marketing personnel and my manager are making certain claims about the product. When I execute a test and the test appears to have passed, I ask a question, "Well it works but does the way it works go against the claim that they make?" and then I find a bug.

Consistency with Statutes: When I execute a test that appears to have passed, I ask a question, "Well it works but does the way it works go against a law that an organization or a certifying authority might have?" and then I find a bug.

Consistency with User's Expectation: When I execute a test that appears to have passed, I ask a question, "Well it works but does it work in a way that the users whom I am aware of might not want it to happen?" and then I find a bug.

Consistency with History: When I execute a test that appears to have passed, I ask a question, "Well it works but does it work in the way the previous releases of this product work?" and then I find a bug.

While writing this post, I got an e-mail from James and Michael that reads...

Dear Pradeep,

We knew that you were spying us for a long time. We also saw the way in which you took notes, asked questions and drew inferences from what you could capture and that impressed us to help you learn more about testing because those are some important skills that a tester needs to have, to add good value for the cost customers pay for. We consider to hire you and we recommend you to quit spying profession and take up testing where you might get more laurels for your skills. Here is a link to know more about more secrets : www.satisfice.com/rst.pdf

Good luck!

-- James Bach & Michael Bolton

I was excited and I did quit spying, to take up testing as a full time activity. I thought I shouldn't post this in my blog because I didn't want those secrets to be given away for free and saved a draft of this post. I am afraid if Blogger has a bug that publishes drafts that are not intended to be published.


-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton


Update : Michael Bolton, blessed me with a post on his blog for this article of mine. I was jumping in joy for at least half an hour reading that and I am pleased to share it with you: http://www.developsense.com/2007/09/if-test-passes-in-forest-and-no-one.html

Mother Nature - A teacher of all good testers

When I started to test, I wasn't aware that I entered a fantastic field that demands me to look at other professions to do a good job as a tester. A little later I realized that people in a specific profession have lots to learn from other professions to perform better at what they are doing.

Everyone, in my opinion does that but the question is: How many do it consciously?

Don't worry, you aren't left behind and here is your opportunity to unlock what you could learn from other professions. Here is an example of how Michael Bolton and Ben Simo talked about "How doctors think and the learning of a tester from that thought process" . During Michael's previous visit to India, we did talk about how his experience of cooking and theater that helped him in his testing and I was very glad to hear those stories.

I intend to pull out my notes of how people in different professions think that can help testers in their testing activity:


Doctors


I had mentioned this earlier and I would like to re-iterate. Doctors ask a lot of questions as a part of treating you. When a patient says "I am having a back pain", I have observed doctors asking questions about the hand and legs and they carefully observe emotions of the patient when they try to press the back, different places in leg and hands.

Software and human body are very complex systems. The "back pain" might be a symptom of a bigger problem and hence doctors ask questions about other parts of the body. So, as a tester if I encounter a bug, I get to think that this bug might be a symptom of another big bug that is hiding and ask questions that help me figure out the big bug.

When a patient says, "stomach ache", I have observed doctors prescribe blood tests and various other tests to take an informed decision. The management needs as good information as possible to take better informed decisions and testers need to supply the information. By prescribing blood and other tests, the doctors are looking for coverage and so, we testers need to look for coverage than to find xxxxxx number of functionality bugs (unless that is the mission).

When a patient is in a critical state and needs to be operated, there are diversified set of doctors who are in the operation theater. For instance, an Anesthesia Specialist, General Surgeon, Neuro Surgeon, Heart Specialist... and NOT all General Surgeons or all Neuro Surgeons do the job. That is a fantastic example of diversity and value addition to the operation's success. A testing team needs to be diversified. Not all testers who know to run tools like QTP, WR, LR ( toolsmith's I mean ) can add value to the project and successfully achieve the mission. Look at my FAQ's for more information on diversity of testing teams and its benefits.


The Indian cobbler ( I don't know of any other country cobblers)


An expensive shoe that we purchase might have been manufactured by a top company with state of the art machines but when it is torn or needs a fix for the sole, we do not go the manufacturing unit but to a cobbler nearby.

The Indian cobbler is a pretty simple guy. He uses the tools that are not state of the art but yet does a fantastic job whenever I or my friends have gone to him to get a problem fixed. He doesn't intend to use a tool because other cobblers think it is state of the art because it doesn't suit his context. Many companies buy tools from vendors who market it as state of the art, and later discover that it isn't suiting their context well but force the testers to use the tool to see some value of the money spent on it and lose the value that could have actually been delivered (if the tools were not put to use).

There are some cobblers who move on road and they dont carry tools that are hard to carry or need electricity to operate and yet complete the mission assigned to them. They do a manual activity and harness the potential in the tools that they carry to maximum extent. No testing is completely manual and no testing is completely automated. Those who think they are testing something completely manual are as much wrong as those who think their testing is completely automated. Those testers who know to carry the tools that suit their context are smarter and will add great value as compared to those testers who carry tools with them because someone said "That's the future".


Police ( What movies has shown me)


Catching criminals is as interesting and challenging as catching bugs. Police look for clues during investigation of a crime, to nab or zero down on criminals. They do not just look at obvious places but non obvious places, too. For instance an investigator looks at a dustbin while investigating a murder, finds a cigar in the dustbin and draws inferences about the criminal. Testers usually do not look at clues that surround a crash or hang and miss the actual criminal that many a times is caught by the end user. Log files is one such clue that has helped me nab several other criminals.

Police personnel ask the same question in multiple ways to the same person at different situations and look for consistency with the answer. Anything inconsistent helps them to get fishy and are one step closer to catching the criminal. If a test passes that doesn't mean it really is a "pass". The same test might fail when executed in a different time frame, different input, different tester executing it, different PC, different network, different hardware... If you observe carefully policemen are using consistency oracles to find the culprit.

I am thrilled that I am looking and learning from many other professions, too and I am glad that I am a software tester who is gifted to see the beauty of testing. I must thank God for his blessings. Not all testers are gifted but they can become one such when they start learning from all possible situations, people, things, objects, happening...

-- -- -- -- -- -- --

When you set your mission to become a wonderful tester, the nature will take care of your learning and all you need to do it to keep all your senses wide open. Mother Nature is the best teacher and you wont realize that by reading this sentence unless you experience it.

James Bach, today's leading test expert and testing legend, whose work has affected the entire testing community, is a strong example of someone who forced Mother Nature to teach him by becoming a self drop out of school during 8th grade ( or Standard ). His relationship and thought process is a gift that Mother Nature bestowed on me when I cried to Mother Nature seeking help for learning to test better.

So, set your mission and cry aloud, SHE will help you. SHE would test your passion under turbulent situations but once you pass HER test, you will get more tests and will start learning and enjoying the experience.


Reminder: Registration for the 500 rupees half day testing mania, is still open, so book your seats to experience such wonderful stuff. For details look at the left hand links section of this blog or simply link click maadi.

How does Testing look from the eyes of Tester Tested?

Michael Bolton, gave me a writing exercise a month back ( to rewrite an article that I wrote without changing the meaning but changing the examples ) when I showed him a piece of my work that was published by Mahantesh Ashok Pattan, a fundoo test professional from Microsoft. Mahantesh currently heads a testing excellence team in Microsoft's Hyderabad campus. Mahantesh honored me when he asked if I could write an article for his upcoming software testing website. So here goes my exercise result ...

How does Testing look from the eyes of Tester Tested? Version 2.0

Testing - from my eyes looks different, each day. Everyday, I ensure that I learn something new, observe something more carefully, listen to something more carefully, think deep on a topic that I choose to think, ask questions, read what others write about testing, teach testing to someone, learn a skill from someone, test, self critique my own testing, use Wikipedia/Google, practice the skills that I have developed yesterday, day before yesterday and what you might call as “past” and think about the future.

That doesn’t mean I am a workaholic! How can that be?

3 years back, I went to meet a friend in Jayanagar 4th block, Bangalore. While we we were having a walk the talk session, I noticed 2 child rag pickers running towards a food packet that was lying in a dust bin. A dog was running from the opposite direction, trying to grab the food packet before the 2 rag pickers could get it. The dog lost battle and the 2 rag pickers put their hands into the food packet. I was shocked witnessing this scene (without knowing that a bigger shock was waiting for me)

Today I can say, "The two rag pickers taught me a wonderful lesson that I can apply to testing, to better my testing"

Here is what happened: While the two rag pickers put their hands on the food packet, I shouted, "Drop that, I shall get you good food in a hotel" and their body language suggested to me that they would come with me to the hotel.

I started walking towards a hotel nearby to get them good hot food. Minutes before we reached the hotel, I turned back to see if the 2 guys are following and I noticed that one guy still held the food that he could hold picked from the food packet while the dog had a thankful look at me as the guys had dropped the food packet.

I turned to him and said, "Hey, I told I would get you good food but why aren't you throwing that junk food?"

He replied, "Well, what if the hotel is closed? what if you change your mind? what if you played a prank on us? I still have something to eat as compared to the other guy who doesn't have anything."

Poverty might have made him say that but I was shocked by his thinking capabilities. I am sure he has not had any formal education and everything that he learned is through his experiences in life and is also having the same challenge that we face, "survival of the fittest".

When this incident happened, I must admit that I didn't think of a testing lesson that I could learn from it but an year later when I sat relaxed in my chair after a good heavy meal and I thought about the rag picker incident, something that is related to testing struck me.

We testers lose control of things many times that might have been of great help to do a better testing. For instance, I have worked with testers who waited for the requirement documents to arrive although the product ( in some shape ) was in hand.

The boy who had a little food in his hand still had control of the situation and thought of possible situations in which he might have lost everything ( the food packet and the good hot food that I promised ) had he not done that act. Taking control of the situations ( no matter how hard it is ) is a skill that testers need to conciously practice.

An oracle ( not the database ) is a principle or mechanism in which we identify problems. If you learn and start using oracles consciously, I guess you might be in a position to take control of a situation which demands you to test without specifications or with less or poorly documented specifications.

Case 1: I hand over you my mobile phone and ask you to send an SMS and you see the application crashing for that operation; would you call it a bug/issue? (Although you don’t know what the requirement document for the mobile phone that I possess say)

Case 2: I hand over my mobile phone to you and ask you to open each application and exit it. In all applications you use the Right Soft Key to exit whereas one application demands you to exit using the Left Soft Key. Would you have something to say or ask?

In Case 1 and 2, we are using “oracles” of consistency and heuristics (fallible method of solving a problem) to find or discuss about an issue. Doesn’t that mean having diversified set of oracles, heuristics, and test design techniques can fetch great value for the testing you do?

We have a control over the software, platforms, our thoughts, ideas, questions we can ask, skills that we can put to practice or use when such a demanding situation arises, knowledge that we can bring in, use different heuristics and oracles and try reaching closer to the solution than to stay far away by saying, “Where is the spec, I can’t test without it”. You might want to make a note or mention a risk that a specific document that wasn’t available to you could have helped you go more close to what others might call as “great testing”.

Thats how I learned an important lesson from a rag picker!

Now, you could observe that I was actually supposed to have fun while being with my friend and not think about testing. I did have fun and at the same time I was conscious to see if I have missed an important learning that helped me better my testing. I usually don’t miss the fun nor the learning and hence I hardly get tired.

Here is another story, which I shall connect to the above one, in an interesting way.

A tester reporting to me introduced me to her husband by saying, “This is Pradeep, a passionate tester.” I love to be introduced that way since I am aware that I am one such.

Her husband immediately shot a question: “My friends who are into testing say that bringing testing to their life has caused some serious damages to their relationships, how has it been with you?”

Read my reply to that question, as careful as possible: “Thanks for asking me this question. I am probably one of the most passionate testers and I am sure that those who really know, what testing is, wouldn’t mess up his/her life with it. It appears to me that your friends who claim to know testing, don’t know that because had they known, they wouldn’t have got it into their personal lives”

I have enjoyed some great relationships with people. If you claim to be a good tester, it means testing has taught you not to get testing into relationships although testing happens between relationships and lives in a sub conscious level.

“Testing is questioning a product in order to evaluate it” – James Bach

Now the time for connecting two stories: The first story says, “There is always a thread about testing that I run in my mind while or after having fun or watching TV or chatting with friends” and the second story says, “If you really know what testing means, you wouldn’t bring it into your personal life”.

Connecting both of them I want to mean, I try to look for opportunities to learn from things that are happening around that I can apply for my testing activity for which I am paid and *not* to use testing as an approach to build/spoil relationships or systems that others are wanting to use after my usage. For instance, after withdrawing cash from an ATM, I do not intend to perform some quick tests, because it might stop the system and someone in emergency might have to go looking for another ATM.

You have been seeing testing from my eyes, time for you to see it from your own eyes, and I hope you see it in a better way than mine, teach me something and help me to see something different from your eyes. E-mail me at pradeep.srajan@gmail.com

---

If you have liked this article, you might want to read a different version of the same and here is the link. If you haven't liked this article, maybe you want to read it again :)

Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton