Changing a part of the landscape of interviews in India

 http://moolya.com/blog/2012/07/26/thanks-so-much-for-interviewing-with-us/ 

Testability Stories for Agile

You have blood, it has a pressure and it has to be monitored if you want to keep yourself healthy and living long enough. Blood pressure is directly related to heart and its function. As you all maybe aware, most deaths in this world are due to heart attack.

We are so bloody lucky. If we were born in the 17th century and needed a blood pressure test, someone would have wanted to cut open an artery, insert a pipe and then measure how much rise and fall the blood has in that pipe. There is no guarantee we may survive that single test. Since 1855, the situation changed when Vierordt worked on a non invasive method of testing blood pressure. That invention has changed the world. One of the greatest physiologists of the nineteenth century, Johannes Muller said, "the discovery of blood pressure was more important than the discovery of blood". Thankfully, we don't need to be as great physiologist as as Muller was to say that the invention of non invasive way of blood pressure measurement has been critical to saving millions of lives every year.

Not just blood pressure. There were lot of medical tests that were invasive and inventions have helped us find non-invasive ways to test. If that had not happened, every time someone goes to a medical checkup, they would come back with a puncture in the body (if they were still alive)


The introduction of non invasive ways to measure blood pressure and many other measurements related to health are excellent examples of good testability. The MRI, CAT, Doppler, CT scanners are result of humans thinking of good test-ability mechanisms. This has helped doctors to do better and faster diagnosis of health problems and indirectly save lot of lives. Without good test-ability layers in health sector, our average life span would not be as high as it is today. Testability has improved so well that my grand mother uses a device at home every week to test her blood sugar levels, despite it being invasive. In the future, an iPhone or an iPad may be equipped with apps and hardware that may do non invasive tests and auto report to the doc or auto fix an appointment based on the reading.

Testability, is "the not as famous as it should have been, yet a fundamental thing to help in diagnosis". Be it in tests for health or tests for software.

My introduction and experience of testability

I first heard & read about testability 3 years after I became a software tester from James and Michael. Once I learned testability, I found that the root cause for many problems in testing software when investigated branches back to testability. No, this is no obsession or confirmation bias towards testability.


In 2008, I got a call from a company in Bangalore. Somebody had pressed the Panic button in the organization about performance of the product and they could only find me to be the independent tester available to give them a neutral opinion. I hadn't done much work on performance but I took it up. They wanted me from evening to next day morning because they had to make a release decision the next day. They gave me a feeling that I was Tom Cruise of Mission Impossible who has to accept a mission midst of nowhere. I took it up. On the way, I browsed through Scott Barber 's work and called Rahul Verma and asked him if he would be fine to assist me on phone if I were to call him midnight :)

Before I could start testing for performance or thinking about calling Rahul Verma, my opening move was - is the code testable for performance. It was not. Tool integration did seem like a project in itself. Doh! Panic, for sure. Also, it couldn't take 5 users, why load the poor guy with 10,000?

Several years after that, I met a bunch of "performance testers" of large services organization and I probed them on their experience of panic and call for performance testing. They had a similar observation as I had, although they didn't use the word "testability".

You may have read this experience report from me on checking and building check automation. James Bach blogged about it and made specific mention of how he perceived this story to be about testability.

Using versus controlling 


About two weeks ago, we were testing a web application used by just a few hundred million people around the year. We found a bug in production which is of  "Oh my Gosh" types. Fortunately, we had kicked in a video recorder and we captured it. We have the evidence that it exists but we are finding it difficult to reproduce it. No, not that we are not aware of what we did but this particular bug occurs after a specific error from the server. To disprove our own assumption, we need to control the server. In some large organizations there exists an IT infrastructure team that controls the staging environments and testers don't get access to control it. They just use it and are happy just /using/ it. Without being able to control system under test, merely using it won't help test better.

The sin of closing a bug because it can't be reproduced & its link to testablity


Going back to my days of being managed as a tester, in several organizations, my team was asked to close bugs that are not reproducible. I have heard and seen plenty of test and dev managers do that. When I picked up some wisdom in life, I realized how foolish the decision of closing not reproducible bugs was. Some testers defend the decision but are unable to reproduce it. They know they have a problem, unfortunately, they don't know they are facing a challenge of not having test-ability.

Control power to testers

There are products that do not have GUI at all. Not having a GUI shouldn't stop someone from testing the way they would do if it did have a GUI. Of course, we could write code that tests but it is difficult to alter a test when its running through automation. The human brain is wonderful at doing that. I change tests as I see some information and get curious if I would get a bunch of different set of bugs and do a different coverage.

Finding Nemo exercise to help teach testability


Have you tried it? There is an exercise I created to help testers practice reverse engineering or what I thought was reverse engineering. If you want to download the exe directly, here is the link.  There was a hidden lesson about testability in it. I now have enough evidence that most testers who worked on it in my workshop never said they were annoyed by the constraints I had put in or asked me how they could get out of the constraint I had put in.

This, I believe is a small slice of how they work in organizations without asking for testability. I wish every tester was first taught testability even before teaching how to test.


Process Explorer as an example of value of testability


One of the utility I discovered through the help of my student Mohammad on Windows was Process Explorer. This gave me an amazing view of a list of printable strings in a running exe.

If you were to run calculator and then go to process explorer, you would see calc.exe running. When you right click and go to properties, you'd end up with multiple tab window open. One of them contains printable strings in calc.exe.

One of them said, "This operation may take a long time to complete". Wonderful. So, if this is the output I have to get, what should be my test was the question.

I did a bunch of tests to figure out that a factorial had the power to make calc.exe say, "This operation may take a very long time to complete". I then applied Ben Simo's FAILURE mnemonic and found how unhelpful the message was plus perusing through other printable strings, I could do a error message coverage for calc.exe. I found an impressive list of problems, rapidly. It was a wonderful feeling and I wanted it to continue across all projects I test.

I thought it is a luxury to have such visibility and testability tools in all projects for the coverage I wanted to achieve. I was wrong. I recognize it is a basic need. I know how to get it. It doesn't matter if I know, it matters if we all know.


There are ways in which I hope we could solve this problem and here are some of them

Testability stories


Believe it or not, there are two types of Agile teams in this world. The first one in which testers and developers have equal responsibilities to test and work as a team with high collaboration to resolve problems. I have heard and read stories that such teams exist somewhere in the world. Maybe they are in India, too. However, almost all teams I have consulted in the last 2 years that claimed to be that type of Agile, were not. Call me unlucky or call me an expert in consulting teams that pretend to be Agile, it would suit me.

I was wondering if we have a set of testability stories to be built while the product is being developed it would be of great value. As you know and agree, testers are also users of the products, there are no stories written considering testers as users of the product. It always focuses on end users (who are probably customers). Here are some examples of stories I am thinking:

• Testers would like to have this specific error occur in order to be able to perform tests related to recovery from the error
• Testers would like to be able to inject values that violates the front end validation because thats exactly what hackers would do
• Testers would like to integrate a code coverage tool to test what areas of code are they not hitting through the tests they think are great
• Testers would like to have test doubles to test for interaction of our software with third party components whose code is not in our control
• Testers would like to control the time delay in which a certain business process starts running to help achieve time based coverage
• Testers would like to try and add users to db via an excel loaded with values instead of having to key in a set of test users every time.
• Testers would like to have a log of every time someone clicks on the following links to monitor how people seem to be using it
• Testers would like to have statistics shared from the production environment on which pages get maximum hits and what are people searching for mostly and what times have peak traffic
• Testers would like to track all malformed requests that come to the server to monitor any hacking activity or trace of attempts to hack

It could be domain specific, product specific, testing generic and much more. Adam Goucher blogged about what Michael Bolton suggested as Design for Testability and Brian Marick has this wonderful essay on Testability and you would see that we all are thinking in the same direction - to help exploratory testers do better testing, faster and more frequently to help them do better test coverage.

If code needs instrumentation to accommodate testability stories and requests, it should be written in such a way that it does not disturb the sytem under test, can be removed or turned off before putting into production. Worried about this branch going to production - discipline of discipline can help prevent such problems or if you love automation, so be it.

The new idea here is about having identified the need for stories dedicated to "testers as users" of the product. This helps add layer and value to testing. When I say testers here and you belong to the real Agile team, please substitute it by "team" because it is everybody's responsibility to test, isn't it.


Review of this idea


I wrote a first level draft of this and sent to a bunch of people (from the huge set of people I could have sent to) whom I knew would help me out bad set of ideas out in public. It is their feedback that have helped me revise this to its present form and I would like to thank Paul Carvalho, Ben Simo, Rahul Verma, James Bach and Michael Bolton for their review. These people helped me identify the strength and the context in which this idea can be highly valuable. I am sure if they read this published piece, they would realize how different the first draft was compared to this one. That  is how much influence their feedback has had on me and my writing on this one. I have thoroughly enjoyed the journey of thinking about this idea to writing down several drafts, researching and re-reading many articles and getting this to a point where you are done reading it :)

Fletcher Lynd Seagull

I am not Jonathan Livingston Seagull. James Bach is. I am Fletcher Lynd Seagull. At a time when I was asking how do I fly fast and swoop like birds should ideally do, I met Jonathan Livingston Seagull who along with Michael Livingston Seagull and plenty others taught me how to fly and swoop.

They set me free from the crowd and here I am, appearing to most likely do what they did to me.  Not all birds want to fly and swoop, they are just happy sitting in the sun and picking up a fish that pops out of the river. It matters a lot to identify, care and help the birds who want to fly, learn how to do it. Every bird is born with an ability to fly but the kind of flight that birds can do requires tremendous amount of skill which comes out of passion to fly, perseverance towards practice and focus.

The beauty of teaching birds how to fly fast or swoop is, I get to learn equally good things from the bird I try teaching. Every time a bird that claims to have learnt something from me demonstrates its flying skills, I get inspired. That inspiration makes me want to do more.

To contradict my previous paragraph, I don't teach birds how to fly, I just act as a mirror and show them how they appear to be flying now and ask them is that how they wanted to fly. They make the corrections, not me. I am humble but as I fly higher than birds who do not want to fly, I may not seem to be. They are not supposed to be there. They are supposed to be here. The air is thick and cold but the fun here is more.

Some of my colleague birds work with me in Moolya and you know who they are. Some want to take a different path and if I did oppose that, it would be ironical to what I have learnt or teach.

Read an inspiring story of a bird from India, who recently set itself free

Signing off for the moment,

Fletcher Lynd Seagull

Weinberg on more than writing - the Fieldstone method

11th March, 2012 in Bangalore was a beautiful Sunday. Now that it is almost over, I can use the past tense "was". Just the weather making it beautiful is one part but what about the weather within me? I decided to make it beautiful within. Everybody has a pile of books that they bought but never paid its due attention. Weinberg on Writing ( WoW as I want to call now) - The Fieldstone Method was one of the books that hadn't got my time yet. So, I read the first chapter at home, closed the book, went to my wife and asked her if she had any plans for the afternoon. As I had taken her out yesterday, she excused herself out of any plans. Whenever a husband asks "What's your plan?" to his wife, it means, "I have a plan and want to be sure if your plan doesn't disturb mine" :). I took my backpack and scooted to a coffee shop nearby. Coffee, book reading and relaxing was my plan.

About 2 PM, I was all set. I wanted to start with Camomile Tea. I had told my wife that I'd be back only when I am finished with the book, so I was hoping there would be a couple of beverages. I realize now that there is nothing like "finishing the book" because The Fieldstone Method provides an experiential learning to those who want to work with it. Not just that.

I have had the great opportunity to meet with Jerry and spend a couple of days, attending his workshop, being a part of a peer conference that had Jerry, a conference and much more in 2008. I have been a reviewer of Jerry Weinberg's book "The Perfect Software and Other Illusions About Testing". I have read a couple of other Jerry's books that have changed me from time to time. With this book, I felt Jerry sitting next to me and coaching me on the Fieldstone Method. This is no hallucination. I am sure the emotions he had while turning the stones that he used to build this book has rubbed on me and has caused some very good emotions within me. Jerry intends that happens with the experiencers of the book. / OK there is no such word as experiencers in any dictionary /

I haven't published a book yet. If you are thinking I haven't written one yet, you are mistaken. I just haven't published it. I see the light now that will lead me to publishing. That's what I seem to be getting out of The Fieldstone Method because I am going to build my books not write it.

So, why am I telling you all this? To get you buy the book and experience it? To write a review of the book?    You will discover why I am writing about it, if you were to continue reading.

It is all about me. I recognized that I had almost stopped note taking. In other words, I had stopped collecting fieldstones. I guess I was just processing whatever my imperfect memory could store. Isn't it amazing that I stopped doing something that I advocate to other testers - note taking. When I realized this while experiencing the book, I picked my backpack and searched for my Indian version Moleskine and a pen. I must have missed observing a trillion stones but thankfully I have now saved myself from my ignorance that would have led to missing trillion power trillion stones in the future.

Here are some of the stones I collected today that I am pulling from my notebook (and typing it for you):

• Thought: When you are alone in a coffee shop, you hear people and their conversation you usually wouldn't bother to hear if you were not alone.
• Retrospective as I read the book: I was probably smart all this while but not happy because I always wanted to be more smart but never wanted to be happy.
• I shook the coffee table accidentally and a glass of water placed on it started to get its vibration and then I wondered: Pradeep, when was the last time you observed water settle down after being disturbed with a vibration. Your 9 month old kid now would watch it with curiosity. Relation to testing: The first time an information looks interesting and everybody pays attention to it. Once they learn how it happens, they lose interest to observe things that they were once curious about and if there was a different behavior? They would continue to assume they know something. Software is volatile, just like the water. 
• I was asking myself a question: Does it matter if something takes a long time to learn? What determines "long"?
• I saw cold coffee on some table and decided to order it. I wrote a note : How did my choice alter after seeing something that caused my brain to demand it.
• Project Gutenberg quoted in Jerry's book - Awesome

  A friend of mine, Nandan Pujar wanted to meet with me for a consultation and I felt, "What a great time for a good friend to come in. Some of the exercises I want to practice requires such an occasion and a trust worthy friend". I took notes as I was consulting for him:

• I was reminded of Warren Buffet interview of why he didn't come and invest in India prior to last year. His response was, "Nobody invited me to do so". 1.2 billion people didn't think of knocking that door that way. I felt sad for myself.
• My friend Nandan said, "Being with mediocre people helps you identify your delta with them and being with intellects helps you fix the delta". I thought it was a cool thing.
• Made note of words he used "Feudal" and "Artificial scarcity"

 I came back home and made other note of stones I saw, heard, observed, felt, experienced...

• My 9 month old daughter had to poo. After cleaning the poo, I wrote in my note: As a kid I must have not known this is called "poo" and why people clean it off. Now that I know, I clean the mess I do. I am wondering if testers who recognize the "poo" of their work will ever clean the mess they do. Some of them are stuck with it and they seem to think as though it is a newly grown part of their system. Some who come new to the system see people stuck with the "poo" of their work and assume that, to be experienced, they also need to be stuck to "poo"

It is fantastic. I love it.  Thank you Jerry. You helped me recognize the "poo" I was carrying all this while and I am not going to be one of those who will not clean it. I will clean it. I will not just clean the "poo" I have been carrying for a while but identify with the help of trillions of stones I can see, to not carry any type of "poo" of my life. I also recognize the idea of heuristics and they are fallible.

Till yesterday, I was thinking, "Now that I have accomplished all this in life, how do I change myself?", Weinberg on Writing (or WOW) - The Fieldstone Method is a change catalyst. 11th March, 2012 was a beautiful Sunday in Bangalore. So can everyday be if I were to collect the stones.

Conferences & Workshop update : Bangalore, Chennai & Pune

To those in Bangalore, Chennai and Pune, I am here doing some workshops and conferences.

Agile India 2012 - Bangalore : February 17 - 19

First, I am speaking at Agile India 2012. I am presenting a 90 minute workshop on Exploratory Testing the coming Sunday - 19th Feb 2012. There are about 750 registrations so far and this looks very exciting. Kudos to Naresh Jain for pulling things together. Moolya is a sponsor for Agile India 2012 but that is not why I got the speaking slot.

After having consulted for projects that claim to be Agile and seeing people fit scripted testing into a two week sprint, I really want to help them. I also want to help those who are doing exploratory testing, do it better. Am hoping Agile India 2012 would help me do that. Well, of course, I'd end up learning from the participants.

It is going to excited as I am going to be meeting a few people whom I met in Oredev 2011 as well as those with whom I interact online.

Invited talk - Bangalore : February 20 

I am invited by the senior management of a large company to present the Moolya way of testing. As of now, I can't name the company.

Corporate Workshop and Consulting : Pune :  February 23 - 25

My Dear Pune, you keep bringing me there often and I love you. This must be 12/13 workshop in Pune alone for a corporate client of Moolya. I would also be consulting a few teams to help them clear themselves of traps they may be getting into. It's so much Fun. Wanna catch up, drop me a note. 

BugDeBug 2012 - Chennai : March 24 - 25

I am doing a workshop and a talk. This is a conference I personally love because it is affordable for many testers. This is the third time I am invited to BugDeBug and I am probably the only speaker to have been invited all three times this conference has happened. This time it has a great line up of speakers. It has Vipul, Rahul Verma, Ashok T, and Anjan. More speakers to be announced on the website. One of the best line up according to me for India.

The sweet part is, at least half a dozen testers of Moolya are going to attend this conference and participate. Moolya is a silver sponsor for BugDeBug and again that is not why they gave me a speaking slot. 

So catch up with me in Chennai at BugDeBug. BTW, you know who else is doing workshop? Santhosh Tuppad & Rahul Verma on Security and Performance testing respectively. As I said earlier, it is great line up.

Register for my workshop here:  Limited Seats. and for Conference, here, I mean HERE

NDS (News Digital Service)  : Bangalore : March 29

I was recently a keynote speaker at Yahoo QE conference. It was great to have been there. I love the fact that among the goodies they gave, a plant was one of them. It is now growing well in Moolya office. Thank you Yahoo!

Some testers with whom I worked in the past during my employment who were now in Yahoo were surprised to see me as a keynote speaker at Yahoo conference. Apparently they were shocked but the great part was they appreciated what I had done rather than bringing in their ego. Fortunate to have worked with good heart people and unfortunate they didn't discover my blog for several years.

I am starting to call this talk., "The next generation tester" a legendary one :) Somehow, everybody wants this talk. I mock so much at the current generation testers (in as humorous as I can get) in this talk.

So, NDS is having a QC Fair and I am doing a keynote presentation to address their testing community of about 500 people in Bangalore. Fortunately, they have given me 2 hours time that I plan to make best use of to address problems that people may be facing to achieve their testing goals or question the goal itself :)

So, that's about speaking to at least 1000+ testers in a month. Why wouldn't my life be good? Catch up with me anywhere you are. Drop a note. Figure out my mail id. It is not that hard if you are reading my blog. Register for my Public workshop in Chennai. 

How Pradeep teaches software testing - Part 4

If you didn't know how Part 4 came up? Here is how it came. I first wrote Part 1, then Part 2 and then Part 3 ;-) So, logically here is Part 4 of the series How Pradeep teaches software testing.

"I know how to do certain types of testing but if someone asks me to explain what I did, I struggle to explain" was a problem statement I heard from a tester today. I replied in a confident tone, "I know why that happens. It doesn't just appear to be your problem but I know a lot of testers who appear to have the same problem." 

I continued, "You should consciously practice explaining what you did, first to yourself and then to your colleagues although they probably know how and why you did it. Why? I am going to explain how your brain works (or how I think it does). It has two nodes. One that contains what you know and the other that controls your explanation of what you know. When you make a conscious effort, you are forcing a connection between these nodes in your brain. When you force your brain to connect those two nodes too often, at some point it will judge the need for a permanent connection and create it for you. After that you have a free flow of what you know and your explanation of what you know." 

After I said the above, I could see a smile in the face that reflects, "Yes, I now know how to solve this problem". I read a person's understanding not by their head nodding or when I hear, "I get it", but by the emotions and expressions on their face and the body. 

James Bach identified that I was a metacog. I didn't know I was one. After that, it has been very helpful for me to understand why I do things the way I do. I guess I turned myself into a metacog because I thought it suits the kind of testing I wanted to. It's not a special status, it is just a way of life.

I don't even know if the brain works the way I explained it to be but I guess you can understand why the above explanation makes sense. I make sure I tell people that I am not trying to misinform them about anything when I use such examples. I am just helping the tester imagine why there is a problem and how to solve it. A lot of my coaching is consulting.

Examples like the one you read above govern a major part of my coaching. I observe a lot. I practiced consciously making connections between what I have seen, heard, thought, experienced and know to being explain it when the context demands.

Analogies and Examples are powerful approaches to teaching. I need to know what connects to my audience very well. Although all my audience are testers, I can't use the same analogies and examples, it just doesn't work. Bangalore testers need a different example than those in Pune. In Pune, I would talk about Raj Thackery and in Bangalore I would talk about Vattal Nagraj, in Chennai about Goundamani and not about Vattal Nagraj. Now, for those of my readers from United States or Europe, you wouldn't know Goundamani or Vattal Nagraj and hence I would use examples of Chuck Norris, Sarah Palin, Julian Assange. If you are a F1 fan, I'd talk about testing through specific GP incidents. That's how the examples need to adopt based on audience. I also use a lot of examples from what I think the world connects to. For instance, I closely read and watch Air Crash Investigation in Nat Geo channel. OMG! There is so much to learn from the way NTSB (National Transportation Safety Board) deals with it. So, basically as a coach, I have to keep connecting to various thoughts and happenings to be able to connect with my audience. 

Similes, Metaphor or if you choose to call them Analogies are interesting and tough piece of cake, if you are the one providing it. People tend to take it in their own interpretation than what you intend to. It is good in a way, I get to learn when not to use what type of analogies to what kind of people :)

I use Fishing for explaining test coverage. I know a few other people have already used fishing as an example for teaching testing, so I didn't invent it but I know how to explain different concepts of testing with the fishing example. I think that most learning happens when you are interacting with the audience and not when you are explaining. That's where I do better with the fishing analogy.

I start drawing different types of fish, from guppies to clowns and from star fish to sharks. I then draw a size and shape of a specific type of net to catch fish. I give them a goal of catching as many different fish as possible in a limited amount of time. Some audience come back and tell me, "Ah with this net the guppies will escape"... what's your immediate thought? You would tell them to build a net that has smaller squares to catch guppies?  (Why say it? They know it) I would probably be Pradeep and say, "Fantastic. A lot of time spent celebrating the success of finding one good bug steals away the opportunity to find more good bugs. How do you want to celebrate? 3 minutes left."

So, after they come out with strategy and different nets, I tell people that just because they have nets (tools) to catch a specific type of fish it doesn't mean they can really catch a plenty of it. I then tell a story and examples from my life. One of them: I consulted for an organization who had bought an expensive automation tool hoping that they would now be able to find more bugs. They spent all their energy to set up tests on it and found fewer bugs over the year. I drive points like: Tools don't help you unless you know how to help the tools to do what you think they can.

For a while people think it is possible to catch a shark from a net. It is possible, maybe. I explain how a powerful shark can bring their boat down if they try to catch it through a net or a fishing rod and how a harpoon is better than a net. Sometimes people try to think of similar approaches to solve many different problems. The example of shark, net and harpoons are cool for people to relate to something they have done in the past that shouldn't have been done the way they did it. I then equate different types of fish to different quality criteria. I ask my audience what type of fish are they mostly catching and I hear a shout, "Functionality". 

After a lot of back and forth between me and my audience, we all discover what good test coverage could mean. I then start probing into their projects and figure out how much of fish they are missing that they shouldn't be and try to help them understand why they shouldn't be catching too much of the same fish. Sometimes it upsets the food chain :) 

While there is fishing in my class, there is also plenty of room for Sine and Cosine for Test Techniques, Brian Marick's Minefield Analogy for Regression Test Strategy, Tom and Jerry examples for How Scripted Testing is dangerous, what lessons can we learn from Saurav Ganguly's come back, how to read Sachin and Kambli's career graphs, farming... It must be fun to sit in my class. I don't know, I have never been able to.

Happy New Year!

Truth about test plan document & test case document

Not that you don't know.

Truth About Test Plan Documents

• 98% of test plan documents that are created are not updated, maintained or cared beyond sign off.
• The first 5 pages of a test plan document contains history that doesn't interest even those whose names are mentioned in it.
• Scope of testing section is the most funniest part of the whole document. Some times when testers report serious problems, some people cite it as out of scope. Hey, its still in the product.
• Customers worldwide could have saved millions of dollars if their vendors didn't care about creating test plan documents.
• 4 years into a project, nobody knows about the test plan document.
• No matter how stupid or intelligent the test plan document is, testers still write the test case they want to write.
• As an offshore services company, writing test plan documents are a cool way of billing customers without actually doing testing.
• Every stakeholder feels a false sense of achievement the moment they have a test plan document irrespective of whether they actually have a test plan.
• The cost of reviewing a document that nobody is going to use it is really high.
• Those who think they are not ready to test because the test plan document is not ready aren't testers by any means.
• A simple maintainable test plan document is far superior than a detailed test plan document.
• A mind map is worth a thousand good test plan documents.
• Test Plan Document is a Document, not necessarily a test plan.
• The next time you ask a tester to write a test plan where she knows it is not going to be used or maintained, she is not going to put her heart in it.
• Some test plan documents are written in a way that it is obsolete in its first draft itself.
• Some reviewers of test plan documents aim for perfection. More funny stuff, they may not even know what the product is supposed to do.
• Those who know about opportunity cost are likely to write a better test plan document.

Not that you don't know.

Truth About Test Case Documents

• ~ 90% of testers haven't bothered to think why there is a "case" in "test case".
• For most people on earth, a test case means a test idea that is documented.
• The expected results column in test case documents are a copy paste of requirements document / stories. So much money goes into re-writing the requirements document into expected results column.
• If you are already laughing at test case documentation, you may roar to a bigger laughter at trace ability matrix.
• Most traditional testing services projects have 50% of their project duration spent on writing test cases. The team members in such projects complain unavailability of time to "actually" test the product. No wonder.
• Unless the context demands, detailing a test case is a sin.
• Detailed steps in test case documentation provided for humans to execute is something I personally consider as an act against humanity.
• More than 99% ( yeah, more than 99%) testers I have met have passed a test case (or a bunch of them) without actually executing it. It is so f****** boring.
• Test case documents bring more money to countries like India than what Bill Gates must have invested in setting up an office in that country.
• Those testers who don't know to test without test case documentation aren't testers.
• More than 98% of projects I have consulted in India didn't have testers doing "test design". Here is a way: Take the requirement and write at least two or three tests to "check" if that requirement can be marked a Pass. That is all the design that happens.
• Test case documents are actually "Check case" documents.
• If there wasn't check case documents, software testing as an industry would have attracted more talent and helped in building more passion for the craft.
• Businessmen love test case documentation. Testers hate it. Businessmen hire testers to write documentation. Testers trade their time for money, end up writing documentation for money.
• Test case pass percentage is a great way to fool stakeholders. People love to be fooled.
• I personally can write test case documentation for any buggy product and make it look like a bug free product. 
• If all test case documents created so far were printed and burnt, we'd have fire for the next one thousand years.
• If you rate testers based on how many test cases they write per day, you'd always find people who can meet the number you want them to achieve.
• As someone said, "Testing at its best is, sampling". If you start writing and detailing the samples, you will have fewer samples than what you can have and you will never get to know about the product.
• If X test cases documentation takes Y hours, the amount of time spent on reviewing it and getting to sign off is 10Y. So, if X goes to 10X, we have 100Y hours of work spent on test case documentation.
• Some projects have great test case documents and no time to run them all.
• If you do a lot of documentation, you cant ship software, you can definitely ship documents.
• If you are hiring people who need detailed test scripts to test software, your hiring has ton of bugs in it.
• Those business people who ask testers to write "how long will this test case take to execute" and make estimations of test cycle complete time, should be executed.
• It is about Opportunity cost and Opportunity or Cost.
• No user has ever bought a product because the product was developed with lots of test case documentation.
• 99.999999999% of test case documentation I have seen so far doesn't care what the users really want.
• If testers read 1000000 words in a test case document the first time they were executing, they only read 10000 the next time and 1000 that next time and 100 for the next. Later, they don't need it.
• Some people think test plan document = test case document.
• The service most companies sell is test documentation, not testing.
• All good testers I have met so far, treat other testers as intelligent as they are and don't punish humans with detailed test scripts.
• Test cases don't assure repeatability of testing, at best it assures repeatability of testers getting bored.
• Funny that expected result of a test case should ideally be, "Software should go kaboom" BUT it is mostly, "We should see a boat sailing smooth as the day is bright and clear and the waters are not turbulent".

Just that, I know.

Trio Exploratory Testing - Session Notes & Discussions & The Big Dice

Oredev conference was super duper. The people I met and what I could learn from them was amazing. Siggie got a bunch of speakers for the test track and I was one of them. I met some context driven testers and good humans I was longing to meet. Selena was full of energy. Zeger was artful. Siggie has a cool style. I'd vote him for the next James Bond. After lot of talk (beer), talk (err, beer) and talk (and err beer) and little beer, the Kung Fu Panda in me said, "Enough talk, lets test". Update: How did I forget my country cousin, Henrik? OMG. He and I had enough beer one day that I need to recover from the hangover of our talk. What I am attempting to do with Moolya is what he is trying with House of Test. Another update: Another beer made me forget my dear friend Ola Hylten. He took me home and it was made this as one of the nicest trip ever. Miss him.

I got hold of Rikard Edgren and Shmuel Gershon and we decided to do an exploratory testing session. Here is what we did.

Click on the image to enlarge

All three of us enjoyed doing this so much that we hope we can do more of this in all future conferences we meet. What was more interesting was the de-brief and discussions that followed. After we finished testing, Rikard said, "Oh my, you write a lot of notes. I would spend that time to find more issues". That is brilliant but I explained to him about why I do it. It helps me tell a story of how I did things even years later. I pulled a test session I did 6 months back and narrated a story of how the session went. I then mentioned that my style of note taking could be useful in situations of high accountability.

Oh, our Rapid Reporter Shmuel was there. We talked about Rapid Reporter and its use in Session Based Test Management. Shmuel brought shared with us a feedback  : A tester doing scripted testing and using Rapid Reporter found that he was taking notes and found it very useful that he could learn a lot. He said he started making notes of things he used to miss in the past without it.

Rikard shared stories of his testing style. His style appeared to be like one where he didn't want documentation to act as a hindrance to the progress he wanted to make in finding bugs. Well, I think he is right. It shouldn't. For those who have seen how I test, I do the notes part so quick that it doesn't compromise the goal I want to achieve. I felt I was defending my style too much. Yeah, to an extent but then hey it is my style.

We then started to talk about how each one of us adopted SBTM what James and Jon had provided long ago. I talked about how I tried doing it with the style I have seen James doing and how I failed :) It also meant lot of questions from James that I could not answer. The only way I could answer to his questions is to adopt SBTM to my style of testing.

We had an interesting interruptions in between. Some people were constantly checking with us what we are up to. One of us took the job of engaging them in a conversation while other two continued the discussion.

It was one of my nicest experiences to have tested along as good testers as Rikard and Shmuel. I simply love them and am a fan of their work. My sleep is interrupted with the potato and rapid reporter :P. Fortunate for me, I use Black Viper Testing Technique to solve many problems.

Shmuel has come out with idea of BIG EXPLORATORY TESTING DICE inspired by Rapid Software Testing Heuristics and I think this is one of the coolest contributions from Shmuel other than his Rapid Reporter. I have a pair of them on my desk.

 Oh you too see the Cartoon Tester in the background? Hate this guy, he is everywhere in Moolya :)

How to use it? Here is the tutorial : Roll the dice and read what is written on it. Ask yourselves if you have asked questions about what you read. For instance, I roll the dice and see "Support-ability". It reminds me to ask questions about what kind of support-ability is built into the system I am testing. How could someone recover logs from crash? How will the support staff know what the user has done? How will the user know how to explain what happened? So cool.

If I have missed anything interesting, I am hoping Shmuel and Rikard can't resist themselves to add their comments on this post.

Gifting someone a life and they f****** it up

Oh my! I was reminded of this story. It is a hard to digest story for me but I can't just hold myself from telling it.

About two years back, I did work for a client in India and they had to pay me about a couple of thousand dollars. Meanwhile, a fresh college graduate who was trained by some testing institute came to me seeking help in getting a job. He convinced me that I should help him. I was emotionally touched by the way his parents struggled to help him complete his engineering bachelor degree course.

His parents were daily wage farmers in some village in Andhra Pradesh. His parents were getting paid something like 3 to 4 dollars per day and they used to save as much as they could in that money to help this guy study.

He also seemed to express interest in doing some creative style testing and thinking. Convinced that this guy needs an uplift in life, I called up my client and said, "The money you were wanting to pay me is important to me. It would make a small difference to my current life style but God has helped me be happy with what I have right now so I need to ask a favor from you. I know this guy, XXXXX , who needs a job to help his family in basic shape. So if you could give him a job and pay him what you owe me, I'd be more than happy to engage with you again"

The client was very open to the idea but checked with me that I am not making up my mind to gift him life because it comes at a cost. I thought I knew how much it would cost.

With a couple of thousand dollars, you could buy lots of things or be happy having it in bank. My wife and parents were proud of what I did. This guy thanked me for gifting him a great opportunity. I told him that he is being given an opportunity with his parents in mind and he needs to work really hard and learn to do very good testing. He nodded like a doll.

I was constantly checking with my client of this fella's progress. The client said he was good and I was happy that he was doing good. A couple of months later when I tried to check again  I found that this guy had not been performing as good as he used to be. He seemed to be enjoying his life more than learning and improving skills. I wrote to him about it but the response didn't seem to suggest to me that he was serious. We then watched him for a couple more days and I called up my client to ask them to fire him because they didn't do it thinking I'd be offended.

He was fired. He lamented about his mistakes over a call with me but it was too late. I lost my money and possibly the reputation I had built with this client. Its sad that I lost money and faith about people asking my help. It reminded me of the stories I read in Panchathanthra. They are so true even after 5000 years.

Well, it helps me become wiser but makes me nervous to help somebody who brings up the emotional angle. Decisions in life are always based on emotions or the lack of it and they are heuristics.

I am not telling all I have helped have done this to me but even if one does, it creates an imbalance to the way I was dealing about helping people. Passion to foster good testing and good testers continue to guide me take decisions like the one above. I hope I have enough strength and money to pursue my goals despite such cases. You never know how much your decisions are going to cost you. I am not pained at having lost couple of thousand dollars but feel sorry for that guy's parents. The last I heard from that guy, he still wasn't able to find another job. 

Reminds me that sometimes opportunity knocks only once. Other times, we have to create it. If we aren't skilled, there is no way we can create opportunities.

Amen!