Mixing Scripted and Exploratory Testing

Over the past few days, I have been talking and listening to a lot of testers, leads and managers in Bangalore IT organizations through my Mirchi Test Masala talk and it's a great experience.

I solved interesting problems that boosts my confidence of solving testing problems that Indian IT companies face and I have already started to receive great feedback from testers and managers who attended my talk. I usually don't believe on the feedback forms but I do believe when people talk the change happening in their workplace and that is always a true feedback.

One such problem that I solved for an organization is this:

Manager: I usually ask my testers to focus on finding issues that are important to the release and I see that not everyone in my team finds those bugs that I/management/customer is interested to see with that particular release we make.

Tester: Well, we do follow what he says but he is not being convinced about the work we do. We are finding problems but the problems we find are not of the stake holders interest sometimes.

Manager: There was once a design and code change to optimize performance and we did not have the test cases ready and I had to ask them to do it in an exploratory way but I could see that it did not work well for any of us.

Tester: Thanks that our manager communicated that he is interested to listen more about performance related issues on a specific release and we did exploratory testing but we are not sure why we couldn't find bugs.

Pradeep: Ah! Interesting problem!

This is what I did:

1. First I launched an explanation of my understanding of the problem and asked them, if I understood the problem.
2. On getting a go from both sides, I started to question them individually.
3. The questioning skill plays a vital role, in seeking more information, analyzing the problem and get hold of contexts in which they are speaking.
4. I asked the manager and testers a couple of common and uncommon questions and made them to listen to each other.
5. A specific question that I asked, "What is your definition of performance and exploratory testing?" revealed the solution to the problem for everyone in the room.
6. Interestingly, the manager had a different definition of Exploratory testing from each of the tester. The testers too had their own definitions of Exploratory testing and it differed from each other.
7. I insisted the testers to hereafter ask for what the manager meant by a word/terminology when he assigns them to do a specific task.
8. After that I got them off to a discussion based on a topic over which the manger thought his testers fumbled upon and did a bad job. Interestingly, both ends were now convinced that they could find those important problems quickly since they questioned each other.

Now, there is another problem that testers in all organizations I have been so far and I am going to be for my one day testing exercises workshop will ask my help to solve:

How to mix scripted and exploratory testing?

I understand the fact that Indian IT companies do not want try exploratory testing as a full fledged activity for their projects unless they see results for themselves and they still want to try it out to see if it adds value to their testing.

In one of the projects I worked, I mixed scripted and exploratory testing and the results that the organization saw was amazing. Ah! it worked for me but I can't guarantee that it would work for you but I can guarantee you that it's worth a try.

Before you know how to mix scripted testing and exploratory testing, I suggest you to go through Jon Bach's video on Exploratory Testing and RST appendices . Now, if you crave for more you might want to have a look at Elizabeth Hendrickson's Test heuristic cheat sheet and form something like that to suit your project needs. Of course don't forget to run my screen saver which is now installed and used by all testers in an organization and I am happy to hear that it has been helping them.

Now, it's time to know how to mix it without adding too much cost to the organization and yet find those important problems, quickly.

For every scripted test case you execute, run a test that is not documented in your test case suite, which is fast enough to run and yet not add too much of time. You might want to consider using heuristics for getting on the fly ideas to test the product that you have been asked to.

It's left to you and your skills, to decide the ratio of mixture of Scripted test and Exploratory test and also don't get surprised at the results as you keep practicing this and learn to do it much better.

Rakesh VK, a Senior Tester in DELL is well known within his company for being the tester who finds most number of important problems quickly. While delivering my talk at DELL, I had a chance to talk with him over the secret of being the tester who finds problems that matters the most.

He simply had to say this: "I mix scripted and exploratory testing and I always have been a tester who finds most number of important problems and I enjoy remaining in this position. Also, this has led my manager to get convinced that ET adds great value and he now is happy of other team members doing ET"


Wow! So who is the next Indian tester to become one like Rakesh?

-- Pradeep Soundararajan - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton