How Pradeep teaches software testing - Part 4

If you didn't know how Part 4 came up? Here is how it came. I first wrote Part 1, then Part 2 and then Part 3 ;-) So, logically here is Part 4 of the series How Pradeep teaches software testing.

"I know how to do certain types of testing but if someone asks me to explain what I did, I struggle to explain" was a problem statement I heard from a tester today. I replied in a confident tone, "I know why that happens. It doesn't just appear to be your problem but I know a lot of testers who appear to have the same problem." 

I continued, "You should consciously practice explaining what you did, first to yourself and then to your colleagues although they probably know how and why you did it. Why? I am going to explain how your brain works (or how I think it does). It has two nodes. One that contains what you know and the other that controls your explanation of what you know. When you make a conscious effort, you are forcing a connection between these nodes in your brain. When you force your brain to connect those two nodes too often, at some point it will judge the need for a permanent connection and create it for you. After that you have a free flow of what you know and your explanation of what you know." 

After I said the above, I could see a smile in the face that reflects, "Yes, I now know how to solve this problem". I read a person's understanding not by their head nodding or when I hear, "I get it", but by the emotions and expressions on their face and the body. 

James Bach identified that I was a metacog. I didn't know I was one. After that, it has been very helpful for me to understand why I do things the way I do. I guess I turned myself into a metacog because I thought it suits the kind of testing I wanted to. It's not a special status, it is just a way of life.

I don't even know if the brain works the way I explained it to be but I guess you can understand why the above explanation makes sense. I make sure I tell people that I am not trying to misinform them about anything when I use such examples. I am just helping the tester imagine why there is a problem and how to solve it. A lot of my coaching is consulting.

Examples like the one you read above govern a major part of my coaching. I observe a lot. I practiced consciously making connections between what I have seen, heard, thought, experienced and know to being explain it when the context demands.

Analogies and Examples are powerful approaches to teaching. I need to know what connects to my audience very well. Although all my audience are testers, I can't use the same analogies and examples, it just doesn't work. Bangalore testers need a different example than those in Pune. In Pune, I would talk about Raj Thackery and in Bangalore I would talk about Vattal Nagraj, in Chennai about Goundamani and not about Vattal Nagraj. Now, for those of my readers from United States or Europe, you wouldn't know Goundamani or Vattal Nagraj and hence I would use examples of Chuck Norris, Sarah Palin, Julian Assange. If you are a F1 fan, I'd talk about testing through specific GP incidents. That's how the examples need to adopt based on audience. I also use a lot of examples from what I think the world connects to. For instance, I closely read and watch Air Crash Investigation in Nat Geo channel. OMG! There is so much to learn from the way NTSB (National Transportation Safety Board) deals with it. So, basically as a coach, I have to keep connecting to various thoughts and happenings to be able to connect with my audience. 

Similes, Metaphor or if you choose to call them Analogies are interesting and tough piece of cake, if you are the one providing it. People tend to take it in their own interpretation than what you intend to. It is good in a way, I get to learn when not to use what type of analogies to what kind of people :)

I use Fishing for explaining test coverage. I know a few other people have already used fishing as an example for teaching testing, so I didn't invent it but I know how to explain different concepts of testing with the fishing example. I think that most learning happens when you are interacting with the audience and not when you are explaining. That's where I do better with the fishing analogy.

I start drawing different types of fish, from guppies to clowns and from star fish to sharks. I then draw a size and shape of a specific type of net to catch fish. I give them a goal of catching as many different fish as possible in a limited amount of time. Some audience come back and tell me, "Ah with this net the guppies will escape"... what's your immediate thought? You would tell them to build a net that has smaller squares to catch guppies?  (Why say it? They know it) I would probably be Pradeep and say, "Fantastic. A lot of time spent celebrating the success of finding one good bug steals away the opportunity to find more good bugs. How do you want to celebrate? 3 minutes left."

So, after they come out with strategy and different nets, I tell people that just because they have nets (tools) to catch a specific type of fish it doesn't mean they can really catch a plenty of it. I then tell a story and examples from my life. One of them: I consulted for an organization who had bought an expensive automation tool hoping that they would now be able to find more bugs. They spent all their energy to set up tests on it and found fewer bugs over the year. I drive points like: Tools don't help you unless you know how to help the tools to do what you think they can.

For a while people think it is possible to catch a shark from a net. It is possible, maybe. I explain how a powerful shark can bring their boat down if they try to catch it through a net or a fishing rod and how a harpoon is better than a net. Sometimes people try to think of similar approaches to solve many different problems. The example of shark, net and harpoons are cool for people to relate to something they have done in the past that shouldn't have been done the way they did it. I then equate different types of fish to different quality criteria. I ask my audience what type of fish are they mostly catching and I hear a shout, "Functionality". 

After a lot of back and forth between me and my audience, we all discover what good test coverage could mean. I then start probing into their projects and figure out how much of fish they are missing that they shouldn't be and try to help them understand why they shouldn't be catching too much of the same fish. Sometimes it upsets the food chain :) 

While there is fishing in my class, there is also plenty of room for Sine and Cosine for Test Techniques, Brian Marick's Minefield Analogy for Regression Test Strategy, Tom and Jerry examples for How Scripted Testing is dangerous, what lessons can we learn from Saurav Ganguly's come back, how to read Sachin and Kambli's career graphs, farming... It must be fun to sit in my class. I don't know, I have never been able to.

Happy New Year!

TASTRO - Tester's Astrology by Rrajesh Barde

One of Moolya's customer, a large IT company in India wanted me help a group of testers think beyond the boundaries they had accustomed to. They wanted me to help them bring out the potential and creativity of testers within their organization. Having thousands of testers,  they did know where to start from - the top 30 they had. Top 30 in terms of demonstrated passion for software testing.

The icing on the cake, they wanted their testers to progress towards becoming brainual testers.

A group of 30 testers were introduced to me and I spent time for a couple of weeks with them on various activities. We ended up doing so many good things together that we accomplished the mission together. There were plenty of great work that they did. Now, those 30 have been successful in inspiring 30 more and this chain reaction is appearing to happen. If it goes on, I am sure this organization is going to rock in the coming years.

Somebody impressed with how I mentored these 30 asked me, "Aren't you giving away all secrets of how testers in Moolya test?", to which I replied, "There is no secret. This is how we test and this is how we live". There is nothing to hide. We don't have any secret ingredient or a secret ingredient soup unlike many services companies. We have watched KungFu Panda and hope you too have watched it. We focus on our skills. Our website tells that story.

In this post, I want to highlight the creativity that came out of the exercises of Brainual Testing.
Rrajesh Barde surprised me that he had been reading my blogs ever since I started it and he told me he had also commented on it. I was glad to meet my oldest (well, he's pretty young) blog reader. The only question I asked him was, "Was it worth your time?"

This guy turned out to be hyper creative. He had a sense of humor, lateral thinking, passion to test, leadership and creativity. We were brainstorming of how do we educate testers without letting them know they are educated. Of course, books are boring to most. What content do we feed them with? We discussed on Andy Glover's Cartoons for it. However, that didn't solve the problem of testers within their organization being able to see Andy working with them.

So, Rrajesh Barde in the meeting interrupted, "If I may, I have an idea..." and then came out with this brilliant idea of TASTRO - Testers ASTROlogy. In a country like India, a lot of people refer to Astrology. They at least want to read if there is something good in it for them. I thought that was a brilliant idea and we had to develop it further. We needed to mix fun and pun into it. We needed the learning touch. We needed people to look forward to their weekly TASTRO.

Here is what we got:

TASTRO – Tester’s Astro – What do your test signs foretell?

Aries		Your stars look good for coming week however you might face an environment downtime. Why not make a quick checklist on how to set it up?
Taurus		Avoid calls during Rahu Kaal. Those who have calls with your on-site coordinators during this time, Beware!!
Gemini		The planetary movements suggest that the build scheduled on week day will be delivered to the Testing team on Friday after sunset. You have plenty of time to read Rapid Software Testing Appendix and practice new testing ideas.
Cancer		Though you think you know it, you have no certainty until you try. You might see a surprise appreciation for your smart work. Smart work could mean, you use oracles to test.
Leo		You are worried with unplanned work load. Read the book – Lessons Learned in Software Testing at home and you may see the change in office.
Virgo		Worried why you your productivity appears to have come down? How long has it been since you took a break? Quick breaks between test sessions are important.
Libra		In the busy times, be prepared to work late, eat pizza for dinner at work, and work for some weekends. Don't wanna do that? Go beyond test cases, you will find more bugs.
Scorpio		There is a chance that your relationship with developers would go sour in the coming week. So treat them with chocolates. Developers are rich source of information for a tester.
Sagittarius		Your customers would be under the influence of aggressive Mars. You would be forced to test whatever is thrown at you. Check for the mission to be achieved to avoid falling into traps
Capricorn		You would be trying to achieve the stars by clicking here and there with your monkey paws. Stop doing that and your career could get better.
Aquarius		Your managers would somehow have a strong notion that you have just been marking those test cases as “pass” without executing them. Honesty is important for a good tester.
Pisces		When you have crashed the software and waiting for the system to boot, prepare your own test idea cheat sheet. For those who do, future has been bright.

Isn't this awesome? I feel testers like Rrajesh Barde are a huge boon to our industry. The beauty of my consulting was, I felt there wasn't just one Rrajesh Barde I met but many. I may cover about others in future posts.

A couple of years ago, I used to go to a consulting assignment as though I am superior and I consult people because they were inferior. These days, I go to consulting to get humbled by people like the ones I met.

Please, everybody, stretch out your creativity, you would find an Andy Glover or Rrajesh Barde in you. For those who want to follow Rrajesh's blog, here is the link. He came out with another concept called Bug Burji (Burji is a dish made out of Egg and we call it Egg Burji, Rrajesh made a Bug Burji out of it). Rrajesh, you inspire me. I hope after reading your work, a couple of others may join me in admiring your work and contribution.

I am telling myself that I was born to witness this beginning of the golden era of software testing. Don't know if you can even see what I am experiencing.

My monopoly in teaching exploratory testing in India

This is a conversation I had with a good friend of mine. She is passionate about teaching testing and has changed her teaching style tremendously over the last couple of years.

I have changed a few words and broke down sentences from the conversation to make it more reading friendly. So, Mansi, when you read this and you spot anything that I have goofed up, please correct me.

Mansi: You are a monopoly for teaching exploratory testing in India.

Me: Yes, thank you. ( Of course, MB and JB visit India )

Mansi: How do you feel about it?

Me: I feel bad but also feel good.

Mansi: Good and bad?

Me: No, Bad and then Good.

Mansi: Why would you feel bad?

Me: I feel good because it helps me make money that gets me to feed my family and myself. I feel bad because being an evangelist of exploratory testing in India, I haven't seen people wanting to teach exploratory testing or even if I have seen, they aren't pursuing it.

Mansi: Oh, you know why people don't want to teach exploratory testing in India?

Me: That's what I have been trying to find out.

Mansi: Its because of you!

Me: You mean, I am the blocker for someone teaching exploratory testing in India? That's funny.

Mansi: You have set an entry barrier to them.

Me: What kind of an entry barrier have I set? I have been asking all my students to consider practicing to teach exploratory testing and I am willing to help them out. Why would I set entry barriers?

Mansi: Well, what I mean by entry barrier is, you would intimidate them with a lot of questions on their effectiveness to teach and all that.

Me: If I ask questions about how they plan to teach, why should they get intimidated by that? As a matter of fact, I would ask questions to them to see if I could help them not do the mistakes I did. I may learn to do a new exercise for my class from them.

Mansi: Because you are THE guy for exploratory testing in India.

Me: James Bach would hate to hear that. I am not an authority trying to prevent anyone from doing anything they want to. Look at you, Mansi, your class is filled with hands on exercises these days as opposed to your past of running through slides. I think I have helped you do what you love to.

Mansi: Yeah, thats all there but I am using exercises that I learnt from you for a class on testing. I don't teach exploratory testing per se.

Me: Oh, you mean, you don't teach exploratory testing by using the title exploratory testing?

Mansi: You play with words.

Me: I am just trying to understand.

Mansi: See, this is what I was telling.

Me: I intimidated you?

Mansi: No, your view of things are different.

Me: So, is yours and thats why we are friends and have a lot to talk to each other.

Mansi: OK, so tell me, why haven't testers in India want to teaching exploratory testing despite you doing so many workshops on it?

Me: Maybe because they enjoy doing exploratory testing than teaching it. Maybe they are working on the skills and one of them might start to do it. Or just as you are doing, "Not by that title".

Mansi: ... or maybe they are intimidated?

Me: You are free to think anything but here is what I can tell to all people. James Bach and Michael Bolton didn't teach me how to teach but they did teach me how to learn things. I carefully observed how they coach testers and tried to ape them initially. I must have failed aping them but in that process, I also found my own style of coaching that appears to be working for me. If I had thought that James & Michael has set an entry barrier for me to teach exploratory testing, I would have been stuck with that idea and wouldn't have progressed at all.

Mansi: I agree but its fun to have such conversations with you.

Me: Convert the fun of conversations with me to some action.

_ end of conversation with Mansi_

So, to any Indian tester reading this. I have been enjoying a beautiful monopoly of teaching exploratory testing in India. Look at my workshops and events page, you'd know I must have done lots of them. Going forward, I am going to be doing a lot, too. I think I am going to go out of India this year and do these workshops. I am enjoying this monopoly not because I am a bully and have intimidated people or set an entry barrier for someone nor I am an approving authority. There is no certified exploratory testing coach certification that you should get to be able to do it. I am enjoying this monopoly because of people like you not seizing the opportunities dancing in front of you.

Here's how I started: I announced free 2 hour talk on exploratory testing on my blog and a couple of organizations invited me to do it. I got free practice doing exercises for testers and developers as my audience and in turn I faced a number of questions. After doing enough talks, I got an idea of what would work for me and then graduated to announce my one day workshop. After doing that for enough number of times and most importantly by gaining experience doing one day workshop, I moved to doing two days. I tried doing Rapid Software Testing, Testing Skills Workshop and Exploratory Testing. Found my sweet spot, worked on a few exercises of my own + borrowed some from James & Michael (with their permission) and tried things.

After a few months, my students started contributing exercises to my workshop. I started to recognize the problems that testers and managers face in India to get Exploratory testing mainstream into their projects and started to do "Accountable & Manageable Exploratory Testing workshop". Session Based Test Management got into mainstream of my workshop and this led to me consulting and helping organizations achieve this. Today, I have success stories with Indian testers, managers and most importantly organizations that beat the s*** out of all other testing training done in India. Again, its you, dear Indian testers, who make me feel like that. So, if you see me as a big ego out there, ask yourselves, "how have I contributed to it?". If you don't know the answer, then here it is, "You have contributed more to my success than what I have done to myself" although I appear to take complete credit. Jon Bach wrote a beautiful blog post on the testing moment around the same topic.

On the other side of my analysis is that more than 99% of Indian testers are not career risk takers. Most among them think, they have taken enough risk in their career by choosing testing.

So, someone coming out of their so called permanent job in India wanting to teach exploratory testing or becoming a consultant is so unlikely. It may happen but that would definitely be surprising to me. If you dare, get in touch with me, please.

While you would just continue to read or maybe enjoy my blog posts, I would continue to play a monopoly for exploratory testing in India. What a shame!

Evolving styles & small "e" exploratory testing

When I first heard about the small "a" agile and the big "A" agile, I was intrigued. I was wondering if it really did exist. Later when I interacted with those who were agile and those who claimed to be agile but were not, I started believing in it.

Similarly, I am going to be talking about the small "e" exploratory testing style.

Every tester I have interacted, tested in pairs, watched them test... have their own style to testing while being exploratory. My own style has changed over the years and its evolving. To detail it out, there are several dimensions.

First dimension

• I Did exploratory testing without knowing it was called that way. To...
• Did learn the term exploratory testing but didn't know if that's what I was doing. To...
• Did exploratory testing based on my own ideas of how it should be done. To...
• Did learn that others have their own style to exploratory testing. To...
• Did practice other testers style of exploratory testing combined with my own style. Plus...
• Learnt that to become a decent exploratory tester, I need to develop various kinds of skills. Plus...
• Started to ask for mission, charter, objectives before exploratory testing. Plus...
• Learned about SBTM and practiced exploratory testing with SBTM (and failed at my first trials). Plus...
• Learned to modify SBTM to my style and my context needs (current state). Plus...
• < To be updated >
• ...

Here is the second one.

• Was clicking on every visible GUI stuff to see if a bug dances out. To...
• Thought I was doing good exploratory testing if I was finding a lot of crashes. To...
• Thought good exploratory testing should always result in finding a lot of bugs. To...
• Learnt that bug investigation is purely an exploratory process. To...
• Learnt that test designing skill is important and learnt & practiced it. Plus... (note the plus?)
• Learnt being conscious of the test coverage is important. Plus...
• Learnt that making notes of all my observations and inferences is important. Plus...
• Learnt that briefing and debriefing is not just like any other discussion about testing. Plus...
• Practicing to debrief. Plus
• < To be updated > 

After second, usually its the third dimension

• Thought of exploratory testing as a technique. To...
• Learnt exploratory testing is an approach. Plus...
• Practiced applying several techniques in exploratory approach. Plus..
• < To be updated >

The fourth

• Unable to explain what I did after a few hours of testing. To...
• Being able to verbally explain what I did. To..
• Being able to explain and document what I did and why I did that. To...
• Being able to do that much better over practice. Plus..
• < To be updated >

Then comes the fifth

• Being able to do all that and trying to keep it secret because I wanted to be a hero. To...
• Realizing that I can better only if I share with others. Plus
• Started to coach testers. Plus
• Started to learn from them. Plus
• I recognized that if I had kept it as a secret, I would have been zero and not hero. Plus Plus,
• < To be updated >

Semi finally, the sixth

• Aping what others are saying. Plus
• Trying out my own ideas. To
• To forming my own style of doing it. Plus
• Trying out different attack (not the attack you know) as a starting point to my testing. Plus
• < To be updated >

Se7enth

• Thinking of all exploratory testing sessions as freestyle. To...
• Learnt that there are loosely scripted & checklists types. Plus...
• Practicing loosely scripted, freestyle, checklists types. Plus
• Learnt that a good exploratory tester cannot be determined in the first few hours of testing but what comes after that. Plus
• < to be updated >

Value

If you were wondering why I moved from one style to the other, I want you to be informed that it was for being more value to my customers and not because someone said so. If you are doing anything because someone said so, without knowing if it suits your context or what value it adds, you are most likely doing the big "E" exploratory testing. Also, if you don't know what many serious testers who are practicing to be good in exploratory testing are talking / practicing, you still are doing the big "E" exploratory Testing. If your style of exploratory testing hasn't evolved over the years, you are most likely to be doing the big "E" exploratory testing. Nothing wrong in being the big "E" Elephant. Everybody started there. Everybody were stuck there. Not everybody remained there for a long time.

Also, you may ask if the points listed above indicate anything about the style. In case you did ask, I want to thank you for that because I plan to write about it here. I mean, here --> As we learn and unlearn, the way we test, changes. This influences the style. 

For example, imagine me thinking of all good exploratory testing to be yielding tons of bugs, I'd grow to be a manager who'd question someone why they didn't find a lot of bugs over the last one hour of exploratory testing and hence making myself stupid.

If you were to take another example: Learnt being conscious of the test coverage is important : Everybody in their first few years of testing, at some point, lose the sense of test coverage they are achieving. They get carried away by bugs and as a matter of fact, the client is happy too because they are seeing important bugs being found. Currently my central theme of exploratory testing is test coverage. I ask myself, "OK, I found these bugs and they are great but what is happening to my test coverage". The client may not know to ask for it but it is my duty to help them with that information.

This one, Yo! Being able to explain and document what I did and why I did that. -  I personally know many testers who seem to do good hands on exploratory testing but being unable to narrate a story of what they did, makes them look bad. The senior management appears to just look at the bugs they report and hence the good exploratory tester looks like a "not so good" exploratory tester. Today, I can tell an engaging story of why I did what I did and what value it adds to them and also why I should be given more freedom and hence more responsibility.

Try this! Practicing loosely scripted, freestyle, checklists types. Plus


Based on how accountable I need to be, I choose which type of exploratory testing is needed. That only comes with practice.

Recently, I was asked to explain my test coverage & strategy. Why did that arise in between the consulting assignment? When I started my first round of testing, I was finding bugs as I learnt the application, then I stopped finding bugs because I was doing some confirmatory tests and during that time, the client was wondering, "Is that all about Pradeep?". A very good question to have asked. So, I presented them with my strategy and coverage plan. Here is snippet from that. (Taking out all confidential information). Level 1.


- Toolbar options
- Mouse hover / clicks / right click / left click
- Keyboard shortcuts
- URL / Path editing
- Menu options coverage
- Claims based testing
- Questioning the purpose of existence, position, alignment and meaningfulness of all UI
- Consistency within the application
- Consistency with Windows standards
- Confirmatory testing
- Process monitoring & control
- Security vulnerability
- Consistency of text, font, sizes, options
- Links / Broken links / paths
- Reliability of user login / logout
- Violating policies - Firewall, System, Third party software
- Hidden details & options
- Scenario based tests -
- Boundary tests
- Hardware based scenarios - Interrupt
- Timing based coverage
- Error message coverage
- Tools / Utilities
- Independent components
- Extended usage
- Multiple connect / disconnect
- Reboot / Sleep  / Hibernate
- Close abrupt / Kill
- User coverage
- Usability standards comparison
- Functional checking
- Supportability testing
- Accessibility
- Navigating between zones / desktops / sessions
- Platforms
- Devices coverage - Bluetooth, Printer, Memory, Ext HDD
- Authentication / Authorization
- Splash screens
- Obvious / non Obvious things for a user
- Brand image versus product behavior
- Comparable products / benchmarking
-  Testability tour
-  Behavior patterns, user expectations and consistency
_snip_


At Level 2, I had a checklist of what tests I did.

Here is an example: http://support.microsoft.com/kb/301583 : I ran through each one of them. I could tell what shortcuts cause what problems and an investigation about the problems. They were highly convinced why they were paying me as much as they did. I just stood up to the scrutiny.

At last, Trying out different attack (not the attack you know) as a starting point to my testing. Plus


Most testers use Functionality / Security as their first few possible ways to start attacking (start testing) a product. I was practicing to try out different things trying to determine how useful it is for short and long term projects. Testability attack has yielded great results for me than any other form of attack. Now, I might have learnt a lesson that the world already knows but what matters is, I did learn it. Hurrah!


If you were to determine how skilled an exploratory tester is, its not the first few hours, its what comes after the first few hours. That's what determines the the big "E", exploratory testing style and the small "e" exploratory testing style.

Also, our testing industry is so damn confused that they might end up asking you about the big E and small e exploratory testing in interviews by those who might happen to read this blog post half way and run to a meeting. Tell them that Pradeep used the small e and big E to explain styles and not to define or redefine anything about exploratory testing. Also, tell them that they didn't read this post completely. 

Future of Indian software testing looks safe

I have spoken bad about testers in India in the past. I received an applause for it from Americans, Europeans & even Indians. I didn't consider it a sin. There is an equal (no, wait, "much more") applause if an American was complaining about testers in India being bad in testing. I just didn't mind that. That was a few years ago.


2 weeks ago, I had an opportunity to speak at STC2010 conference in Bangalore. The turnout was awesome. About 400+ people. For the first time, I witnessed latecomers looking out for chairs that were vacant. You could simply call it house full.


Critical thinking demonstration


For me, software testing conferences in India is more about meeting interesting people (who don't have a public presence, yet do lots of good work) and less about the keynote speakers or their content. Some talks, in my humble-less opinion were pretty bad but that's why the good ones amidst them were shining and bright. 


Ajay Jain, from Adobe, Noida in particular gave a talk that I stood up and clapped for what he said. He said things like, "If something takes time, the immediate instinct is to automate but how about letting certain things take its own sweet time because there could be a value in it."


There appears to be hungry vultures waiting to find an opportunity to automate something that is taking time and amidst those people, Ajay Jain, is a star because he is thinking critical. I didn't know him before that talk. Later, I went up to him and talked a good deal. There are many undiscovered Ajay Jain's in India.


ET & SBTM experience reports


Moving to a poster presentation by Shaham Yusuf and his lead Vivekanand Suman from Delloite, Mumbai where they published an experience report of exploratory testing & session based test management. They talked about how their exploratory testing influenced the developers and in turn how its doing good to their product. The most important thing about these guys is that they had sought permission from their organization to present some of their actual reports in a conference and allowed more people to know that exploratory testing and session based test management is put in use at large organizations and in large projects. 


Heading the certification campaign by opposing it


I then picked up a conversation with a tester at the conference lobby on certifications. She works for a large organization and is responsible for certifications in her organization other than her usual work to find bugs and report them. I assumed she was certified too but it turned out that she wasn't. She doesn't believe in certifications and had the guts to say, "Don't enforce a certification on me. You want me to be a good tester, I can prove it to you at work. You are welcome anytime to my project and I am willing to answer your questions about my work" to the senior management. When I was thinking that I know of all testers in India who are as bold as me, she proved me wrong. 


So, how is she leading the certification responsibility? She is helping people choose a certification that suits their mindset. When she identifies a tester wanting to improve the skill and not for the sake of getting one, she is making those people aware of BBST course. She said, "If I didn't take up the responsibility of leading the certifications group, then I am not sure if the other person would have suggested BBST for a few to whom I did". 


Fantastic. We would imagine girls in India to be the shy types and say, "I want a 9 to 5 job. Got to take care of my in-laws" but then someone like this (and of course Parimala, Meeta, Jassi, Krishnaveni...) are a blessing to India and its future in software testing.


Testing in Testing Institute, not Certifications


Then I met a person who is running a testing institute. I had perused his website sometime back and after seeing ads on certification, I thought, "Yet another testing institute wooing people with certification" but talking to him changed that perception. He said, "Ah, you believe everything that is there on a website? Shouldn't be doing that" and continued, "Our institute focuses on trying to help people develop testing and thinking skills. We don't stress on certifications or their content but still if they want, we don't say No".


These people are like soldiers in the border of your country guarding you, whose names you don't know. In the above two cases, I feel, you shouldn't know their names. They are doing a fantastic job. I think they should come out and speak in public what they spoke to me after they have achieved some more great success.


For more...


Our own test automation power


I was glad to meet Narayan Raman, the developer and product owner of Sahi, a web application testing tool. We had met more than a couple of times in the past. He was a very special invitee at Google Test Automation Conference 2010. Over the last two meetings, I realize how much important is Narayan Raman for India. He has the zeal, skill and enthusiasm to put India on a higher scale. He is giving a run for tools like Selenium and with his tool starting to support Flex from next couple of months, I think Sahi is a rock star. Check out the comparison between Sahi and Selenium 


Weekend Testing & Weeknight Testing


If you don't know about Weekend Testing, you should visit http://weekendtesting.com and spend enough time there to learn this big revolution started in India and now the whole world seems to be catching up. Americans are extremely happy & excited of having their own chapter. Europeans are enjoying it. It was also featured in Eurostar and got a standing ovation. James Bach had a dream of seeing Weekend Testing becoming Weekday Testing. His dream got closer to reality during London Testers Gathering where Mike Scott proposed the idea and Sharath joined the bandwagon with some other good people to kick off Weeknight Testing. I didn't believe till then that it's only in United Kingdom that (K)nighthood is bestowed to people. The knights didn't wait for the queen though.


Peer conferences


Bangalore Workshop on Software Testing is a peer conference inspired by my own experience with Toronto Workshop on Software Testing. We have been running it over the last two years and the next one is coming up in Feb / March 2011. We are planning 2 days instead of just one by looking at how many more people want to join and how much they are enjoying it. 


Exploratory Testing & Rapid Software Testing


The interest for Exploratory Testing & Rapid Software Testing has grown to a great extent in the last few years. I myself have trained about 1000 testers on it ever since I started doing Exploratory & Rapid Software Testing Workshops. You are seeing a lot more testers demanding freedom and ready to take up that additional responsibility that accompanies freedom because they are working on their skills.


Hands on Testing Coaching for College Graduates


I don't know if you have gone through this report on excerpts of work done by participants of hands on testing training. There were businessmen in India who volunteered to allow me to experiment a complete hands on testing training with hardly one hundred slides for one month of training.


The outcome of of doing this with one batch is, we have Santhosh Tuppad as the multiple bug battle competitions winner who is giving a run for other country testers (and even other testers from India) a real hard run to be able to win the bug battles he is competing. The other people who chose not to write and work as public as Santhosh Tuppad are doing excellent and their employers are way too happy to pay them well.


Good blogs from India


When you were thinking a lot of testing blogs from India are horrible copy paste and plagiarized stuff, you also saw the rise of some good bloggers. Parimala Shankaraiah stands as one who started blogging less than two years ago and people like Lisa Crispin who is the author of the book Agile Testing and has been writing for long, considers Parimala as her  hero. We have many other testers like Dhanasekar, Nandagopal, Vipul Kocher, Rahul Verma, Ajoy Singha, Santhosh Tuppad... joining to the band of good testing bloggers. The last few recently started blogs never had a copy paste but original content. Testers have started to write down their experiences. 


So, So, So, So, So, So, So, So, So?


A couple of years ago, if you heard someone talk bad things about all testers in India and you laughed at what they said because you thought they were speaking truth, you did the right thing. We were bad.


Henceforth, dear other country folks, if you hear someone talk bad about all Indian testers, I still encourage you to laugh but for a different reason that they don't know or are ignorant about what is really happening here.

A note to Indian testers: On a second thought, I wouldn't encourage you to laugh at them if they are from America or Europe or elsewhere because some Heads of Testing in India themselves don't know about all these. Silently giggle if you are working in one such company and get on because you are the future. Ensure, the future generations don't giggle at you because you are not going to know that even if they did. Is your Head of Testing aware you giggled right now? If giggling isn't your types, then go educate them.


Remember, you are the future and work even more harder and smarter. 


Jai Hind!

Experience report of testing versus checking

Many thanks to my client in Bangalore who encouraged me to blog about the experience, value and outcomes we had in separating tests and checks. If you are reading this, it means my client has approved this for publishing.  

Thanks to Michael Bolton who posted a series of posts on his blog under Testing versus Checking and those who commented on the posts, whose ideas and thoughts helped me to think about delivering the value of tests and checks to my clients. Tests and checks series of posts has helped me communicate things better and organize my thoughts better. We were using the term "check" and "test" much before Michael blogged about it; however, he gave a cognitive structure to it in our brains.

The context

One fine evening, there came a client looking for me and the story goes like this... 

I was hired for a couple of weeks, to consult and test various products produced by a large organization whose headquarters is in Europe. The moment I was hired, there was a need of a hand to run through a few tests on a product. The product was used for analysis of something across many country deployments. It has an engine loaded with business rules, lots of history data in a database and report viewer application. How do we know if the business rule engine didn't violate any rule? The reports had to comply with complex business rules that at the first sight appeared to me as contradicting one another. The core development of the product was being done by a vendor in the US of A, who was reputed within my client's office for pumping in poor builds, as frequent as they could. I wish I could tell you more than that.

Video test cases instead of lame text

On starting off, I realized that the learning curve on the project appeared to be steep and I couldn't cope with it in the time I had. A demo was given to me from a colleague who had been running tests ever since he was hired. Does a demo suffice? No way. 

Building a little credibility with that colleague helped me convince him to record videos of the demo and tests with the help of Windows Media Encoder 9.0 for Windows XP. So rather than following dumb scripted tests, I was watching a video on one monitor and running tests on the other. Video test cases were more reliable than documented ones. Based on how quickly I picked up things, the videos are now being planned as training material instead of asking someone to go through test cases. 

Recognizing checks being called as tests

On running them, I realized, what I had been running were checks than tests. To quote Michael Bolton, Checks are machine dependable and tests require sapience. I was checking if the report complied with all business rules. While I was checking, I instantly knew that these checks could be automated.

Testability, Test Coverage Issues & Stakeholder Interests

I noticed a couple of bugs in the software were preventing me to run other kinds of tests as fast as I thought I could. Anything that blocks me or slows down my testing is a serious issue for me and to those who have hired me. With the proposal to test for various quality criteria being accepted, I made a presentation to the stakeholders of how the quality can be improved at least with respect to testability and other quality criteria (such as Usability, Performance, Security.. ) accompanied by a test report.

They instantly liked the improvement of testability idea because to check if the business rules are not violated for every installation, they had been taking 2 - 3 days. They needed rapid feedback to take better decisions.

Imagine looking for a specific value across many excel sheets. By the time you are on the fifth, you might almost forget what you saw on the third and where you saw it. However other kinds of quality criteria didn't matter to the stakeholders for their current context. I wish they had considered it but that's a business decision. My bug advocacy was strong though. I had stakeholders wanting to automate the checking part of our testing. That was a winner.

Partnering with developers

I was sitting in an organization whose policies and people are new to me. I was proposing things that required lot of dependencies on other people in functions such as development, business analysts and on-site personnel with of course the management. The time frame I required to achieve this was really short. Writing emails in a manner that helped the people realize the importance of the value was a big winner for me. For instance, here is an email I sent to the development team that sits in my client's location who mostly do some integration and writing wrappers.

Dear Developers,

Greetings!

I am glad to be interacting with you. As we are committed to reduce the time we take to analyze a release of XXX XXX, we realize that a tremendous saving to the organization can be contributed by making a few changes. We have made a few things from our end, such as, having a formula do the job for learning about XXX-XXX rule being satisfied for all installations. Creating a repository of possible combination of tests we need to do to be able to say we have exercised the business rules very well.

In that path, we realize that with your help, we can do a lot more. I understand that you might be busy with other things on the table but as I feel we are working together towards a common goal, it makes me request you to help us.

We request the following things from you:

• Currently we are spending about 3 – 4 hours manually to verify the XXXXXX business case for a small installation.  
• We have investigated and conjecture that if you help us have the XXXXXX ( such as XXX or XXX details ) to be available in a list associated with corresponding XXXXX and the XXXXX, it would take us less than 5 minutes for us to perform the business rule validation.  
• If these details are a part of the export of Weekly Report (highly preferable) or as a separate excel list (less preferable) it would help us achieve the goal.  
• Assuming a set of testers will be testing XXXXXX for all country implementation in future, you would be helping us bring down the cost of testing by a couple of thousand dollars ( Well, Euros, as we are European based )  
• Additionally, as we are doing it manually, we are not exceeding more than 10 categories per installation. With your help we would have the capability to go as much as the number of categories the system can support. Thereby, we test for larger samples and increase our data coverage.

We would be in a position to provide you any help you might need from us in achieving this task or would be glad to offer back any other help that you may require to make your work more effective.

Thanks & Regards,

-- Pradeep Soundararajan

30 minutes, no response to the email. At the 31st minute, a developer was at our desk. It happens that he was serving his last day for the organization and carved out time for us. I considered that as a great gesture by a developer to support test team and our test team brought a chocolate for the help he offered. I couldn't get a "Thank you" card; else I would have wished to have given that as well.


That developer helped us with stored procedures and SQL queries that we need to extract data out of the db. 


The Checker Tool Development & Testing

So, instead of looking through reports exported through the GUI, we now had the option to extract values directly from the database. We got one step closer, to what we wanted to achieve.

While I was planning to write a Perl script to extract the values and dump it to an excel sheet, I became aware of a developer in test who joined the organization and had not yet been assigned to a project. The good news for me was she hadn't got her computer to start working. I requested the manager to get her time for helping us write a script and gave her my PC for 2 days while I mostly roamed around saying "Hi" to people around and getting a hang of the culture and talking to people about their work. I could have done all by myself using Perl but I know how slow I'd be as compared to a developer in test.

I provided her with requirements and all necessary tools that she'd need to develop and test the script that could dump the values in an excel sheet in a format we wanted. It's a complex data structure and in order to perform an analysis we had to develop a report format. With the help of my colleague, we zeroed in on the report format and in 2 days time, she could help us see what we wanted in the report format.


TDD for writing Excel Formulas

Off to validating business rules through Formulas in Excel. I knew the power of Excel and its formulas but I didn't know its limitations. I remembered that I had received an email long back from someone containing  MS Excel formula dictionary as an attachment. I pulled that out to help myself.

Excel shouted a warning and error if we exceeded 7 nested loops. It didn't warn us when we had a typo in one of the formula, a star * symbol instead of 8. Copy pasting formulas from one sheet to another had reference to the previous sheet. I know lot more about excel than what I knew earlier. 


I wasn't doing a good job and I knew it very well. A month and a half back I attended a demo on TDD from Venkat Subramaniam. It struck me at the right time. Instead of continuing to do a mediocre job, I started collecting data, scenarios and information that should make the formula fail or pass. I started to write the excel formulas based on the data we had. My colleague who had been in this project for a while helped me get the set of data that had to result in pass and those that should result in fail. If you may permit me to say that writing formulas as, "coding"  then we were pair coding.


This was very helpful. After starting with that approach, we could find bugs in previously written code. Prior to the TDD and paid coding approach, I wrote a formula, loaded the data and checked to see if it failed or passed. For a condition in which it should pass, if it failed, I used to tweak the formula to see a Pass. That was dumb, especially, after I knew that making a Pass could impact other things that were supposed to fail. 


With TDD in, we achieved faster progress on our business validation formulas. We were now at a speed lane. We cracked the puzzle too. We could check for business rules violation in a matter of few seconds.


Comparing humans and tools for speed

I am against the idea of comparing human testers with test automation. I still am against the idea but I saw myself in a situation where I was comparing automation with manual effort. Why did I do this? Should I remain silent about what I have done and continue telling people who attend my workshop that; comparing a human and a tool for speed is a bad idea? I was reminded of James Bach's blog post "Manual tests cannot be automated" and Jonathan Kohl's interview where they put a lot of things so well that I couldn't have put it then.

In self retrospection, a learning emerged. We didn't automate tests, we automated checks. As humans who were not supposed to be doing checks were doing it, I was forced to make that comparison. In fact, I should have said, "Please do not see this as a faster way to do things by comparing how testers were doing it earlier. They weren't even supposed to be doing that. I just wish there was more freedom and a little bit more authority for them to express it".
  
Getting our tool tested by domain experts

"Yipee! It works". That's not what we thought. When we developed this tool, we were sure that the users of this tool could be beyond the test team and how good our tool was to be decided by a "domain expert". When we showed some of our unit test results to a business analyst in house, he was excited at how simple this would be for him to use when he goes to assess a country installation in the future. He vowed to support all that we needed. He offered help writing us a few formulas that we needed. I tweaked it to suit our needs. 


He had a logic that I wouldn't have touched upon. So, that really helped. We didn't need to be a domain expert all by ourselves to be able to do this. A testing brain and a domain expert brain coupled together yielded a lot of value.

Added to that, he was a key stakeholder we had in house. He acted like a customer in house. He suggested the changes we needed to make it more suitable to other stakeholders' usage. Having been on site, he could bring the perspectives of people who could use this tool and their contexts. We needed lot of clarification on the business rules. Imagine a set of rules contradicting each other but still plausible under certain contexts which had to be shown as pass and not fail. Whoof! I liked it :)




Live Testing of Business Rules Validations Checker Tool V 1.0

So, our in house customer was happy with it. We showed a demo to the test management and they were excited. The test manager was excited enough that he pulled out the product manager out of his busy schedule to show our tool to him. The product manager was about to be in a meeting and he asked us, "How long this would take?", because he knew the business rules was taking a few days. When he asked us that question, we initiated the script, took a deep breath and said, "It's over". We showed him the report and there was a smile that our test manager was hoping to see. The only skepticism he had is about business analyst's approval of this tool. A bug in this tool would really misinform the decision makers.

So, the tool was shipped to people in Europe who tested it on the live system. They took about 2 days to get back. It wasn't a very nervous moment at all for me. I was just impatient. They had to take two days to approve this tool because they had to compare the results of the tool against their manual checks. While we were waiting for the reply, we continued testing our tool to see if we could spot problems. We ran it for several test installations and we made notes on how we could improve but no bugs found. 


The climax

On the next day when I was traveling to my client's location by bus, I get a call from a person who heads the testing for the group which I was hired. "Are you in the bus? Can you get down? I want to talk to you. I will come there soon". Before I could ask anything he disconnected the call.

That's really crazy. I suspected something must have gone wrong. I thought the Head of Testing wants to give me a bad news that my consulting contracted is terminated. He arrived at the place, I got in, and he showed me an email and said, "Congratulations man. This is great news". The tool was approved by the business analysts who were on site in country installations. They said it was accurate and also proposed some modifications to a dashboard we had created in the excel. It hit the target on the bulls eye. 


The Head of Testing and I together prepared a presentation of the whole story to circulate it throughout the organization. The Head of Testing currently hopes the concept of separating tests and checks is carried over across all other projects. 

At one side I was extremely happy for being such a value to my client and on the other hand, I was telling myself, "Consultants like me are made to look good for doing what everyone else is supposed to be doing". Had my colleagues at the client location been people who were active in learning, I wouldn't have sounded like an expert tester to them. I just feel pity and can offer the help I can. I have to move on and learn more things.


Preventing bad builds from sneaking in

The story doesn't end there. I had another proposal up my sleeve. I wanted to ship this tool to the vendor in US who was pumping in bad builds. We are planning to use the Business Rules Validation Checker Tool as acceptance checking of a build into the client's location. The idea of automated acceptance checking appears to be interesting to my client since they work with several vendors, delivering broken builds not as frequent as the vendor stated above :)


Return on Investment, in its true sense.

The ROI is not about the speed with which a certain task is done. The actual ROI is that the testing team time is now freed from checks to do exploratory testing, bettering the test coverage, faster feedback, better informed decisions, focusing on things that they were not focusing on AND preventing bad builds from entering the client's location.

I thank my colleagues (nicknamed) Rag and Tham for the support they offered me in the three weeks. Without a management providing freedom to a tester like me, this wouldn't have happened. So, thank you Mr. Head of Testing, Test Managers and Business Analysts for all the support and for approving this blog post.


The only thing pending is a party to celebrate. Mr. Head of Testing, are you reading this? :-)