How Pradeep teaches software testing - Part 3

I hope you have read Part 1 and Part 2 of the series of "How Pradeep teaches software testing" and I welcome you to the Part 3. I don't yet know how many parts it is going to take for me to complete this series. I am hoping that I would write down the most important parts and keep adding whenever I need to update. In this part, I am going to focus on my journey of doing exercises and hands on stuff I do at my workshops.

Having been a guinea pig of James Bach and Michael Bolton's online coaching, I myself went through quite a few exercises that were under experimentation. I still remember James showing me a set of pens and then hiding all of them away from me but one to ask me which one was being shown to me. All of them looked identical so it was difficult to tell which pen was he showing me. I had to figure out a way to question him to help me identify the difference between those pens. This is one such exercise that didn't make it to the Rapid Software Testing class. However, I benefited from all of them.

When I started to teach my own version of Rapid Software Testing in India, I sought permission to use a few exercises from the original version. I got permitted to use the Mysterious Spherical Ball and Dice Game. Oh, the Triangle, too. I needed more and I had to create my own.

Even before that, I had to modify the exercises to suit me and the point I wanted to drive. I did that. I have a few variations of the game and exercises and at times I drive a different point from the original one. If I were to have been a parrot repeating what they taught me, I wouldn't have been able to survive or inspire people.

I tried my own exercises. I tried them out with James & Michael. It was hard to teach them back something with my exercises. That made my exercises or my ability to deal with them stronger. I then tried it out on a couple of testers here in India and it worked wonders. So, I consciously didn't take all of my exercises to James & Michael. Not that I wanted to avoid them but as their good student, I wanted to appreciate their time. I first wanted to get good at something before asking them to work on it.

I published some exercises on my blog. For instance the telephone puzzle and other brainstorming exercises helped me to experiment some of my own.

Every time I did the same exercise with new batch of testers, a new idea or an approach used to emerge that used to teach me a lot. I started to focus on what testers in India need to learn. If their fundamental is flawed then it is not good to teach them some things that appear to be Greek and Latin.

I made a list of things that I think is fundamental and started to work towards exercises for the same.

I am listing a few of them below

Skills

• Observation
• Questioning
• Lateral Thinking
• Reverse Engineering
• Scripting
• Investigation

Technical

• Test coverage
• Testability
• Mission focus
• Heuristics & Oracles
• Bug Reporting

I built exercises for each category I wanted testers to get good at. As an example, I built an exercise for reverse engineering practice for testers: Finding Nemo  . Sorry to Mac folks unless they have a Windows emulator in it.

I wondered if I could really help in creating good testers with my crazy set of exercises and ideas. I was consulting for Edista Testing Institute and sought an opportunity to experiment my crazy ideas with two batches of fresh college graduates. The results were beyond imagination. I could bet on these testers against all of the ISTQB passed fresh college graduates put together. Here are excerpts of the work that they produced after a month of training from me. 

I was all the more convinced about creating my own exercises to help creating good testers in India. As an evidence of how skilled a tester could get beyond those 30 days is here - Santhosh Tuppad, my student of the fresh college graduate training is now a co-founder of Moolya Software Testing Private Limited. Not just he, other testers from those batches are top testers in the organization they are working for. There are a few in them who haven't yet made it large but if you talk to them you'd know it may happen, if not today, tomorrow.

He is the youngest testing entrepreneur to the best of my knowledge at the age of 23 and this story being created in India and me playing a small role in it makes me happy of the path I am heading towards in coaching software testers. I am specifically going to write about my students and their journey after attending my training in a part dedicated to them.

The exercises created curiosity in them to learn more. So, I didn't do the learning for them, they did it for themselves. What every training for a software tester needs to do - is to create curiosity with pointers of how they could do it. What ISTQB is doing is a super reverse of that. They damage the gene when it is being built and create business opportunities to themselves in the context of helping such genes upgrade and repair.

Some of the ISTQB trainers in India who perceive that my work has an influence on them, use some of my exercises in their workshop. I allow them to do so because that's the best hope for me that someone would then question the value of what is being taught as ISTQB and get curious to learn about testing.

My exercises teach people to test their own ideas of testing. I'd like to build thousands of them and give it away. Over the last few years, I have seen lot of action from Context Driven Testing folks on the exercises. People come up with their own exercises and share it with others. It is the safest community to be in irrespective of whether you agree to the principles or not. People like Sebi, Markus Gaertner, Matt Heusser are the ones on top of my head who contribute testing exercises to the world.

You will have fun cracking my Finding Nemo exercise. You would trick yourself to believing you have cracked it and then if you do a few more tests, you'd discover you haven't. Santhosh and I worked on something called Guess the Password - Version 1 & Version 2 . Don't go to Version 2.0 before completing Version 1.0.

This is what I am doing in India. This is how I coach testers. The future is all about such testing exercises, if it were to be a bright one. So, all of India isn't all that bad in testing as you may be imagining it to be. Note that!

In future parts of the series, I am going to be covering on aspects of my interaction with testers in the class, humor that works for me in my class, feedback and what I did with it, interacting with trainers in software testing and lots more. Stay tuned, it looks to be completely safe.

Story of how a hang becomes a crash (because testers love reporting crashes)

I am doing my first exploratory testing workshop for 2011 and first from Moolya Software Testing Pvt Ltd. and I am excited about it. You can find the details by clicking on this link . If you are in India and plan to attend it, please do it quickly. I usually don't take a lot of participants although I know I'd gain more money out of more participants. Ever since I announced it yesterday, there are 4 seats that got filled within 5 hours.


Now, I am going to tell you a story that I had been hiding for quite a while. Not intentionally hiding but was planning to blog about it and the time is right now :)


I provide many applications to test in my workshop. At the end of a testing session, I check with the participants if they found any crashes with the application. Not to be surprised, a room full of testers, they do report crashes.


I ask them to reproduce it with me watching what they do and here is a shocking information; most of what they report as crash weren't crashes at all. What you are about to read is also a great example of confirmation bias that I have witnessed.


Here is what they are doing: They are performing an input constraint attack or providing a large set of characters as an input to a field and hits the "Submit" button. They wait for 10 seconds to see what the application does and on seeing the application in not responding state, kill the application and report that as a crash. 


Awesome! Isn't it?


Now, I got conscious of the fact that testers seem to be reporting a hang as a crash and was keen in looking at live projects during my consulting assignments. I get access to the bug tracking system during my consulting (wow) and I see patterns of such reports. 


I go filter out some of the crash reports from the bug tracking system and try to attempt what the tester did to report that type of a crash and bang, that's a hang.


I have made this point to testers who attend my workshop in order to help them be more conscious of what they are seeing versus what they are reporting versus what they are eager to report versus what they should be reporting. 


It seems to me that testers have an anxiety to report crashes. There's nothing wrong in it but it goes horribly wrong when you report a crash by fooling yourselves and the people around you. I recently blogged about the Obsessive Checking if being mentioned disorder that I was suffering from and here is a relevant excerpt from it, "Months later, I started replacing every word by my name till I saw my name on others post. So, I may have read a few posts without actually learning anything from it because all I saw is "Pradeep" & "Tester Tested" on those posts."


Now, why is that relevant to this post? I see that the testers I have witnessed who report hangs as crashes also appear to have a similar problem of obsession towards reporting crashes that they appear to not see a hang but see a crash. I would love if you ask if these testers ever report a hang? Yes, they do. If an application recovers before they kill, that's a hang.


I understand that an application might have hanged because it has crashed but how do you know? Oh yeah, allowing the GUI to fool you and me?


Here is an excerpt ( and tweaked to hide confidential information ) from an issue that I reported on Sep 30, 2010 @ 12: 13 PM IST (thank you Jira) while testing a new OS that is soon to be launched. No, not Chrome OS.


"I opened Media Player application and made an attempt to subscribe to a podcast. 
The application appeared to hang and it also did not allow me to close using the X mark on top right corner.


In an attempt to kill it, I navigated to XXXXX (product name masked) and tried closing it from there. However, the XXXX is open and Media player still seems to be remaining in the hung state.


From that point, I might be forced to reboot the system to get out of it or can escape by waiting for a longer time that I waited for Media Player to recover (>5 mins). No user to my knowledge would wait for more than 5 mins to see the Media Player function. Even a reboot appears to be a faster option" 

Now, if that was the Description, here is the summary :

"Unable to kill a XXXX when the application in it appears to hang or is busy"

• Why I posted the above? Is it because I wanted people to note the usage of the word "appears"?
• Why did I make a few words in it bold? Is it to highlight it?

Long ago, well, not so long ago, I blogged about how teaching testing is helping me test better. The above post and the bug report I posted, is another example of how teaching testers has helped me do it a lil better.


Now if you are going to be sitting at my workshop and be worried if you are going to make mistakes that I am going to highlight, please be informed that you are paying to my workshop to get into the safest environment to fail. No managers watching you. No clients frowning at you. No appraisals being done. Just your own dream of personal excellence aching you for not being there yet but being happy that you know why you are not there. Its a wonderful feeling.


Now, for some marketing again :) You can find the details of my upcoming exploratory testing workshop titled "Accountable & Manageable Exploratory Testing" by clicking on this link 


Happy Republic Day folks!

The domain particle

The inspiration


The title is inspired from Angels & Demons where they talk about the "God particle". The topic is  inspired by so many testers asking or answering to, "How much important is domain knowledge?", "Can we test without domain knowledge?", "What happens if we test a product or technology without domain knowledge?" ... 


Every person who writes in public and in forums have been asked these questions about the importance of domain knowledge to test a product. Going through those forums, you'd know that there are answers like : Yes, domain knowledge is very important to test a product OR Oh yeah, you know without domain knowledge I couldn't have found the show stopper I found a few days back...and others saying, "I did manage to learn quickly, so it doesn't matter as long as you too can", "I think it depends on what you are testing"... 


Passing time by failing to respect others time


The question I have to most testers out there is. If we, who reply to all those questions of yours, say that it is very important that you should be an expert of the domain to test the product, are you really going to become one or even try to? 


You are aware that bug investigation skills are important, without we having to say that, what have you done about that? or would you go pick up the test framing skills?


I am convinced that there are many hundreds and thousands of testers who are a trap not just to themselves but to others wanting to help testers. Every testing related forum is infected with people asking questions who actually don't intend to do anything about it despite unintentionally (or maybe intentionally) wasting a lot of passionate testers time & energy. There are genuine people in there but in a rare occurrence. Most genuine people aren't asking questions but answering them and not all who answer questions are genuine. You may want to consider me among those who are answering questions but not genuine, if that pleases your ego.


I have respect for CDT mailing list and Software Testing Club. There are serious people, asking, answering and watching. I just hope there are a few more forums like that.


Why do we have labels?


Some testers are in a trap of calling themselves "Telecom testers", "BFSI testers", "Web App testers", "Mobile App Testers", and what not? The question I have is, should it matter? BTW, I call myself an exploratory tester because its an approach I follow to test any software & not a domain.


As how programming is a mindset and a good programmer wouldn't call themselves "Java only programmer", testing is a mindset too. Oh, there is a skillset along with the mindset.


Henrik Andersson of Sweden is going to do a Webinar on October 12 : Do we need labels? Are we all not testers? I read the abstract of the webinar and was excited. I envy Henrik a lot. He is the guy who is blessed to be in the happening places - be it PSL, AYE or CAST. Other than that he is the energetic test consultant & partner at a testing consulting firm based in Sweden. I think you all should register and listen to what Henrik has to say. 


Henrik made me ask these questions: Why am I calling myself an exploratory tester, rapid tester, and whatever fancy stuff I had been calling myself when I know all testing is to some degree exploratory? What is the need for me to do so?


I need a label to differentiate myself or to communicate to people what I specialize at. My label helps my potential clients to gain interest to check about my services based on their needs. My label acts as a filter. People know what questions to shoot me. So far I have received very few emails from people asking if I can help them record and play QTP scripts. Most emails are about seeking help on how to develop themselves, think & test. I love to talk to them because they teach me things I want to learn.


So, I see label as a filter but I am against the labels that filter my own opportunities to learn. For instance, if I call myself a "Multimedia Tester", I would have blocked my learning on many technologies and contexts that I have experienced. I am starting to use "Brainual tester" these days. I am also giving a lightning / lightening talk in GTAC this year about it. James calls it Sapient and I call it Brainual. We both worship the same God but we call them in different names. I prefer the word "Brainual" because it is quicker to replace the word "Manual" to those who see "Manual" as a hand activity of testers than brain. People are used to saying, "Lets automate those manual test cases" but I expect the reluctance to set it to say, "Lets automate the brainual work thats being done now".  Their own ego would hurt them. They don't want to be seen as a fool.




Filter-less


Being a consultant and added to that the twist and turns I have had in my testing life has got me to test software ranging from wireless, mobile applications, medical devices, multimedia, Retail, CRM, desktop applications, dating applications, billing solutions, video surveillance, stock market, auction systems, kiosks, cloud computing, testing tools, games and what not. I have never bothered what domain each of them belong to as long as I have 

• An understanding of general principles of how software works (which I constantly refine)
• A skill to quickly learn and convert the learning to tests & churn more learning out of it.
• Ability to build models of learning to speed up my learning.

I try understanding how people who invented things might have thought. I fancy thinking that all these technologies have emerged out of observing something. Some of the key observations about human being and living things have led to many inventions and discoveries. Should I tell you that birds were an inspiration for Wright Brothers?


The human connection to technology


# 1 DHCP as to how it has evolved out of human behavior : When we go to the theater to watch a movie, we ask the personnel at the ticket counter to check if there are tickets available for a specific movie (sending request from a client to the server) of what we want to watch and our preferred time (details about client to see if server has anything for us). 

The ticket counter personnel in the theater gives us a ticket based on availability (assigning an IP address from the available pool) with the seat numbers to watch the movie.

In the ticket, there exists a seat number ( IP address ), duration of validity ( lease period ). If you prefer to watch the movie again and in the same seats ( Static IP ) you need to advance book or renew your tickets  
( ipconfig / renew )


# 2 Why do connectors have a male pin & a female pin? How did gender come in to technology without having observed humans and how they interact?


I have a huge list of things that computers/ technologies do which are based on how humans communicate or how humans could have. Think of SIP protocol or a client server host architecture and relate it to human communication. You'd probably enjoy as much as I do.


Simplification


In Rapid Software Testing class of  Michael Bolton & James Bach, there is an interesting dice exercise. I didn't crack it the first time but on learning the meta pattern, I am likely to crack it for any pattern or will get closer to solving it.


I have my own version of the dice game but with a different learning objective - simplicity. Based on running the exercise on thousands of testers in India (and a few folks outside India), I am making a conjecture that I see people struggling to cope up with simplicity. 


The first set of thoughts that comes to a human mind is not necessarily simple and the next set of thoughts are usually more complex than the previous ones. So, I think humans keep building on the already existing complexity of their previous thought.


Unless you train your brain to think simple, you are unlikely to crack many things you can.


The reason I am telling this here is because those who set out to learn a new domain forget that there are simpler sub systems of the domain they already know either because they also exist in other domains they have tested or have used such products extensively. You may want to figure out the meta pattern of software & how they are supposed to work. 




Practice to be fit

Quoting Parimala, "What you know is not as important as what you can do with what you know". 

I run an exercise in my workshop in which I allow people to do freestyle exploratory testing. At the end I ask them the techniques they consciously used while testing. Although many of them know many techniques to test, at least the first 10 minutes of most testers freestyle exploratory testing is, "Clicking here and there to see if a bug dances out on the screen" If a few bugs do dance, "Wow, you see I did exploratory testing". I think of that as an inferior self standard setting to understanding exploratory testing.  


How do we use what we already know to achieve better results? PRACTICE!


The biggest shame for ISTQB / CSTE certification bodies doesn't come from those who oppose it but from those who are certified & don't show traces of having gained anything from it. I have consulted at least for a few organizations here in India who hire these certified testers. Not a single test case document that these certified testers have produced has anything related to what they might have learned from the certification. 


There is a scripted test design technique that is a best practice and the most widely used one although it doesn't have a name - copy paste a sentence of requirement document into expected results column and then write the test steps accordingly. This is the most successful trick ever invented in testing. 


As a side note: Do you play any outdoor sport? Have you taken a long break from it and then went back to it? You might have been good at it at some point but when you get back after a break, you no longer feel the same comfort you had. If I were to speak to Indian testers alone for a moment, how does it feel to hold a cricket bat and face a fast bowler after taking a break from cricket? 


Bits & Pieces


I admire ants. They have a way to deal with problems. They don't say, "Oh my God! That cake is about 1000 times bigger than my size. How can I eat it?". They break the huge cake into bits that they can process, carry it home and then come back for the next bit. Doing it bit by bit helps them to achieve the goal of moving the entire cake into their colony.


While testing a product whose domain / technology that I don't know, I try to remember the ants. I learn one bit, use that bit to frame tests. The tests I perform with the bit I learned, help me learn about more bits of the system and I choose to eat bit by bit or byte by byte if my mouth has got bigger.


I may know nothing about a system when I start but at the end I can learn so much about it that it would amaze me or my audience if I tell what I discovered going bit by bit. I demonstrated this in one of the workshops where I learnt the user base of the website, traced the kind of users, identified why those users might be coming to the site, what kind of problems such users might be facing, what kind of tests to be run, what could be the most important problems that is making the audience to not give more sales + found 14 issues + 10 questions to the developers - all in one hour. I had enough feedback to the development team that they got busy working on a few things that gave me the time I wanted to go learn what I wanted about the product.


Live oracles


What are the business analysts doing? Are your sales and marketing folks so busy that they don't have time for you? How much do you interact with them? Have you invited them for a paired testing? Have you talked to them about the importance of they being with you for an hour in a month while you are testing?


If that's one set of questions, here is another set: Why in the world do you want to know everything about a domain when you know its not possible for any human being to do that?


Fooled by foolishness


Some organizations who hire testers only because of their domain knowledge, are fooled in other ways such as, those testers probably know little about testing to be called as testers. Its opportunity cost. I have spent all my time trying to be a better tester, you ask me to troubleshoot a network router, I may end up testing it or learning about it by testing it. Ask a network admin to test a router, she may end up troubleshooting or reconfiguring it.


Some organizations who continue to think that domain knowledge is the most important skill for a tester to even apply or be interviewed by them are blind to the fact that most issues that their customers are reporting has probably got nothing to do with having testers with extreme domain knowledge in the team.


There are tons of testing problems these organizations are not paying attention to, while they see their problems are because they don't have enough testers with enough domain knowledge.

Everyone of us choose to be not fooled by something and that exposes us to be fooled by something else.

The domain particle


Do you have a domain particle that makes you think "domain knowledge" is the important thing for a tester to perform well? 


I don't want to take it out. I want it to be there so that I can help you mutate it. If you could help yourself mutate it, and make the domain "learning" instead of "BFSI", "Telecom", "Multimedia", "Whatever"... I think you would have cracked what you are likely to in 20 years from now. That's an opportunity to be wise without needing to age for that.


Did you say that?


Those who speak about exploratory testing are asked, "Are you saying there is no value for scripted testing?". Those who speak about using brains to test are asked, "Are you saying there is no value in automating tests?". Those who speak about Check Automation are asked, "Are you saying checks are not tests?" while the speaker/author didn't mean any of that. Those who speak against certification are asked, "Are you saying people shouldn't get certified at all?". Those who speak about wasteful documentation are asked, "Are you saying documenting is a bad idea?". Reading some of the above paragraphs, some of you might have had a question, "Are you saying all ISTQB/CSTE testers don't know test design?"


So, whatever anyone says, there is a group of testers aggressively waiting to ask such questions. They are not doing anything wrong. They are just being themselves. Some representatives of the group are likely to ask me, "Are you saying domain knowledge is not needed to test the product?" and I am going to punish them by asking them to re-read this big post.


BTW, I am looking to hire testers of Banking / Web Application domain testers. Please send me your resume as soon as possible. 


Also, if you are a tester from Hyderabad and want to meet Rahul Verma, Dhanshekar and yours truly on 27th evening, email me. We wont mutate your particles! We will be attending Google Test Automation Conference in Hyderabad. 


Post bio for my self reference: Took 5 days to write the first draft, changed the style and contents 3 times, 1 external review, 3 self review, 5 hours of editing and finally publishing it. Published from my cousin's place in Delhi. Laptop: IBM Thinkpad

Coaching testers on Bug Reports, Advocacy & Credibility

Here are some of the funny/awful bug reports that I have seen in my experience:

• "When I open the application and click on that button I get a error massage saying it fatally crashed"
• "The spelling of Transport Parametre is miss spelled"
• "When I perform a submit the application has error message and thrown on the right side"
• "I open and it crashes"
• "The application is giving me unexpected message to work"
• "Clicking on that link is taking me somewhere and I am unable to return"
• "Applications is throwing pops up when I put my mouse on links of ads"
• "Click fast links and get error message"
• When trying to establish connection with devices some device say I am not available
• Everytime I execute test case TR234 my PC rebooting
• Test case TR 343 fails
• When users click Submit twice & hangs.
• After waking from sleep, application don't respond. (Hope you got what the tester was trying to say)

Impact

• Now, that's funny for the moment but what about the credibility of such testers? 
• Why would they be respected? 
• Why would their bug reports be even read further? 
• Why would they need a hike? 
• Why wouldn't they be treated not on par with others? 

Unfortunately, more number of such bug reports have made developers sick of reading bug reports. That's why you'd see developers call testers to their desk and say "Show me that bug". I am concerned about this problem because it is yours and mine. When we who report bugs in a credible manner join a team and report bugs, the developers still ask, "Show me that bug". That's how bad the effect has been on most developers that I have come across in India. Other country folks reading this could share their experience.


Bug reports are one of the key factors that make or break the credibility for testers.



You know about Hands on Software Testing Training from India? If you haven't, you must take a look at the excerpts of students work. You might be surprised and ask me if fresh college graduates from India really reported such bugs in such a credible way.

Santhosh Tuppad, a tester who chose me as his mentor, striked this distinction - he was profiled at Utest for gaining credibility for the highest bug approval rate. You should read about it, I think.

Santhosh, just has an year of experience. However, the practice he did, in that one year, is what has made him get profiled at utest and win the credibility of the highest bug approval rate. Not just Santhosh, his batch mates are doing exceptional as testers /for their year of experience/ in the organization they work for. That even includes his girl friend :)

I just did what any training is actually supposed to do. So, I am not going to be bragging about myself or about the magic / voodoo stuff of coaching testers. Its safe, you can read it without getting hurt.


Its a shame that some of us are being looked upon as good thinkers for just doing what every tester is supposed to be doing. That's the state in which our community is. I think the difference is that we /those who are considered good thinkers/ are bolder, respect ethics, have higher self respect and want to see the community better.

Let me just share how I coached testers on bug reporting and bug advocacy, so that it could either help you coach yourself or others. The ideas to coach testers that I have is abstracted, borrowed, redesigned, modified from what I observed while getting coached by James Bach & Michael Bolton. I also did a context driven approach to suit India.

Coaching testers to report bugs

There is a one week of training & practice session on Bug Reporting in my Hands on Testing Training class. In typical approaches like ISTQB, CSTE training, they appear to cover it up in 2 hours or less. Wow! Those guys have scalability and I don't.


So, I set up the bug tracking system, provide them a buggy application (mostly any software) and ask them to report all bugs they see. I clearly state that it is not about who finds the most number of bugs but who reports bugs in a credible manner. I don't teach anything about bug reporting when I start this exercise and leave it to them to learn it through the experential approach.



I am logged in as admin from my laptop. The first bug report arrives and I call that participant to my seat. I critique the report in following ways:

The developer view

"I am a developer and I don't understand your bug report. I don't know if this is happening while running in Vista because I see a different behavior in XP. As you have no indication to me about your OS, I have no clue what to do with your report. This is why I don't understand your bug report. I am deleting this report because it doesn't make sense to me. Can you report it again in a way I can understand it?"

• The participant goes back and attempts to report the same bug with additional points based on what I mentioned.
• That participant whispers the message across to other participants and I just act as though I don't know about the whisper.
  

Some kinda smart people pick up a little more and mention other details such as Service Pack 2/3 so that their bug report doesn't get deleted again but others don't get it yet. No problems, Pradeep is there to help.

The next bug from another participant arrives and also gets invited to my seat. "I am a developer who is sitting in US of A and I can't understand your English. Maybe it would help me if you could send me a screenshot or a video. I am going to delete your bug report as I don't find it helpful to me.

•  So, that participant goes back with a little disappointment that although the System configuration was mentioned, the bug report was deleted.
• Whispers about screenshot and videos gets started. I didn't hear any such whisper :)

The next bug from another participant with system configuration and screenshot. A 6 MB BMP attachment."I am working in a dial up connection and this attachment is taking 20 minutes to download. I have lost my interest to see the screenshot. Maybe you logged a great bug but I would have been able to see it if it were a JPG file that takes lesser space"

• Whisper! Sssssh!
• I don't know :)

Another bug, another participant. Spelling and grammatical mistakes, filled and stuffed in the report. "Thanks for coming to my seat when I asked you to come. It shows the respect you have for me. I wish I could respect you that much or maybe even more. However, I see that your English looks so bad that my colleagues would laugh at me if I respect you. In order to prevent that from happening, I have no option than to delete your bug report"

• "Hey, please correct this report" whispers one to another.
• I then announce "Microsoft spelling and Grammar check is your best friend".

As a Developer who loves to say, "No user would do that"


"Hey, the bugs you have reported are not actually bugs. So, I am deleting them all".
"What? Sir, we put in so much of effort and you are deleting it all" (Hate the "Sir" part though)


Well, if you'd not want to me to delete any of your bug report then I need strong evidence and investigation of why you think it is a problem. I wish I knew which Oracle you used but none of you reported any such.

• At this moment, I hear people wondering what oracle they used to spot the problem and trying to add to their existing bug reports.
• I announce, "You could ask me which oracle if you are not sure". For the next 15 minutes I am your friend who knows how to test and the oracles for the bugs you reported.
• Those 15 minutes is real busy time for me.

Next iteration. "The oracles you mentioned seem to be OK but I am still not convinced if that is a problem. It might help me understand the problem if you could tell me the impact it has on the user. Hello! Bug Advocacy! Cem Kaner! Google!"



So that goes on and on and it takes them 3 days to get one bug report to not get deleted by the developer. By then they have surpassed most of the ISTQB, CSTE or any other similar certification training. However this approach that I am talking about here sucks. Man! it doesn't have scale.



As Test Manager


So, from today I am a Test Manager. Not a developer. I hear a sigh of relief. They think their reports wont be deleted as much as it did with the Developer.



"Ah! My manager has called me for a meeting to give an update. When I look into all your bug reports, I get no clue what it means. Your summary is either too long or I am unable to understand them. I don't have time to go through your bug reports in detail. I shall delete all bug reports that do not help me make a quick assessment of the quality of the product"


"Hey! Here is how I report bugs!"

• No Whispers this time.
• They start copying my style and when they are learning it for the first time, that's Okay.
• All bug reports gets changed to a style that I follow.
• The beauty comes when some people try to modify my style to their own style. That's the Indian Masala I like the most.
  

Now, that kind of thing happens for a few iterations and it goes for one full day or so, till I am satisfied about their bug reporting as a test manager.


As a co-learner


"Guys and gals. Now, we are pretty OK with bug reporting and bug advocacy. Let's try to critique the bug reports that are public and try to learn if there is something interesting that others do that we didn't so far"


I open bugzilla.mozilla.com and go through many reports and ask them to point out the good and bad points in the bug report that we are observing. We have a discussion, argument and debates on it. So, we end up refining ourselves and I drive home some important points of bug reporting.


"Folks! Lets test your bug reporting skill. Let's work on another project and see if we repeat any mistakes or make new ones".


Fail fast, Fail Safe


I failed miserably in a couple of exercises that James and Michael did with me as a part of Rapid Software Testing. However, for me, it was safe to fail in front of them than to fail in front of a client. I was glad I was exposed to a context that I failed although that kind of context had not been on my work yet. When the context actually arrived, I said, "Aha! The Wine Glass". Its a RST secret :)


I have a checklist of things I would make the participants fail and get them corrected of it with respect to bug reporting and here are some of them:

• Spelling, Grammar & Typos checks
  
• No usage of SMS language
• Crispy & Useful summary
• Serving different stakeholders
• Observations
• Investigation
• Risk to the user
• Inferences & Conjectures
• Cost versus value
• Screenshots & Videos
• Log files & Supportability
• Symptom versus problem
• Cost of fixing / not fixing the bug
  
• Questions to the developer
  
• System, Browser and all other relevant configuration
• Test Environment awareness & details
• Self critique of ideas and bug reports
  
• Peer review
• Input / Output / In between : test data, system state, environmental changes
• Hardware

Enough! You might ask me how big is my bug report usually. Well, it depends on the bug and the audience not on me. In the last two assignments that I am executing here in Delhi, a light weight bug reporting with all above information is helping me win accolades. Sorry for bragging although I said I won't. Its kinda bad habit, you see.


Oh, you know, I also won the Top Best Bug at Utest in the Bug Battle of Search Engines. I wish Utest could publish that bug report. Since then, I see some testers attaching videos of their bug in utest bug battles and releases. It really solves the problem of language barriers, steps to reproduce for most of the bugs. I didn't invent the idea of video recording bugs but I practice it and advocate it to be used in contexts where it helps.



So, coming back to the topic of Bug Reporting, Bug Advocacy & Credibility. So, are you surprised that Santhosh Tuppad who went through such a training achieved the status of highest bug approval from Utest and is invited to many more releases to test?


Whenever I go and speak to businessmen in India with the results of this kind of training approach and the value it can bring to the community the thing they say after saying "Great" is "This doesn't have as much scale as ISTQB or CSTE where I can make any Tom Sick Hary to replicate the training. Slides are replicate-able. Good luck".


So the message to you is.Forget all this hands on approach and other blah blah. ISTQB has scale. CSTE has scale. CSQA has scale. ISEB has scale.Go there and learn to memorize the CBOK and practice to puke it on the exam.


I don't coach people to memorize. That's not how I was coached. Go find a guru who would teach you good stuff. Don't waste time with businessmen talking about a stuff that they too know is good but they think won't scale.


I will show the world that good stuff can scale. If you didn't like the word "I", let me replace it by another.


"We will show the world that good stuff can scale".



The Avatar!

Attractive professions and software testing

When I was a child I never dreamed to be a software tester and always wanted to be a fighter pilot, a naval commander, a cricket player, an astronaut, a doctor, aeronautical engineer, an actor, an army commando, a scientist.

Why didn't I think of being a software tester? Ah! Maybe because there wasn't anything I heard about software in 1980's but what about children today? They still say, "I want to be a pilot, a naval commander, a ramp model, a cricket player, a tennis player, an actor, a Formula1 driver, a Software Programmer, a Scientist"

I was investigating the history of how certain professions and sport are more popular and aspiring to many young people and discovered a few things about it.


Visibility: The idea of what a human does in a specific profession must be visible or made visible to a larger group of people who might not be in that field. For instance, my brother decided he'd want to be in army after he watched a movie "Border" and wanted to fight for the nation. In 1990, I watched "E.T." ( no, not Exploratory Testing but Extra Terrestrial ) and wanted to be an astronaut to be able to talk to aliens. A friend of mine watched Jurrasic Park and wanted to be a software programmer and a hacker. In parallel, I watched Kapil Dev's cricketing skills and wanted to be like him. Today my favorite sport is Table Tennis. Table Tennis has undergone a huge change in last few years to make the game more visible to the general public. Can you believe the ball size has been increased?

If you look at the Bolywood, you would see that one person inspired another to get into acting, music or direction. It was finally a movie ( Edison - The Man ) that changed my whole life downside up where I witnessed Thomas Edison's invention, hardship, work, experiments, challenges. I wanted to be an inventor and so am I, today, an inventor of tests.

Money & Richness: As a child, I didn't know the money behind each of these profession that I wanted to be in. When I discovered that cricketers are some of the richest people in India, I did dream a little bit about it and then the dream changed to movies, and then to something else that interested me at that time - complex electronic circuits. So, I did my Bachelors in Electronics and Instrumentation Engineering to help myself earn money being an Electronics Engineer. I am so close to electronics always, you see. When I am typing this, I realize a thin slice of key board separates me from a complex circuit. Being an independent consultant in software testing in India, I must admit I make more money than 98% of testers of my age or years of experience. Forget money, I feel, I am rich with testing knowledge and skills. I admit I am greedy to be more rich this way.

Parents: are a huge influencing factor. My father always talked about science to me. I used to ask him questions about science and engineering and he kept encouraging me in many ways by getting books on science or taking me to a science fiction movie. My mother was interested in Biology but I wasn't. I think my parents helped in fostering my curiosity. For a birthday gift at my age of 11, I got an Electronics - Make it Yourself kit from my parents that I assume as another key factor. So, testing suits me since it demands curiosity. In many countries, I do see the influence of parents on their children. The Bach Brothers ( James & Jon ) despite being software testers are also writers, like their father Richard Bach.

Community Respect & Relatives: plays an influential role for some people. I have heard during my relatives marriage and other social meeting about people boasting that their son or daugther got a job in IBM as a software programmer or a job with Microsoft at Redmond. So, hearing such words inspire other people to think of it so that they could boast about a similar thing later. They go back home and ask their children to aim for being a software programmer in IBM or Microsoft. Today my parents communicate proudly to their friends, "My son is a tester who is quite reputed for his work around the world." ( Ah! They do know about some of you who hate me and would not agree to that statement. )

Friends: often influence each other by introducing them to new professions that one might not have heard about. I still remember, a friend of mine who influenced me to be more serious about Electronics by showing a pocket radio that he had assembled. I wanted to do something like that and maybe even more. I quenched the thirst during my 3rd year at college by making the first telephone controlled gas stove knob with a circuit a size of Nokia Pocket communicator.

Mentors: During my first year at work, I worked with a tester who was trying to get on to the development team of the same project as he didnt like running test cases. I was influenced by him to learn programming and try my hands on programming. Although the organization who had hired me as a tester didn't want to take me as a programmer because they felt I was being useful to them as a tester, it helped me learn some bit of useful programming. I tried learning programming with the help of a System Architect and a Senior Programmer of the team, they helped me learn how equally challenging was testing activity by testing every program I wrote.

Professional Gurus: Oh! You know this. James Bach and Michael Bolton are my gurus. I have witnessed their testing online and have been constantly tested by them. They helped me clear the darkness I used to see in software testing and become a constant learner and practitioner.

Challenges & Fun: are something that everyone look forward to in any profession they want to be in. What most of the testers and people kept hearing about testing so far is - There is a process, we simply follow it. There is a test case, we simply follow it. There are those two things, we simply follow it. There is a tool we record and play everyday. There ain't any fun there ain't any challenge in following and merely recording and playing.

Exploratory Testing and Rapid Testing is gaining huge ground in places like India, ( ask me how many people are booked for my upcoming Exploratory Testing workshop and from where all they are coming and who are they ) software testing is becoming more fun, more challenging, and testers are constantly exploring lots of new things and discovering and inventing and more...


My analysis of Why Software Testing hasn't been that much of an attractive field so far?

• It lacked (past tense) visibility
• It lacked information about testers making money
  
• It lacked community respect
• It lacked parents awareness of the profession
  
• It lacked friends awareness of the profession
• It lacked good mentors
• It lacked enough professional gurus
• It lacked enough freedom being given to testers and hence wasnt challenging
  

My strong conviction of why the situation would drastically change:

Videos like this one which aids more visibility into our profession



and like this one which aids the curiosity of existing testers



and like this one that demonstrates the financial gain



and like this one that demonstrates the fun in being a /thinking/ tester as opposed to being a certified tester



And like this one that demonstrates the challenges

And like this one that demonstrates the exploration of new ideas to find bugs



While you are reading this, a child somewhere in the world might have opened Youtube and searched for E.T and hit this video and might be dreaming of becoming a software tester. I bet this is happening. I realize we haven't seen a father son tester combination but we have heard of Bach brothers.

Today is the foundation for tomorrow. What are we doing today?

Testing Experience Magazine : The voice of ISTQB

I agree you need money to run a business but then...

If you are subscribed to Testing Experience - The magazine for professional testers, you would have got an e-mail recently ( a day or two before this post ) announcing first year anniversary of the magazine. I am subscribed to them from their third issue and in an year they have grown to about 350,000 subscribers ( as per their own website claim )

I congratulated Jose Diaz for running the magazine and for completing one year with it, and offered my respect to him for doing it. I also wrote in the same e-mail to him that I see the magazine becoming too much of ISTQB stuff to which he seemed to disagree with me. I am surprised with that.

I decided to investigate if I could be wrong and presented my findings to Jose over an e-mail:
( modified the construct for your reading and not the content)


Here is my experience of going through Testing Experience magazine

• When I open the website www.testingexperience.com I am forced to look at a pop up ad for ISTQB. Only after closing that I am allowed to browse through the pages. [ As on May 29, 2009 2300 IST ] Do you see anything different?

• Let's look into the latest issue : Security testing: Page: 2 : The opening ad of Testing Experience is ISTQB. It kind of puts in my mind the thing that Testing Experience means ISTQB.

• Page 3: Jose Diaz has personally covered ISTQB in letter from Editor section and he says, "Last but not least, I want you to pay attention to our new e-learning portal www.testingexperience.learntesting.com . You can register for ISTQB Foundation Level and very soon for Advanced Level. Enjoy Learning". When the chief says pay attention to something it indicates to me that Testing Experience is supporting ISTQB because they are feeding Testing Experience with enough money to do that.

• Page 6: Yet another ad of ISTQB in German language or something similar to that. Hey look, before I even get to read the first article in a magazine, I am asked to cross 4-5 instances of ISTQB.
  

• Page 13: Another instance of ISTQB half page attractive ad
  

• Page 21: ISTQB again

• Page 24: Erik Van Veenendaal writes about ISTQB and wooing people to go for it. Here is an excerpt from that article: "The ISTQB Advanced Level scheme offers a complete 5-day module on technical testing which may seem irrelevant to testers today, but I'm convinced will become more and more important to the near future or even already today".
  

• Page 27: Interview with Mike Smith: Yet another ISTQB masala flavor in the form of LearnTesting. 3 pages full of that. As though - Learn Testing means only about learning ISTQB way of testing.

• Page 30: Yet another ad for Learn Testing who in turn promotes ISTQB and run by Testing Experience.
  

• Page36: An article from Rex Black, a popular founder and face of ISTQB. To his credit, he doesn't talk about ISTQB.
  

• In what way is his article on page 66 to 70 that talks about Advanced Software Test Design Techniques is related to Security theme of the magazine? Do we have so less people on Security Testing that we'd have to compensate it by a non security article.

• At least aren't there any other brilliant ISTQB certified testers who can write good articles on Security Testing or maybe even the Advanced Test Case Design Techniques?

• Page 52 & 53 has the ISTQB and information such as "We are the community with 1,10,000 people. Are you on the right side?". Its a Shame. Rex Black speaks against the idea of Schools of Testing but has ads asking testers, Are you on the right side? with a link to ISTQB below it. What is the message?
  

• Page 58: ISTQB in France

• Page 70: Another article from Rex Black
  

• Page 78: RBCS & ISTQB
  

• Page 99: There is ISTQB, yet again and a full fledged article on it for 2 pages.

• Page 103: Is yet another mention about ISTQB being there and being here for those who didn't know it will soon be everywhere .

• Oh and Page 104 also ends with ISTQB training and other details. ( There are only 104 pages of this magazine )
  

• Compare their first edition of magazine and there on, you would see ISTQB increasing with every edition. I wouldn't be surprised that the future editions is only about ISTQB and I see the future in their latest edition.

Testing is Questioning

Where is all the money that many testers across the globe paying to ISTQB certification and training going? Asks Michael Bolton on James Bach's Blog

Is the magazine becoming an official mouth piece of ISTQB or there is so much of bias that it makes me think so?

Whose money is all this? Your money. You didnt know you can make someone as rich as you wanted to be?

I think anyone who is running a business can cling to a brand to build their own brand but not to an extent that the other brand swallows it. If that is the mission, I think Testing Experience is headed on the right track.

I don't know who is in the review board and I suspect that it might not be diversified and have only ISTQB promoters.


What do you think?

• So, you reader of this post - What is your opinion? After seeing all this would you think the magazine is neutral or is serving the purpose of promoting ISTQB?
• Should magazines like these that claim to be for Professional Testers be promoting one kind of stuff or be helpful to the community in allowing different kinds of people to mix with?

Test Republic is a social community for software testers and a not for profit service by Mohan Panguluri, COO, Edista Testing Institute. You can witness that by visiting it. It has events of ISTQB posted in the past, it has CAST conference details and QAI conferences. It has a blend of all kinds of thought process in software testing and arguments. They allow all kinds of information to be posted from all kind of communities and schools. They keep aside their business and do this for benefiting the community. Another example is STC by Rosie Sherry. I think there is lots to learn from people like Mohan Panguluri and Rosie Sherry, for the rest of the world who also claims to be doing it for Professional testers.


The philosophy

Jose seemed to be OK with my above response of facts about ISTQB in Testing Experience's latest issue but wasn't comfortable after I mentioned to him that I plan to blog about it. He cited that his lawyers might have a look into my blog if I do so.

So they might be reading this post along with you. If I question ISTQB, the dynasty of ISTQB will hire lawyers to see if I can be attacked for questioning them?

How cheap? Software Testing is such a nascent field, we haven't even crossed 50 years and already someone is saying "The best practice and right ways of testing" and "we control the field and there is no room for disagreement".

Kudos ISTQB certified testers. You are sinking the field you are working on without knowing you are doing it. Fantastic, go ahead get certified!