"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Tuesday, August 25, 2009

Testers & Blocks Consulting - Puzzle 1

At 9 AM on Aug 13th, 2008, Testerlock, Principal Consultant of Testers & Blocks Consulting, walks into the lobby of Seven Inc in Whitefield, Bangalore and asks for Peter, the Head of Testing. Testerlock, was supposed to attend a meeting with Peter and his team of Test Managers. There had been some testing problems in Seven Inc that had been causing concerns to top management. So, it was Peter's boss Nick Fry, who hired Testerlock to figure out what's going wrong.

After all security clearances, Testerlock was welcomed to a conference room by Peter where his team of managers, Sasha, Rubin, Maria, Vibeesh, Ratan and Ankit were waiting for the meeting. The conference room was quite big and could even fit 20 people with an oval table in the center and Featherlite chromium hand rest chairs . Testerlock's name was already written in one of the paper clips marking his seating position near the door. Everyone in the room had heard about Testerlock through his articles and published podcasts and interviews. As Testerlock sat in his seat introducing himself and shaking hand with the team, he smelled the coffee aroma and while putting his laptop bag down on the table, he asked, "Ah! Can I have a Nescafe too?" and that put Peter's team in surprise about Testerlock's ability to identify Nescafe aroma from other available ones.

The meeting started with Peter displaying the agenda planned and discussing each point with a little bit more detail than an e-mail communication that happened a week back. When Peter stopped, Sasha took over.

While these things were happening Peter was getting confused if Testerlock was listening to all that because Testerlock was constantly writing something on his Moleskine. Testerlock had to several times nod to acknowledge what Peter had said. Nodding has been a practiced way in India to acknowledge having heard something from the other.

Sasha, a Test Manager with Seven for about seven years. That's right, 7 years. Sasha had been a star performer at Seven's Texas office and she looked like one and spoke like one, too. Sasha then started explaining her team's challenges of unable to achieve 100% testing and test case being complex, bugs not being fixed and so on....

Testerlock had just one word to say, "Interesting".

From Sasha to Ankit, and from Ankit to Ratan the problems the team faced were...."productivity of testers, tester developer relationship, lack of good process, best practices not working, budget is too low for good testing to be done, test automation not yielding ROI..." and yet again Testerlock had to say one word, "Interesting" and kept writing a lot of notes on Moleskine.

When it moved from Ratan to Vibeesh and Vibeesh was explaining the challenges his team faced, Testerlock interrupted the meeting asking for directions to a rest room. Before he left for the rest room, Testerlock asked the team to continue sharing the problems faced by them.

When he walked into the room again, he had a kind of style in his walk that meant that he knew what the problem was. 

By then it had shifted from Vibeesh to Maria, and then new set of problems being listed.  By then Testerlock had stopped making notes and was sitting and listening to what Maria was saying.

It was about 11:45 AM when Maria completed flushing her list of problems. Peter was excited to ask Testerlock a question and without hesitating much, he asked, "So, Mr Testerlock, what do you think the actual problem is?"

This time, Testerlock stood up and gave a one word answer. It wasn't the word, "Interesting" but the answer to the question Peter asked.

If you were Testerlock, what would your answer be? ( Your answer need not be one word but it could be. If your answer was already listed by other people who commented think about a different answer or expand on the latter. Maybe you might hit a better one )

Thursday, August 13, 2009

Bangalore Weekend Testers: Fun, Learn & Contribute

Let me brag a little bit and then get to the core. I don't believe in feedback forms that are asked to be filled by participants at the end of a training program. That's not a time to fill the feedback form because the value of the training can be determined only when participants get back to their work and apply the ideas they gained.

However, I ask all participants to fill it because the client who hired me wants it, so I don't necessarily change myself per se after looking at the feedback forms unless someone takes time to talk to me about it. As Mohan Panguluri says, "Pradeep, you are like Himesh Reshamaya. Either people fall in love with your music or they hate you at core" and its so true. My feedback forms hardly have an average rating. I am hoping that you saw this feedback report that I bet with all trainers of the world as hard to achieve. I either get a -30,000 rating or a 6 on a scale of 1 to 5. So, you see, how much they hate me?

Now, the actual feedback for me is when people go back and perform better at work. Of course you know about Sharath's great story that fetched him several awards at Mindtree. That's history now.

So, let's look at present. Quoting Jon Bach, "I prefer testers who are more curious than technical. Being technical does not make you more curious, but curiosity can make you more technical."

I mentor a few testers who are as curious or maybe even more curious than me. A co-incidence that these people also attended my workshops on Exploratory Testing and Rapid Software Testing. Among several good things they have done so far, I am starting to like their initiative of - Bangalore Weekend Testing

So, here is the deal of Bangalore Weekend Testing

They ( Ajay Balamurugadas , Manoj Nair, Parimala, Sharath Byregowda ) get together online, pick a product ( preferably open source ) and test together. At the end they publish a report that is helpful to the organization, team or open source project owners. Most important of all they have great fun and learn together.

I dont think you should be deprived of such fun and learning especially when it comes for free.
  • It can happen from wherever you are and is a great way to have fun during weekend if you claim that testing is your passion.
  • You will always have something to take back to your office on Monday and try out new things to help your organization.
  • You would get to meet a lot of other testers online and network.
  • You would learn from each other and better your ideas in testing.
  • You could end up meeting them and doing more testing together.
  • These people will also help you set up a blog and help you publish your experiences and could even mentor you.
  • You help the community of software testers by demonstrating your skills and or through your reports.
  • You help open source projects better their next release or plan for a next release.

  • Once you are subscribed to weekendtesting@gmail.com you will receive updates on time and projects that is planned for the weekend or it may happen as you find the registrants online
  • A chat group is created on Gtalk by a facilitator ( say Ajay ) and invite all registered testers to it. ( Registration means sending an e-mail to weekendtesting@gmail.com saying "hey, I am curious" )
  • With the help of a Session Tester, testing for a product would go on for about 2 - 3 hours or even more as the excitement goes on.
  • Participants then spend time preparing their reports and share it across e-mail, get it reviewed and then publish it on their blogs ( if they want to ) or in a website that is coming up.
  • Will be so much fun as you are in direct control of your tests. No manager or Lead watching you and no time pressure and no customer waiting for your report. Just do it!
  • You wait for the next weekend.

Examples from the past:

01st August 2009.
Ajay, Parimala.
Blogs posts:

Note : These people are doing thing to bring the community together. It does not matter who you are, what certifications you have, what school you belong to, whether you like our ideas or hate them. All it matters here is - do you have the curiosity and passion to have fun through testing and yet be valuable to the open source community. Only fun and learning can unite us all - that's their motto.

Curious? Wanna have fun and learn to test better? Shoot an e-mail to weekendtesting@gmail.com

Join Facebook group of Bangalore Weekend Testers

or Test Republic group to get updates about it or to keep a tab on their reports and activities.

To all those who were concerned that community was constantly being divided, here is what could make us all one - fun while testing together. Here are the guys who are doing it. Be a part of it and have fun.

Update: Aug 18th, 2009

Check out how Bangalore Weekend Testers - 3 went and see if you are fine missing the 4th?

Parimala's Report : http://curioustester.blogspot.com/2009/08/bangalore-weekend-testing-3-bwt-3.html
Ajay's Report: http://enjoytesting.blogspot.com/2009/08/weekend-testing-session-report.html

Saturday, August 01, 2009

Traditional Software Testing Process - The cash cow demystified

So, I don't need to tell you that most of the software testing in the world is outsourced. I live in Bangalore and I assume a considerable amount of the world's outsourced testing projects are here. I meet testers from most of these organizations and hear the same process of testing software although called by different names their organization executives named them.

Most testers I meet lament about the process (or sucked within it for survival ) and approach they are forced to follow and try talking to their management about better ideas and better ways to test. Those executives don't seem to listen much nor willing to change.

Wondering why they wouldn't?

They have hit a cash cow.

Mark Crowther and I were discussing about Code Coverage & Requirements Based Testing in Test Republic and some how drove me to write about the way most businessmen are fooling their customers through the process they love to follow.

So, here it goes with editing and expansion. Assuming Mark from UK or maybe even you outsourced a testing project to me:

  • I would spend a couple of days analyzing your requirement document and bill you for X hours per person involved in my team.
  • I would spend a couple of days writing a test plan document ( but not refer to it ) and bill you for 2X hours per person involved in my team for preparing it.
  • I would spend a month or two writing test case document ( and refer only to it ) and bill you for 10X hours per person involved in my team for preparing it.
  • I would then again create a traceability matrix ( just to fool you and your boss about our coverage ) and bill you for 5X hours per person involved in my team for preparing it.
  • So far, total of 18X hours per person involved in the team is the billing.
  • Assuming X is 50 hours and there are about 10 members in my team, that's 18 * 50 * 10 = 9000 hours of billing with no single bug found yet. ( Mark wrote about it, too )
  • If you are paying $20 an hour per person on an average, you would have actually given me a business of $180,000 without me or my team finding any bug yet.
  • So after investing $1,80,000 on me and my team, you would want some benefits of that. So, you wouldn't pull the project out or move it to another vendor because more or less he would do the same and you would end up paying another $180,000
  • Then comes the test case execution cycles for our documented 10,000 tests out of a possible hundred million tests
  • For every new bug that you find out of the releases I make, my team would spend documenting the new test case, getting it reviewed and resulting in slower testing for you and more money for me.
  • So assuming running 10,000 tests take 2 weeks for a team of 10 members to execute. Also assuming least 50 cycles of testing, you would have paid me about $140,000 for a coverage whose value might be not worth of the money.
  • Of course there is additional documentation of missed test cases and other template filling activities that will be billed to you.
  • To fool you further, I would instruct my team to use some expensive license based tools ( what else will I do with the money you are pumping in ) to give you a sense of faster testing ( by foolishly comparing it with human speed of testing ) and call it "Automation Testing". It turns out that these tools could have helped me find bugs that are of not a great value to you but hey, we want to see test case pass more than fail.
  • Your coverage isn't improving much because we have converted manual tests to automated tests ( although its not the same test ) but to show you the speed of our tools.
  • So think about adding another $25,000 and giving you an illusion of an ROI of $100,000 while pumping multiples of $100,000 from your bank account to mine.
  • The CMM, TMM, Six Sigma, ISTQB scams are built around this eco-system to enable more money flow for hardly any value. Who knows there could be a cut for the people who know all this and yet do it.
Why wouldn't a businessman be glad about the traditional approaches to test software?

Are you asking about what happens to the users of the product?

  • Lets bother about the users of your product later during our maintenance billing phase. Don't you know SDLC ends in Maintenance Phase? If we do everything right in the previous stages then how do we prove we follow SDLC when there is hardly any work in maintenance phase?
While you are reading this, you shouldn't be thinking of this happening only in India but in most parts of the world and even within places like United States and Europe. There are smart businessmen everywhere. At one end they pay us but at the other end use us to make more money. We need money and they need us to make that. Don't make smart businessmen exclusive to India and leave your own country out of it.

I hope those who outsource start pushing for services that doesn't fool them. Who would actually listen to this argument is testers turned businessmen and testers turned outsourcing heads and testers turned business leaders.

Whenever I visit a testing services company and see a testimonial of a customer who talks about the great ROI they got and faster testing, I wonder what a heavy price they paid to believe so.

Exploratory Testing ++ , Context Driven Testing ++ , Rapid Software Testing ++ or else YourMoney --

Don't want to be fooled by outsourcing software testing? One of those who could be of help to you among many folks I know, is myself.