"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Wednesday, September 12, 2012

Story from a company that built "the best software testing tool"

I am sure you have heard of stories of UFO and Alien sighting from many parts of the world. Ask the sales team of this famous company that built "the best software testing tool" and they would tell you that alien visits are because of their tool. The aliens, according to the sales team, needs a one stop solution to all their automation testing needs and that is why they visit us.

If you think I am exaggerating, you are unnecessarily optimistic about your intelligence ;-P. Here is a story that I partly witnessed.

The company that built the best software testing tool, apparently, also built other software products. One of the software products they built was a product not related to testing but something that a lot of people need to use on their computers.  They thought they needed some checking activity to be done. As an example, they wanted to check if certain links load as per mock. This was a no brain job and this company believes in automation. So did they automate those checks? If your guess is - yes - congratulations - you are wrong again. 

They outsourced the work to some large services company in India and called it as "Compatibility testing work" while it wasn't. Large Indian services companies only care for three things - How long is the work? How many people can we bill? How will this help me answer investor questions on the revenue? 

They neither care about testing nor about the future of testers they hire. The job for a tester on this project is to "check" whether the page loads, all images load and a bunch of other things load as compared to mock and design. So, testers have hundreds of links to open and give a report of how many links did not load properly.

What is happening to testers on this project? 
  • The testers on this project feel /testing/ is boring job. They are made to believe that they are doing testing while they are not.
  • The only thing they have been doing over the last couple of years is to wait for the links to load and say "Pass" or "Fail". That's pure checking and not to be done by humans for those thousands of links.
  • The bigger problems are written below
  • As they try to move out, the world isn't considering them as testers because the only thing they have been doing is to wait for links to load. 
  • Of course, those testers can put big brand names in their resume but they realized nobody cares for it anymore.
So, is that the end of the story? Nope! The above is just a premise of the actual story. 

Several months later, the same large services company of India got an enquiry from one of their existing customer for testing page load compared to mocks but they did not need humans to do it, instead needed automation solution. Notice how carefully, I am not using the word "test automation", in previous sentence.

The services company agreed to do it and a team of automation testers by designation pulled out the best software testing tool and wrote a bunch of tests and at times record playback then tweak to get it done. One time effort and as long as the mocks or designs don't change - this solution works. Its shipped. Works well on rough seas.

Somebody within the services company noticed these two projects and said what I would have said, "Hey, the best testing tool serves the purpose of what we seem to be doing manually all this while" and then the businessmen inside "Sssh! No, it does not. We have a 5 year contract for 20 people and you know what that means to the business?" As far as I can tell you, it is in million(s) US dollars.

However, the company that built the best software testing tool is not a company that is ignorant of this. They are well aware of their tool capabilities but seem to fail in putting their own tool to use for another product within their company. When someone actually told this to the company that built the best testing tool  - they were excited. Excited not because they could save some humans for what they were not supposed to be doing but to change their marketing communication with sentences like, "Case study of how a large services company used our best testing tool to test millions of links". Interestingly they show how the large services company saved millions of dollars for their customers whilst they themselves were paying millions of US dollars to get manual checking done where they did not want to use their tool. 

Moral(s) of the story : 
  1. Humans are made to run tests that humans were never supposed to. 
  2. When automation kicks in - comparison to humans and automation also kicks in - indicating how poor their understanding of testing is.
  3. Business decisions can decide how boring testing can become.
  4. Most large services companies remain large at the expense of killing software testing and upcoming testers.
  5. Software testing tools are as useful as spoon and fork - they are needed everyday but shouldn't cost as much as a Ferrari.
  6. The world needs more bold people than just more skilled testers at the moment. So if you are focusing on training testers, don't just teach them testing.
  7. Those who trade their time for money and are designated as testers aren't testers anyway.
  8. Most often, companies that are proud of building the best or popular software testing tool are actually putting the field to shame.
  9. The best software testing tool is always the human brain. It can operate in "non thinking" mode too and unfortunately seems to be the popular mode among most testers since they are paid for it.