News - Moolya Software Testing Private Limited launched

I have been patiently waiting to write this post. Many times I wanted to but then I postponed it to this moment and I would tell you why. The news is: Santhosh Tuppad and I have co-founded Moolya Software Testing Private Limited in Bangalore.

What this means to customers, you and the testing community?

That's an important question we'd like to answer. We are going to be offering testing services. Take a look at our logo and that would tell you the first part of the story. Do you know what Moolya means?

We wanted our name to depict what kind of testing we practice and offer our customers. The central theme of our testing is "cost versus value". Moolya is a Hindi word which has two meanings: cost & value. We simply loved it the moment we discovered it. Added to that was domain name availability. It was available and we just jumped on it. 

We crowd sourced the logo design and ITdude (as the designer likes to be called) from Philipines did this excellent logo for us that symbolizes what we wanted to depict as our next theme of testing than cost versus value - using brains - thinking skills & not rote procedures. My uncle who has been a logo designer himself, gave a little touch of his own to the logo to add a vibrant color and choose the font and design our business cards and letter heads.

Then on the services part, we choose to target people who value good testing. Fortunate for us, even before our incorporation, we had a customer wanting our services. Our first project was to test a Cloud Based Operating System. We felt blessed to have that kind of a start for our company. The client was and is happy. They have given us more business.

For those planning to outsource their testing work or seek consulting, the services we offer are

• Offshore testing services
• Exploratory Software Testing Services & Session Based Test Management
• Check (Test) Automation & Test Design Automation
• User Rejectance (Acceptance) Testing & Beta Testing
• Consulting & Training
• Staffing++

So what, everybody provides that? As every company says they are different, we'd like to say, we aren't just like those different folks out there. You should consider visiting the Services page for more details.

For testers  

We hire testers, isn't that good news? ;-) More than that, our interviewing isn't going to be traditional or easy. You don't send us your resume, you send us your test report, a contact number and an email id we could get in touch with you. We would discuss about your test report and get you to test software and put you in different contexts. That's our way of hiring. Even if we scale to thousand testers, maybe, in a couple of years, we will hire that way. Right from freshers to senior people, they should be good at testing. If in case you have a certification, we don't have a problem with that. We don't care about your certification because we care for you. We are working hard at planning and creating a women friendly organization. We are also going to be tapping a lot of hidden talent.

We believe and practice what Fiona Charles said, "Lousy customer service often comes from unhappy employees. Treat people well & they'll pass good will on." 

We will treat you well. You may like going through our Careers page

For the community

We are working on putting our office open to all testers during Sunday. We are going to be stocking lots of good books (The Weinberg, The Kaner, The Bach, The Koomey, The Guaspari, ...), providing you power and internet and you may make the best use of it. You may end up meeting other testers who have come in to read books or practice testing, and can enjoy what you enjoy the most - learning to test & continuing to be better at it. You might find Moolya employees on Sunday chilling out if they need with you folks. Wouldn't it be great if we provide free wi-fi to those who wouldn't download movies but use it to learn testing? Yes, it would be.

We currently have a 25 seater office and as our business expands, we'd like to care for more of you. We just hope to make this company a dream company for good testers. We know we will make it.  

Why we think Moolya Testing is special?

Moolya belongs to a very important time period in testing. It is started by those who started their career as a tester, have remained hands on & shall remain hands on. They may hold the designation as the Director or CEO but they can test good if not great. They have at least played a small role, if not big, to stir the beginning of Renaissance in Indian Testing. They know what customers want, they know what testers want. They are young yet experienced, with lots of energy & passion. Most important of all, they have nothing to loose, so they can be crazy, creative, sound jazzy, cool, and do things that other companies just isn't doing. They are the Microsoft & Apple of 1970's. They are the Google of the 1990's. This would probably be the first Indian services company that would not talk about "head count" but talk about "brain count" of their employees.

Our culture

Just because we are testers, it doesn't mean we haven't educated ourselves on anything else. We have put special emphasis on our culture. We have been working on things that can make a great culture to a company. What is that? We'd get our employees tell you those stories, than we letting all of it.

My wishes to Santhosh Tuppad

I'd also like to mention that by having co-founded Moolya with me, Santhosh Tuppad, must have become the youngest tester entrepreneur. Congratulations to him. You are a bravo.

Going forward, India might see more such. As I said, Moolya Testing is the first of its kind company from India that is aligned with the Indian Testing Renaissance. 

Thanks

A note of thanks to my colleagues Parimala, Sharath Byregowda, Manoj Nair, Dhanasekar S & Mohan Panguluri, Vipul Kocher, Nandan Pujar, Satish Thakur, James Bach & Michael Bolton for the support they have offered so far. Needless to say, without our parents, siblings & family help, we wouldn't have been able to achieve this. 

Follow Moolya Testing

Feel free to join us on Twitter, Facebook & LinkedIn. Peruse through our website. I postponed writing this post multiple times to be able to write this from the office of Moolya Software Testing Private Limited. We are in JP Nagar 2nd Phase, Bangalore. Feel free to contact us. The story of starting Moolya Software Testing Private Limited will be launched as a book in 2011. It's going to be self-published.

"Consider engaging Moolya for your testing needs, we'd like to put smiles on *your* customers face." 

The domain particle

The inspiration


The title is inspired from Angels & Demons where they talk about the "God particle". The topic is  inspired by so many testers asking or answering to, "How much important is domain knowledge?", "Can we test without domain knowledge?", "What happens if we test a product or technology without domain knowledge?" ... 


Every person who writes in public and in forums have been asked these questions about the importance of domain knowledge to test a product. Going through those forums, you'd know that there are answers like : Yes, domain knowledge is very important to test a product OR Oh yeah, you know without domain knowledge I couldn't have found the show stopper I found a few days back...and others saying, "I did manage to learn quickly, so it doesn't matter as long as you too can", "I think it depends on what you are testing"... 


Passing time by failing to respect others time


The question I have to most testers out there is. If we, who reply to all those questions of yours, say that it is very important that you should be an expert of the domain to test the product, are you really going to become one or even try to? 


You are aware that bug investigation skills are important, without we having to say that, what have you done about that? or would you go pick up the test framing skills?


I am convinced that there are many hundreds and thousands of testers who are a trap not just to themselves but to others wanting to help testers. Every testing related forum is infected with people asking questions who actually don't intend to do anything about it despite unintentionally (or maybe intentionally) wasting a lot of passionate testers time & energy. There are genuine people in there but in a rare occurrence. Most genuine people aren't asking questions but answering them and not all who answer questions are genuine. You may want to consider me among those who are answering questions but not genuine, if that pleases your ego.


I have respect for CDT mailing list and Software Testing Club. There are serious people, asking, answering and watching. I just hope there are a few more forums like that.


Why do we have labels?


Some testers are in a trap of calling themselves "Telecom testers", "BFSI testers", "Web App testers", "Mobile App Testers", and what not? The question I have is, should it matter? BTW, I call myself an exploratory tester because its an approach I follow to test any software & not a domain.


As how programming is a mindset and a good programmer wouldn't call themselves "Java only programmer", testing is a mindset too. Oh, there is a skillset along with the mindset.


Henrik Andersson of Sweden is going to do a Webinar on October 12 : Do we need labels? Are we all not testers? I read the abstract of the webinar and was excited. I envy Henrik a lot. He is the guy who is blessed to be in the happening places - be it PSL, AYE or CAST. Other than that he is the energetic test consultant & partner at a testing consulting firm based in Sweden. I think you all should register and listen to what Henrik has to say. 


Henrik made me ask these questions: Why am I calling myself an exploratory tester, rapid tester, and whatever fancy stuff I had been calling myself when I know all testing is to some degree exploratory? What is the need for me to do so?


I need a label to differentiate myself or to communicate to people what I specialize at. My label helps my potential clients to gain interest to check about my services based on their needs. My label acts as a filter. People know what questions to shoot me. So far I have received very few emails from people asking if I can help them record and play QTP scripts. Most emails are about seeking help on how to develop themselves, think & test. I love to talk to them because they teach me things I want to learn.


So, I see label as a filter but I am against the labels that filter my own opportunities to learn. For instance, if I call myself a "Multimedia Tester", I would have blocked my learning on many technologies and contexts that I have experienced. I am starting to use "Brainual tester" these days. I am also giving a lightning / lightening talk in GTAC this year about it. James calls it Sapient and I call it Brainual. We both worship the same God but we call them in different names. I prefer the word "Brainual" because it is quicker to replace the word "Manual" to those who see "Manual" as a hand activity of testers than brain. People are used to saying, "Lets automate those manual test cases" but I expect the reluctance to set it to say, "Lets automate the brainual work thats being done now".  Their own ego would hurt them. They don't want to be seen as a fool.




Filter-less


Being a consultant and added to that the twist and turns I have had in my testing life has got me to test software ranging from wireless, mobile applications, medical devices, multimedia, Retail, CRM, desktop applications, dating applications, billing solutions, video surveillance, stock market, auction systems, kiosks, cloud computing, testing tools, games and what not. I have never bothered what domain each of them belong to as long as I have 

• An understanding of general principles of how software works (which I constantly refine)
• A skill to quickly learn and convert the learning to tests & churn more learning out of it.
• Ability to build models of learning to speed up my learning.

I try understanding how people who invented things might have thought. I fancy thinking that all these technologies have emerged out of observing something. Some of the key observations about human being and living things have led to many inventions and discoveries. Should I tell you that birds were an inspiration for Wright Brothers?


The human connection to technology


# 1 DHCP as to how it has evolved out of human behavior : When we go to the theater to watch a movie, we ask the personnel at the ticket counter to check if there are tickets available for a specific movie (sending request from a client to the server) of what we want to watch and our preferred time (details about client to see if server has anything for us). 

The ticket counter personnel in the theater gives us a ticket based on availability (assigning an IP address from the available pool) with the seat numbers to watch the movie.

In the ticket, there exists a seat number ( IP address ), duration of validity ( lease period ). If you prefer to watch the movie again and in the same seats ( Static IP ) you need to advance book or renew your tickets  
( ipconfig / renew )


# 2 Why do connectors have a male pin & a female pin? How did gender come in to technology without having observed humans and how they interact?


I have a huge list of things that computers/ technologies do which are based on how humans communicate or how humans could have. Think of SIP protocol or a client server host architecture and relate it to human communication. You'd probably enjoy as much as I do.


Simplification


In Rapid Software Testing class of  Michael Bolton & James Bach, there is an interesting dice exercise. I didn't crack it the first time but on learning the meta pattern, I am likely to crack it for any pattern or will get closer to solving it.


I have my own version of the dice game but with a different learning objective - simplicity. Based on running the exercise on thousands of testers in India (and a few folks outside India), I am making a conjecture that I see people struggling to cope up with simplicity. 


The first set of thoughts that comes to a human mind is not necessarily simple and the next set of thoughts are usually more complex than the previous ones. So, I think humans keep building on the already existing complexity of their previous thought.


Unless you train your brain to think simple, you are unlikely to crack many things you can.


The reason I am telling this here is because those who set out to learn a new domain forget that there are simpler sub systems of the domain they already know either because they also exist in other domains they have tested or have used such products extensively. You may want to figure out the meta pattern of software & how they are supposed to work. 




Practice to be fit

Quoting Parimala, "What you know is not as important as what you can do with what you know". 

I run an exercise in my workshop in which I allow people to do freestyle exploratory testing. At the end I ask them the techniques they consciously used while testing. Although many of them know many techniques to test, at least the first 10 minutes of most testers freestyle exploratory testing is, "Clicking here and there to see if a bug dances out on the screen" If a few bugs do dance, "Wow, you see I did exploratory testing". I think of that as an inferior self standard setting to understanding exploratory testing.  


How do we use what we already know to achieve better results? PRACTICE!


The biggest shame for ISTQB / CSTE certification bodies doesn't come from those who oppose it but from those who are certified & don't show traces of having gained anything from it. I have consulted at least for a few organizations here in India who hire these certified testers. Not a single test case document that these certified testers have produced has anything related to what they might have learned from the certification. 


There is a scripted test design technique that is a best practice and the most widely used one although it doesn't have a name - copy paste a sentence of requirement document into expected results column and then write the test steps accordingly. This is the most successful trick ever invented in testing. 


As a side note: Do you play any outdoor sport? Have you taken a long break from it and then went back to it? You might have been good at it at some point but when you get back after a break, you no longer feel the same comfort you had. If I were to speak to Indian testers alone for a moment, how does it feel to hold a cricket bat and face a fast bowler after taking a break from cricket? 


Bits & Pieces


I admire ants. They have a way to deal with problems. They don't say, "Oh my God! That cake is about 1000 times bigger than my size. How can I eat it?". They break the huge cake into bits that they can process, carry it home and then come back for the next bit. Doing it bit by bit helps them to achieve the goal of moving the entire cake into their colony.


While testing a product whose domain / technology that I don't know, I try to remember the ants. I learn one bit, use that bit to frame tests. The tests I perform with the bit I learned, help me learn about more bits of the system and I choose to eat bit by bit or byte by byte if my mouth has got bigger.


I may know nothing about a system when I start but at the end I can learn so much about it that it would amaze me or my audience if I tell what I discovered going bit by bit. I demonstrated this in one of the workshops where I learnt the user base of the website, traced the kind of users, identified why those users might be coming to the site, what kind of problems such users might be facing, what kind of tests to be run, what could be the most important problems that is making the audience to not give more sales + found 14 issues + 10 questions to the developers - all in one hour. I had enough feedback to the development team that they got busy working on a few things that gave me the time I wanted to go learn what I wanted about the product.


Live oracles


What are the business analysts doing? Are your sales and marketing folks so busy that they don't have time for you? How much do you interact with them? Have you invited them for a paired testing? Have you talked to them about the importance of they being with you for an hour in a month while you are testing?


If that's one set of questions, here is another set: Why in the world do you want to know everything about a domain when you know its not possible for any human being to do that?


Fooled by foolishness


Some organizations who hire testers only because of their domain knowledge, are fooled in other ways such as, those testers probably know little about testing to be called as testers. Its opportunity cost. I have spent all my time trying to be a better tester, you ask me to troubleshoot a network router, I may end up testing it or learning about it by testing it. Ask a network admin to test a router, she may end up troubleshooting or reconfiguring it.


Some organizations who continue to think that domain knowledge is the most important skill for a tester to even apply or be interviewed by them are blind to the fact that most issues that their customers are reporting has probably got nothing to do with having testers with extreme domain knowledge in the team.


There are tons of testing problems these organizations are not paying attention to, while they see their problems are because they don't have enough testers with enough domain knowledge.

Everyone of us choose to be not fooled by something and that exposes us to be fooled by something else.

The domain particle


Do you have a domain particle that makes you think "domain knowledge" is the important thing for a tester to perform well? 


I don't want to take it out. I want it to be there so that I can help you mutate it. If you could help yourself mutate it, and make the domain "learning" instead of "BFSI", "Telecom", "Multimedia", "Whatever"... I think you would have cracked what you are likely to in 20 years from now. That's an opportunity to be wise without needing to age for that.


Did you say that?


Those who speak about exploratory testing are asked, "Are you saying there is no value for scripted testing?". Those who speak about using brains to test are asked, "Are you saying there is no value in automating tests?". Those who speak about Check Automation are asked, "Are you saying checks are not tests?" while the speaker/author didn't mean any of that. Those who speak against certification are asked, "Are you saying people shouldn't get certified at all?". Those who speak about wasteful documentation are asked, "Are you saying documenting is a bad idea?". Reading some of the above paragraphs, some of you might have had a question, "Are you saying all ISTQB/CSTE testers don't know test design?"


So, whatever anyone says, there is a group of testers aggressively waiting to ask such questions. They are not doing anything wrong. They are just being themselves. Some representatives of the group are likely to ask me, "Are you saying domain knowledge is not needed to test the product?" and I am going to punish them by asking them to re-read this big post.


BTW, I am looking to hire testers of Banking / Web Application domain testers. Please send me your resume as soon as possible. 


Also, if you are a tester from Hyderabad and want to meet Rahul Verma, Dhanshekar and yours truly on 27th evening, email me. We wont mutate your particles! We will be attending Google Test Automation Conference in Hyderabad. 


Post bio for my self reference: Took 5 days to write the first draft, changed the style and contents 3 times, 1 external review, 3 self review, 5 hours of editing and finally publishing it. Published from my cousin's place in Delhi. Laptop: IBM Thinkpad

What is our competition on?

There is a trend that is picking up in the software testing industry. To be more precise, I am talking about a trend of competitions, prizes and rewards. This is good. I love the idea of being able to compete globally and with testers whom I may not have known otherwise.

When someone wins a testing challenge and the report is published to those who participated, it influences others to ape some of the good practices and end up creating their own styles out of it. The flip side, what the winner does becomes best practice for wannabe winners of the next competition. For instance, having been practicing video recording as a part of my bug reporting style, I posted videos of bugs in Utest bug battle last year. When I coach testers, I also help them understand what kind of bugs need video recording and help them practice it. Santhosh Tuppad who is a regular bug battle winner in Utest told me that many testers have started to upload videos instead of screenshots and he published an experience report where he mentioned that point.

At this point of my post, I'd like to appreciate all companies putting up competitions that is helping to bring out new, fresh and different kind of talent to the public view. I like to support you in as many ways I can. I have been a participant, creator, winner and loser of such competitions. Its all fine.


However, I am starting to have some concerns about the competitions that are coming up which demands voting by public to decide who wins. Now, I need to clarify something. I am not talking against those companies who are doing it. As a matter of fact, I am not talking against "anything". The company wants a way to get more people to know about them and such voting based competitions help in doing that. If I start an organization, I'd like to do things that helps in getting a lot more people to know about the services my organization offers. 

The problem with the voting to decide a winner, could be hurting, to those who have put in lot of efforts but couldn't gather enough votes. For instance, Eurostar conferences organized a Videostar competition. By looking at the marketing flyer of the competition which said something like, "Put on your Holywood director hat" and about creativity, I assumed the video with the Holywood movie type creativity will probably get me to be the Videostar and did the Joker act. Anne Marie did a video that I personally liked. Nothing less of all that was from Rob Lambert who had a different way of putting things that he wanted to talk. Maybe all that was driven by seeing how creative were Eurostar folks who put up this video. Finally, the video that won, by that I mean, the one that got the most number of votes, was not so exciting as others in the list. We are independent consultants who don't have a mailer list to whom we can send and help generate a lot of votes for ourselves. Being an employee of a large company and gathering votes is much easier, especially if the Head of Testing is the one asking for votes. I guess that's what happened. Now, I am not questioning about anyone's ability but I am talking about the system.

I could have still won. I could have got a 99% lead over all others if I wished to. The Videostar page recorded a vote from a browser-computer and registers it or probably sets a cookie as well so that I cant vote twice. However, I could clear the cookies, refresh the page and vote for me again. How long would it take to automate this and leave it running overnight to wake up in the morning to declare myself as a winner?

I didn't win the Videostar. That is a testimonial that I didn't try winning with the "hacked way" I discovered. Oh, if you think there would have been an assessment of votes coming in from the same IP, I could have gone a step ahead and used the cloud to make it up or use tools that helps me mask my IP and hence not reveal where the votes came in from.

In that case, should winning be decided by a panel of judges who may not understand my skills? For instance if I take up the ISTQB Foundation Level exam, I will fail. I did fail in a mock test that I took online. Does that mean, I don't know how to test or I don't know the foundation of software testing? 

A competition has a set of rules. I am just asking if the rules can be made in such a way that the skill factor plays a vital role in winning and not the votes or my ability to memorize answers and vomit it out on an exam.

Let me repeat this: I am fine with companies organizing competitions to increase their visibility but I want to know if there can be a better way to do it than the voting system. I am dreaming of testing competitions where the winner is judged based on the skill demonstrated. A report is published by the panel as to why the winning entry amongst others were eligible for the top prize. Every step we put must help us move forward. Movement is important but there is a vector in it. What direction are we moving in and by what magnitude?

Bug battles are one kind of a competition that I like. There are more kinds of skills that companies can try to focus on. When I organized a testing challenge for Test Republic, I did a Bug Advocacy Challenge. I think there needs to Bug Investigation Challenges, Rapid Test Planning Challenges, Test of agility challenges, Test Management Challenges, Interviewing Tester Challenges, Collaborating with Developers Challenge, Understanding Requirements Challenge that can help in bringing out winners with skills of different kinds. 

So, there was a competition announced from Eurostar on blogging. I was about to pounce on it because I wanted to go to Eurostar and I have been one of the earliest Eurostar blogger. I skimmed through to see if there was anything related to voting and there it was. I decided to not enter the competition. 

I would be highly stupid if I was trying to write this post to create a negative impression of Eurostar or any other organization. That is not my intention. I'd like to say that Eurostar is a great conference I want to go but not because I got a lot of votes than someone else. I want to go if my skills get me there. I understand why Eurostar or companies that are running a voting based competition might be doing these competitions but I'd like them to think if there can be a better way to do things. I have respect for organizations like Utest and Eurostar because they are trying to do some work that is helping people change things the way they do. That is the reason I want them to be able to cause a much higher and better influence on the community, to take it forward. 

A tip to the winner

If you, a tester, happen to win any testing competition, here is something that I hope you think about from what I say to myself from my experience of winning several testing competitions; I won because the mightiest chose not to compete. This thought helps you a lot when you work with people. They shall embrace you not see you as a person they should stand away or just merely stare at.

Competitions could remain but what is our competition on? 

• Choice A.  Number of votes
• Choice B. the breadth and depth of skills? 

To add a little humor to this serious post; what is your vote for? A or B?

Experience report of testing versus checking

Many thanks to my client in Bangalore who encouraged me to blog about the experience, value and outcomes we had in separating tests and checks. If you are reading this, it means my client has approved this for publishing.  

Thanks to Michael Bolton who posted a series of posts on his blog under Testing versus Checking and those who commented on the posts, whose ideas and thoughts helped me to think about delivering the value of tests and checks to my clients. Tests and checks series of posts has helped me communicate things better and organize my thoughts better. We were using the term "check" and "test" much before Michael blogged about it; however, he gave a cognitive structure to it in our brains.

The context

One fine evening, there came a client looking for me and the story goes like this... 

I was hired for a couple of weeks, to consult and test various products produced by a large organization whose headquarters is in Europe. The moment I was hired, there was a need of a hand to run through a few tests on a product. The product was used for analysis of something across many country deployments. It has an engine loaded with business rules, lots of history data in a database and report viewer application. How do we know if the business rule engine didn't violate any rule? The reports had to comply with complex business rules that at the first sight appeared to me as contradicting one another. The core development of the product was being done by a vendor in the US of A, who was reputed within my client's office for pumping in poor builds, as frequent as they could. I wish I could tell you more than that.

Video test cases instead of lame text

On starting off, I realized that the learning curve on the project appeared to be steep and I couldn't cope with it in the time I had. A demo was given to me from a colleague who had been running tests ever since he was hired. Does a demo suffice? No way. 

Building a little credibility with that colleague helped me convince him to record videos of the demo and tests with the help of Windows Media Encoder 9.0 for Windows XP. So rather than following dumb scripted tests, I was watching a video on one monitor and running tests on the other. Video test cases were more reliable than documented ones. Based on how quickly I picked up things, the videos are now being planned as training material instead of asking someone to go through test cases. 

Recognizing checks being called as tests

On running them, I realized, what I had been running were checks than tests. To quote Michael Bolton, Checks are machine dependable and tests require sapience. I was checking if the report complied with all business rules. While I was checking, I instantly knew that these checks could be automated.

Testability, Test Coverage Issues & Stakeholder Interests

I noticed a couple of bugs in the software were preventing me to run other kinds of tests as fast as I thought I could. Anything that blocks me or slows down my testing is a serious issue for me and to those who have hired me. With the proposal to test for various quality criteria being accepted, I made a presentation to the stakeholders of how the quality can be improved at least with respect to testability and other quality criteria (such as Usability, Performance, Security.. ) accompanied by a test report.

They instantly liked the improvement of testability idea because to check if the business rules are not violated for every installation, they had been taking 2 - 3 days. They needed rapid feedback to take better decisions.

Imagine looking for a specific value across many excel sheets. By the time you are on the fifth, you might almost forget what you saw on the third and where you saw it. However other kinds of quality criteria didn't matter to the stakeholders for their current context. I wish they had considered it but that's a business decision. My bug advocacy was strong though. I had stakeholders wanting to automate the checking part of our testing. That was a winner.

Partnering with developers

I was sitting in an organization whose policies and people are new to me. I was proposing things that required lot of dependencies on other people in functions such as development, business analysts and on-site personnel with of course the management. The time frame I required to achieve this was really short. Writing emails in a manner that helped the people realize the importance of the value was a big winner for me. For instance, here is an email I sent to the development team that sits in my client's location who mostly do some integration and writing wrappers.

Dear Developers,

Greetings!

I am glad to be interacting with you. As we are committed to reduce the time we take to analyze a release of XXX XXX, we realize that a tremendous saving to the organization can be contributed by making a few changes. We have made a few things from our end, such as, having a formula do the job for learning about XXX-XXX rule being satisfied for all installations. Creating a repository of possible combination of tests we need to do to be able to say we have exercised the business rules very well.

In that path, we realize that with your help, we can do a lot more. I understand that you might be busy with other things on the table but as I feel we are working together towards a common goal, it makes me request you to help us.

We request the following things from you:

• Currently we are spending about 3 – 4 hours manually to verify the XXXXXX business case for a small installation.  
• We have investigated and conjecture that if you help us have the XXXXXX ( such as XXX or XXX details ) to be available in a list associated with corresponding XXXXX and the XXXXX, it would take us less than 5 minutes for us to perform the business rule validation.  
• If these details are a part of the export of Weekly Report (highly preferable) or as a separate excel list (less preferable) it would help us achieve the goal.  
• Assuming a set of testers will be testing XXXXXX for all country implementation in future, you would be helping us bring down the cost of testing by a couple of thousand dollars ( Well, Euros, as we are European based )  
• Additionally, as we are doing it manually, we are not exceeding more than 10 categories per installation. With your help we would have the capability to go as much as the number of categories the system can support. Thereby, we test for larger samples and increase our data coverage.

We would be in a position to provide you any help you might need from us in achieving this task or would be glad to offer back any other help that you may require to make your work more effective.

Thanks & Regards,

-- Pradeep Soundararajan

30 minutes, no response to the email. At the 31st minute, a developer was at our desk. It happens that he was serving his last day for the organization and carved out time for us. I considered that as a great gesture by a developer to support test team and our test team brought a chocolate for the help he offered. I couldn't get a "Thank you" card; else I would have wished to have given that as well.


That developer helped us with stored procedures and SQL queries that we need to extract data out of the db. 


The Checker Tool Development & Testing

So, instead of looking through reports exported through the GUI, we now had the option to extract values directly from the database. We got one step closer, to what we wanted to achieve.

While I was planning to write a Perl script to extract the values and dump it to an excel sheet, I became aware of a developer in test who joined the organization and had not yet been assigned to a project. The good news for me was she hadn't got her computer to start working. I requested the manager to get her time for helping us write a script and gave her my PC for 2 days while I mostly roamed around saying "Hi" to people around and getting a hang of the culture and talking to people about their work. I could have done all by myself using Perl but I know how slow I'd be as compared to a developer in test.

I provided her with requirements and all necessary tools that she'd need to develop and test the script that could dump the values in an excel sheet in a format we wanted. It's a complex data structure and in order to perform an analysis we had to develop a report format. With the help of my colleague, we zeroed in on the report format and in 2 days time, she could help us see what we wanted in the report format.


TDD for writing Excel Formulas

Off to validating business rules through Formulas in Excel. I knew the power of Excel and its formulas but I didn't know its limitations. I remembered that I had received an email long back from someone containing  MS Excel formula dictionary as an attachment. I pulled that out to help myself.

Excel shouted a warning and error if we exceeded 7 nested loops. It didn't warn us when we had a typo in one of the formula, a star * symbol instead of 8. Copy pasting formulas from one sheet to another had reference to the previous sheet. I know lot more about excel than what I knew earlier. 


I wasn't doing a good job and I knew it very well. A month and a half back I attended a demo on TDD from Venkat Subramaniam. It struck me at the right time. Instead of continuing to do a mediocre job, I started collecting data, scenarios and information that should make the formula fail or pass. I started to write the excel formulas based on the data we had. My colleague who had been in this project for a while helped me get the set of data that had to result in pass and those that should result in fail. If you may permit me to say that writing formulas as, "coding"  then we were pair coding.


This was very helpful. After starting with that approach, we could find bugs in previously written code. Prior to the TDD and paid coding approach, I wrote a formula, loaded the data and checked to see if it failed or passed. For a condition in which it should pass, if it failed, I used to tweak the formula to see a Pass. That was dumb, especially, after I knew that making a Pass could impact other things that were supposed to fail. 


With TDD in, we achieved faster progress on our business validation formulas. We were now at a speed lane. We cracked the puzzle too. We could check for business rules violation in a matter of few seconds.


Comparing humans and tools for speed

I am against the idea of comparing human testers with test automation. I still am against the idea but I saw myself in a situation where I was comparing automation with manual effort. Why did I do this? Should I remain silent about what I have done and continue telling people who attend my workshop that; comparing a human and a tool for speed is a bad idea? I was reminded of James Bach's blog post "Manual tests cannot be automated" and Jonathan Kohl's interview where they put a lot of things so well that I couldn't have put it then.

In self retrospection, a learning emerged. We didn't automate tests, we automated checks. As humans who were not supposed to be doing checks were doing it, I was forced to make that comparison. In fact, I should have said, "Please do not see this as a faster way to do things by comparing how testers were doing it earlier. They weren't even supposed to be doing that. I just wish there was more freedom and a little bit more authority for them to express it".
  
Getting our tool tested by domain experts

"Yipee! It works". That's not what we thought. When we developed this tool, we were sure that the users of this tool could be beyond the test team and how good our tool was to be decided by a "domain expert". When we showed some of our unit test results to a business analyst in house, he was excited at how simple this would be for him to use when he goes to assess a country installation in the future. He vowed to support all that we needed. He offered help writing us a few formulas that we needed. I tweaked it to suit our needs. 


He had a logic that I wouldn't have touched upon. So, that really helped. We didn't need to be a domain expert all by ourselves to be able to do this. A testing brain and a domain expert brain coupled together yielded a lot of value.

Added to that, he was a key stakeholder we had in house. He acted like a customer in house. He suggested the changes we needed to make it more suitable to other stakeholders' usage. Having been on site, he could bring the perspectives of people who could use this tool and their contexts. We needed lot of clarification on the business rules. Imagine a set of rules contradicting each other but still plausible under certain contexts which had to be shown as pass and not fail. Whoof! I liked it :)




Live Testing of Business Rules Validations Checker Tool V 1.0

So, our in house customer was happy with it. We showed a demo to the test management and they were excited. The test manager was excited enough that he pulled out the product manager out of his busy schedule to show our tool to him. The product manager was about to be in a meeting and he asked us, "How long this would take?", because he knew the business rules was taking a few days. When he asked us that question, we initiated the script, took a deep breath and said, "It's over". We showed him the report and there was a smile that our test manager was hoping to see. The only skepticism he had is about business analyst's approval of this tool. A bug in this tool would really misinform the decision makers.

So, the tool was shipped to people in Europe who tested it on the live system. They took about 2 days to get back. It wasn't a very nervous moment at all for me. I was just impatient. They had to take two days to approve this tool because they had to compare the results of the tool against their manual checks. While we were waiting for the reply, we continued testing our tool to see if we could spot problems. We ran it for several test installations and we made notes on how we could improve but no bugs found. 


The climax

On the next day when I was traveling to my client's location by bus, I get a call from a person who heads the testing for the group which I was hired. "Are you in the bus? Can you get down? I want to talk to you. I will come there soon". Before I could ask anything he disconnected the call.

That's really crazy. I suspected something must have gone wrong. I thought the Head of Testing wants to give me a bad news that my consulting contracted is terminated. He arrived at the place, I got in, and he showed me an email and said, "Congratulations man. This is great news". The tool was approved by the business analysts who were on site in country installations. They said it was accurate and also proposed some modifications to a dashboard we had created in the excel. It hit the target on the bulls eye. 


The Head of Testing and I together prepared a presentation of the whole story to circulate it throughout the organization. The Head of Testing currently hopes the concept of separating tests and checks is carried over across all other projects. 

At one side I was extremely happy for being such a value to my client and on the other hand, I was telling myself, "Consultants like me are made to look good for doing what everyone else is supposed to be doing". Had my colleagues at the client location been people who were active in learning, I wouldn't have sounded like an expert tester to them. I just feel pity and can offer the help I can. I have to move on and learn more things.


Preventing bad builds from sneaking in

The story doesn't end there. I had another proposal up my sleeve. I wanted to ship this tool to the vendor in US who was pumping in bad builds. We are planning to use the Business Rules Validation Checker Tool as acceptance checking of a build into the client's location. The idea of automated acceptance checking appears to be interesting to my client since they work with several vendors, delivering broken builds not as frequent as the vendor stated above :)


Return on Investment, in its true sense.

The ROI is not about the speed with which a certain task is done. The actual ROI is that the testing team time is now freed from checks to do exploratory testing, bettering the test coverage, faster feedback, better informed decisions, focusing on things that they were not focusing on AND preventing bad builds from entering the client's location.

I thank my colleagues (nicknamed) Rag and Tham for the support they offered me in the three weeks. Without a management providing freedom to a tester like me, this wouldn't have happened. So, thank you Mr. Head of Testing, Test Managers and Business Analysts for all the support and for approving this blog post.


The only thing pending is a party to celebrate. Mr. Head of Testing, are you reading this? :-)

Test coverage : Life beyond functionality

At a class recently 

Scene 1:

"What have you all been doing for the last few years in this organization?"

Chorus: "We test for functionality of the software?"

"That's nice! Why do you think it is called /functionality/?"

Silence

"OK, what is a function?"

Silence

"No problem. Let's fix it. Where else have you heard the word /function/? Math? So we all might have studied f(x) = y and that could have been our introduction to working with functions. Later we moved on to parabolas and hyperbolas which had functions of different behavior and complexity. So what we are doing when we say /functional testing/ is, we provide inputs X to a function f(X) and monitor Y. We claim to know that f(X) should be Y and we also claim to know the range of Y. So, that forms our oracle to find bugs with functional testing. /Y/ doesn't necessarily need to be a numeric value in functional software testing".

Scene 2:

"So, what else do you test beyond functionality?"

• "Our bug reports get rejected if we report non functional bugs so we don't test for it", 
• "There is a team in US who is supposed to do it"
• "When we find a non functional bug and report it, we are asked why did we spend time deviating from the scope given to us"
• "We have enough test cases in functional testing that we don't have time for other kinds of testing"

"Are you doing Quality Assurance or Software Testing?"

"What?"

"Let me explain. In Software Testing, you provide quality related information to help the management take better informed decisions. I think in Quality Assurance, you claim to provide confidence to the management that you have checked and tested a lot of things (not just software) and things didn't change after your checks and hence its OK to use the word /assure/ with another word /quality/."

• "I think we do a mix of both"
• "Our designation is Software Tester but we are internally called QA"
• "I don't know. I am doing my job and earning my paycheck"
• "Yikes, this is confusing"
• "I thought Testing == QA" 

 Scene 3:

 "I appreciate the diversity. Tying it back to our previous topic, irrespective of what you do (Functional Testing or Functional QA, the information you provide to your management is so weak as compared to what you were supposed to be doing."

"How is that?"

"Test coverage is the extent to which we have modeled and tested the system -- Michael Bolton. You appear to have modeled the system for functionality and you could model your system in more ways within and beyond the scope of functionality to learn more about the product and to be of more value to your management. Assuming you are interested to do that. Here is a list of thing that you might want to focus on:

Product Elements Coverage

• Data
• Platforms
• Operations
• Time
• Structure
• Functions

Quality Criteria Coverage

• Capability
• Reliability
• Usability
• Scalability
• Performance
• Installability
• Compatibility
• Supportability
• Testability
• Maintainability
• Portability
• Localizability

Technique Coverage

• Scenario based
• Claims based
• User based
• Flow based

( refer to Rapid Software Testing slides for more )


"That looks like a lot of thing to do and I know these are important for quality. We don't have time for functionality and how can we think about all these things?"

That's a nice thing that you realize you don't have time even for functional testing to be done. How about asking the management to re design and re think about the time that is given to you and the value you can deliver?

Scene 4:


"I know for sure. They won't allow us to do all this"

"Interesting. Is that your assumption or a fact?"


"That's what is going to happen if we ask and we know that."

"Have you tried it?"


"No, but we know them very well."

"How much do you think they know about you and your skills?"

"They hardly know about me and my skills."

"So do you about them. If they assume you are not skilled to do other kinds of testing and also assume you heard them say /We are sure, they don't know to test beyond functionality/, how would you react to it?"

"I would show to them that I can"

"How about giving them a chance to show that they too can change?"

Silence

Two things:

• Give each of your colleague a chance, explain to them your problem.
• Give yourself a chance, focus on your test coverage and explore a life beyond functionality.

Test effort estimation: Getting it wrong before getting it right

While the world thinks test estimation is a very tough thing to do, I think, everyone gets test estimation right. The point at which they realize their estimates were wrong is the time they get the estimation right. I wish they corrected themselves. Mostly, it remains unfulfilled.

So, the deal is to communicate the number when you know you are getting it right and buy time to get it right. By "right", I mean, more meaningful than you would have when you provided an estimate even without looking at the product.

I don't know what's wrong in going wrong the first time. Most of us learned to ride a bicycle, someone did it in a day and others took a month. How would you have answered a question, how long would you take to learn to ride a bicycle? Some people might need more time to get out of the fear of falling from it and others need time to get the balance right. It varies and hence cannot be determined before even someone got on to a bicycle?

Why testers need to learn to write code that works?

One of the things you thought I don't do often as a tester is - to write code. You are not completely wrong. I immersed myself all these days trying to be a tester who is /mostly/ black-boxish and interacts with the software through GUI. Did you read it as, "A black box tester doesn't need to write code?". Stop reading it that way! Read a sentence the way it is written and not the way you expect someone else to make a mistake that you like them to do.

You shouldn't be surprised to know that in the past I have written little tools, utilities and batch scripts that helped me or a test team I worked with. At the worst case I used to edit those scripts written by others to suit my needs or the mission. No matter how small they were, the value was the key.

Perl has been my favorite (you think I explored others enough, nah) since I first discovered it being used to automate checks in my first job. A couple of weeks back I decided to focus on shaping myself to be a tester who can write code in Perl for automating checks for most kinds of software I test. Irrespective of whether I am hired to do so or not, I'd want to be equipped.

In order to practice stuff that I learn in Perl, I decided to create exercises, puzzles and games for testers. That way I am trying to have more fun learning Perl. Perl itself is fun in its true nature and imagine adding more fun to it.

So wanna check out the puzzle I have for you? Hold on, don't be in a hurry.

Title 1: Investment plans for software testers. Title 2: Michael Bolton RST training in India

Is it a best practice that a post should have only one title? ;-)

Sharath Byregowda has won the Best Performer award at Mindtree. You know what it means to win the best performer award in an organization that has about 8000+ technical force. According to Mindtree there was a special guest who was invited to give away that award and that special person for them that day was me.

Sharath's manager, Murugan, wanted to make the award ceremony a very special one and surprised Sharath by bringing me in for the award ceremony. A manager so excited about giving away an award of Best Performer of the organization to his subordinate - made me feel wow. While traveling together to Mindtree office, I discovered that Murugan had a good diversity throughout his career and even tried doing business in the United States long ago with his friends. With all that experience, he thought, freedom plays a vital role in testing and hence provided it to Sharath who seeked it. India needs more Murugans. I think they have lot of Sharaths out there.

Freedom with responsibility made Sharath get him the award. This is the second time my student is getting an award at the organization level. Most of you might not know much about Shaham Yusuf but then he won awards for slogging important bugs and an unmatchable record of the highest number of important problems found in Deloitte India.