How Pradeep teaches software testing - Part 4

If you didn't know how Part 4 came up? Here is how it came. I first wrote Part 1, then Part 2 and then Part 3 ;-) So, logically here is Part 4 of the series How Pradeep teaches software testing.

"I know how to do certain types of testing but if someone asks me to explain what I did, I struggle to explain" was a problem statement I heard from a tester today. I replied in a confident tone, "I know why that happens. It doesn't just appear to be your problem but I know a lot of testers who appear to have the same problem." 

I continued, "You should consciously practice explaining what you did, first to yourself and then to your colleagues although they probably know how and why you did it. Why? I am going to explain how your brain works (or how I think it does). It has two nodes. One that contains what you know and the other that controls your explanation of what you know. When you make a conscious effort, you are forcing a connection between these nodes in your brain. When you force your brain to connect those two nodes too often, at some point it will judge the need for a permanent connection and create it for you. After that you have a free flow of what you know and your explanation of what you know." 

After I said the above, I could see a smile in the face that reflects, "Yes, I now know how to solve this problem". I read a person's understanding not by their head nodding or when I hear, "I get it", but by the emotions and expressions on their face and the body. 

James Bach identified that I was a metacog. I didn't know I was one. After that, it has been very helpful for me to understand why I do things the way I do. I guess I turned myself into a metacog because I thought it suits the kind of testing I wanted to. It's not a special status, it is just a way of life.

I don't even know if the brain works the way I explained it to be but I guess you can understand why the above explanation makes sense. I make sure I tell people that I am not trying to misinform them about anything when I use such examples. I am just helping the tester imagine why there is a problem and how to solve it. A lot of my coaching is consulting.

Examples like the one you read above govern a major part of my coaching. I observe a lot. I practiced consciously making connections between what I have seen, heard, thought, experienced and know to being explain it when the context demands.

Analogies and Examples are powerful approaches to teaching. I need to know what connects to my audience very well. Although all my audience are testers, I can't use the same analogies and examples, it just doesn't work. Bangalore testers need a different example than those in Pune. In Pune, I would talk about Raj Thackery and in Bangalore I would talk about Vattal Nagraj, in Chennai about Goundamani and not about Vattal Nagraj. Now, for those of my readers from United States or Europe, you wouldn't know Goundamani or Vattal Nagraj and hence I would use examples of Chuck Norris, Sarah Palin, Julian Assange. If you are a F1 fan, I'd talk about testing through specific GP incidents. That's how the examples need to adopt based on audience. I also use a lot of examples from what I think the world connects to. For instance, I closely read and watch Air Crash Investigation in Nat Geo channel. OMG! There is so much to learn from the way NTSB (National Transportation Safety Board) deals with it. So, basically as a coach, I have to keep connecting to various thoughts and happenings to be able to connect with my audience. 

Similes, Metaphor or if you choose to call them Analogies are interesting and tough piece of cake, if you are the one providing it. People tend to take it in their own interpretation than what you intend to. It is good in a way, I get to learn when not to use what type of analogies to what kind of people :)

I use Fishing for explaining test coverage. I know a few other people have already used fishing as an example for teaching testing, so I didn't invent it but I know how to explain different concepts of testing with the fishing example. I think that most learning happens when you are interacting with the audience and not when you are explaining. That's where I do better with the fishing analogy.

I start drawing different types of fish, from guppies to clowns and from star fish to sharks. I then draw a size and shape of a specific type of net to catch fish. I give them a goal of catching as many different fish as possible in a limited amount of time. Some audience come back and tell me, "Ah with this net the guppies will escape"... what's your immediate thought? You would tell them to build a net that has smaller squares to catch guppies?  (Why say it? They know it) I would probably be Pradeep and say, "Fantastic. A lot of time spent celebrating the success of finding one good bug steals away the opportunity to find more good bugs. How do you want to celebrate? 3 minutes left."

So, after they come out with strategy and different nets, I tell people that just because they have nets (tools) to catch a specific type of fish it doesn't mean they can really catch a plenty of it. I then tell a story and examples from my life. One of them: I consulted for an organization who had bought an expensive automation tool hoping that they would now be able to find more bugs. They spent all their energy to set up tests on it and found fewer bugs over the year. I drive points like: Tools don't help you unless you know how to help the tools to do what you think they can.

For a while people think it is possible to catch a shark from a net. It is possible, maybe. I explain how a powerful shark can bring their boat down if they try to catch it through a net or a fishing rod and how a harpoon is better than a net. Sometimes people try to think of similar approaches to solve many different problems. The example of shark, net and harpoons are cool for people to relate to something they have done in the past that shouldn't have been done the way they did it. I then equate different types of fish to different quality criteria. I ask my audience what type of fish are they mostly catching and I hear a shout, "Functionality". 

After a lot of back and forth between me and my audience, we all discover what good test coverage could mean. I then start probing into their projects and figure out how much of fish they are missing that they shouldn't be and try to help them understand why they shouldn't be catching too much of the same fish. Sometimes it upsets the food chain :) 

While there is fishing in my class, there is also plenty of room for Sine and Cosine for Test Techniques, Brian Marick's Minefield Analogy for Regression Test Strategy, Tom and Jerry examples for How Scripted Testing is dangerous, what lessons can we learn from Saurav Ganguly's come back, how to read Sachin and Kambli's career graphs, farming... It must be fun to sit in my class. I don't know, I have never been able to.

Happy New Year!

TASTRO - Tester's Astrology by Rrajesh Barde

One of Moolya's customer, a large IT company in India wanted me help a group of testers think beyond the boundaries they had accustomed to. They wanted me to help them bring out the potential and creativity of testers within their organization. Having thousands of testers,  they did know where to start from - the top 30 they had. Top 30 in terms of demonstrated passion for software testing.

The icing on the cake, they wanted their testers to progress towards becoming brainual testers.

A group of 30 testers were introduced to me and I spent time for a couple of weeks with them on various activities. We ended up doing so many good things together that we accomplished the mission together. There were plenty of great work that they did. Now, those 30 have been successful in inspiring 30 more and this chain reaction is appearing to happen. If it goes on, I am sure this organization is going to rock in the coming years.

Somebody impressed with how I mentored these 30 asked me, "Aren't you giving away all secrets of how testers in Moolya test?", to which I replied, "There is no secret. This is how we test and this is how we live". There is nothing to hide. We don't have any secret ingredient or a secret ingredient soup unlike many services companies. We have watched KungFu Panda and hope you too have watched it. We focus on our skills. Our website tells that story.

In this post, I want to highlight the creativity that came out of the exercises of Brainual Testing.
Rrajesh Barde surprised me that he had been reading my blogs ever since I started it and he told me he had also commented on it. I was glad to meet my oldest (well, he's pretty young) blog reader. The only question I asked him was, "Was it worth your time?"

This guy turned out to be hyper creative. He had a sense of humor, lateral thinking, passion to test, leadership and creativity. We were brainstorming of how do we educate testers without letting them know they are educated. Of course, books are boring to most. What content do we feed them with? We discussed on Andy Glover's Cartoons for it. However, that didn't solve the problem of testers within their organization being able to see Andy working with them.

So, Rrajesh Barde in the meeting interrupted, "If I may, I have an idea..." and then came out with this brilliant idea of TASTRO - Testers ASTROlogy. In a country like India, a lot of people refer to Astrology. They at least want to read if there is something good in it for them. I thought that was a brilliant idea and we had to develop it further. We needed to mix fun and pun into it. We needed the learning touch. We needed people to look forward to their weekly TASTRO.

Here is what we got:

TASTRO – Tester’s Astro – What do your test signs foretell?

Aries		Your stars look good for coming week however you might face an environment downtime. Why not make a quick checklist on how to set it up?
Taurus		Avoid calls during Rahu Kaal. Those who have calls with your on-site coordinators during this time, Beware!!
Gemini		The planetary movements suggest that the build scheduled on week day will be delivered to the Testing team on Friday after sunset. You have plenty of time to read Rapid Software Testing Appendix and practice new testing ideas.
Cancer		Though you think you know it, you have no certainty until you try. You might see a surprise appreciation for your smart work. Smart work could mean, you use oracles to test.
Leo		You are worried with unplanned work load. Read the book – Lessons Learned in Software Testing at home and you may see the change in office.
Virgo		Worried why you your productivity appears to have come down? How long has it been since you took a break? Quick breaks between test sessions are important.
Libra		In the busy times, be prepared to work late, eat pizza for dinner at work, and work for some weekends. Don't wanna do that? Go beyond test cases, you will find more bugs.
Scorpio		There is a chance that your relationship with developers would go sour in the coming week. So treat them with chocolates. Developers are rich source of information for a tester.
Sagittarius		Your customers would be under the influence of aggressive Mars. You would be forced to test whatever is thrown at you. Check for the mission to be achieved to avoid falling into traps
Capricorn		You would be trying to achieve the stars by clicking here and there with your monkey paws. Stop doing that and your career could get better.
Aquarius		Your managers would somehow have a strong notion that you have just been marking those test cases as “pass” without executing them. Honesty is important for a good tester.
Pisces		When you have crashed the software and waiting for the system to boot, prepare your own test idea cheat sheet. For those who do, future has been bright.

Isn't this awesome? I feel testers like Rrajesh Barde are a huge boon to our industry. The beauty of my consulting was, I felt there wasn't just one Rrajesh Barde I met but many. I may cover about others in future posts.

A couple of years ago, I used to go to a consulting assignment as though I am superior and I consult people because they were inferior. These days, I go to consulting to get humbled by people like the ones I met.

Please, everybody, stretch out your creativity, you would find an Andy Glover or Rrajesh Barde in you. For those who want to follow Rrajesh's blog, here is the link. He came out with another concept called Bug Burji (Burji is a dish made out of Egg and we call it Egg Burji, Rrajesh made a Bug Burji out of it). Rrajesh, you inspire me. I hope after reading your work, a couple of others may join me in admiring your work and contribution.

I am telling myself that I was born to witness this beginning of the golden era of software testing. Don't know if you can even see what I am experiencing.

Software testing black swan bites cause pain

This post is about an experience I had recently. An experience that proves to me that there are more hard working people than me and I shouldn't feel too proud of what I have achieved so far. 


I was at a conference recently. I was at several conferences, so don't get confused with what is shown in my Events page. This conference was not listed and doesn't have a conference page either. It happened in Bangalore on 2nd Nov, 2010, right after my return from Google Test Automation Conference - Hyderabad, India.


Due to my popularity, I was hoping people would come, recognize me and talk to me about their testing problems. It just didn't happen. Some other person was getting all attention and there were people surrounding him and asking questions. There were so many people around him that I couldn't get a chance to see the man who was getting all this attention. I was wondering who is this guy getting an edge over me in India?


I then thought it must be someone who traveled to India from a western country. Only then people forget their local guy. Nothing new to me. I was just waiting for my chance to shake hands with that guy and talk to him about testing. Well, I just wanted to know what makes him so special that he is getting all attention in India. 


Check at my stats, I am supposed to be the popular guy out here. If you are seeing the popularity hungry Pradeep now, I must say; I too saw him.  The difference between us is, you might feel ashamed of having known Pradeep and I don't.


An hour passed by and still I could not get to meet that guy. I saw a tester coming out and asked her, 
"What's about the guy in center?" 
"Oh, you don't know him? He is an expert in test estimation"
"Everybody in here is an expert, din't you know that? So which country is he from?" 
"India" she said. 


What? an Indian? and I don't know him yet? I know everybody who blogs from India. At least, everybody who blogs from India knows me. How come I don't know about this guy who seems to be more popular than me? Maybe he doesn't blog but even then I should have known him.


All this was driving me crazy. Added to that were some of the talks I wanted to attend and it had started. I just pushed myself into one of the track hoping I could catch hold of that guy at one of the lunch tables. I have never waited for lunch so much, not even when I was very hungry. I wanted to meet this guy. He was a challenge I wanted to face.


At lunch, same bloody thing that happened in the morning repeated. People surrounded by him and I just cant get to meet him. My ego hurts me a lot if I have to go introduce myself to him amidst other testers who might think that I am not as popular as they thought. So, I picked up a plate and tried to eat alone. Fortunately, some testers who couldn't get to talk to him saw me and approached to have a conversation. My mind was somewhere else. I guess I don't know if I did answer the questions those testers asked me. Maybe they would have stopped reading my blog as I don't know how crazy my answers were. I just wanted to meet that guy.


Finally the moment arrived. The only way I could corner him was in the washroom. I was waiting for him there adjusting my shirt and trousers making it look to other people as though I care too much about my how neatly my shirt is tucked in. There he came. I didn't mind if his hands were wet but just put my hands forward and said, "Hi, I am Pradeep Soundararajan". He shook hands with confidence and said, "Oh, I know you. I read your blog and follow your work closely". 


At one end, I felt happy that the man who was sought much more than me follows my work but it was still aching as to how this guy managed to be the center of attraction amidst my presence. I took courage and asked him, "How come I don't know you. What's so special about you that these people are flogging you?"


"I have learned to help people estimate their work in a way that makes them feel successful following my advice" he said that with a soft and gentle tone. I put a step towards the door closeby and turned to him and said, "Why don't we discuss this off the washroom?"


My intention was to steal the idea. After several years of hard work, I can't allow someone to steal away the limelight I have been enjoying. When I say I wanted to steal, I mean, I wanted to know what his education was. How different was it from mine? 


We sat on a couch and I asked a question that was designed for deception or to learn about what he has learned


"So, what's your source of learning?"
He had a smile on his face before he said, "I read Bach, Bolton, Kaner, Jerry and you"
"Sounds interesting. I do the same too but how come you seem to be doing better than me?"
"I don't know"
I was pissed off but couldn't let it out because I still hadn't got the secret out of him. 
"So, you are suggesting that you learn something more from them than me?"
"No, I haven't met them Pradeep and they don't know about me"
"Pretty sure because if they did know, they would have let me know about you"
"Yeah"
"So, let me stop beating around the bush. How do you help people with their estimation problems?"
"I do it ............................................................................... this way"
"Wow. That's cool"
"Where did you learn that?"
"You are so humble Pradeep. You have read it, too. I picked up ideas from Michael Bolton's Test Estimation & Black Swan series of posts, experimented with them, made my own notes, refined them for a while to arrive at this point" and then he walked away saying he had to deliver a talk and it was getting late.


If you had been to the conference, you would have seen me crying on the couch post that meeting. I wasn't crying because someone gained an edge over me but was crying that I learnt the cost of not reading those lengthy posts just because it was lengthy. 


I finished crying and went around looking for that person to thank him for the lesson he offered. He had already left. The series of posts from Michael Bolton on Test Estimation & Black Swans had been lying there on his blog and I just kept feeling lazy to not read those lengthy posts. I am probably in the Twitter Era. I want people to say anything great or stupid in 140 characters and I also hope they say that around my timeline. 


Walked around with disappointment. I decided to go out of the conference venue. I wanted to go home, have a drink and get a tight sleep to forget all this. I thought I was reading everything by Michael Bolton. When he posted about estimation, I thought I had already read enough of estimation from him and he was packaging the same stuff. Just then, the conference was getting over and the final lightening talk was mine. I was called to the stage. I went on the stage with tears still dropping at 1 drop every 10 seconds, forgot about my talk and asked the audience in a shrill voice, "Have you people read the series of post on Test Estimation & Black Swans from Michael Bolton?". 


The responses were, "Its lengthy", "We didn't find time to go through it", "I got a call in between and almost forgot to continue", "I was too busy with my project". Almost everyone were saying the same thing, "No, I didn't read it because it was lengthy". I laughed out loud and walked away as though I had seen myself in hundreds of mirrors placed in front of me. There are so many Pradeep's in our industry. Some Pradeep might not even have got this far on this post because he might have thought, "Oh, this is lengthy". 


If you don't look like Pradeep when you stand in the mirror, here are the posts:


Project Estimation and Black Swans (Part 1) 
Project Estimation and Black Swans (Part 2)
Project Estimation and Black Swans (Part 3)
Project Estimation and Black Swans (Part 4)
Project Estimation and Black Swans (Part 5): Test Estimation


The next time you don't want to read a post just because its lengthy, remind yourself that if you miss spotting the Black Swan, it doesn't mean Black Swans don't exist.  They bite hard to remind you that they existed and you didn't pay attention to them.

Heart of a tester

In 1954, when software testing was just about taking birth, there were two groups that started to form. I was as curious as you are right now, to know what those two groups stood for. One of the groups christened as, "Kuzusu", had a thought that good testing would reduce the number of billable hours to deliver a good enough product and hence had to be avoided. The other group christened, "Shidachi", stood for good testing that can save a lot of stakeholders time and money to deliver a good enough product.

Things started getting hostile. People from the two groups tried killing each other. That's how much they believed in the group they claimed to represent. One or two people being killed once in a while didn't make a difference.  Just a few days before the Christmas that year, there was a huge battle and at least five dozen people from both groups were killed. That's when a lot of people started to propose a treaty and cease fire between the two groups. On 3rd March, 1955, the famous treaty in history was written. "Treaty of brain fight instead of blood", was signed by all members of those two groups in front of the President of United States, South Africa, India, China, Japan, Sweden and Britain. 

Signing the treaty meant; anyone who violates the treaty shall be killed by their respective country administration. The country where many such killings are observed might lose its eligibility to get outsourced testing projects. There was a twist to the treaty. It not just included the names of those who signed the treaty but all future software testers were presumed to have signed on it.

The first meeting of the Kuzusu group after the treaty was held in 8th October, 1956 in an undisclosed location. The outcome of that meeting was; more the number of people on Kuzusu side, the easier it is to dominate the Shidachis.

While Kuzusus were still at the meeting, Shidachis had a plan for the future. The plan was pretty simple; to discover, invent, learn, practice and demonstrate thinking skills that would lead to better testing and better product. So, Kuzusus were focused on Head Count in their group and Shidachis were focused on Brain count in their group.

The common thing they realized through a series of meetings is that they needed to lose their identity and mingle among future software testers. All websites, boards, banners, ads, real estate, documents, registration, news paper citing, notes, floppy disks, photographs and tapes were destroyed. The only thing existing to prove about the existence of these two groups is the treaty document. The next time you visit National Meuseum of American History - Kenneth E Berhing Center, Washington DC, make sure you see beyond certain wall posters there. One with a sword marked on the right bottom corner has the treaty document in its back. 

Ever since 1957, these two groups started to influence the upcoming generation of software testers with their principles but not in the name of Kuzusu or Shidachi. We never know if these groups are continuing to meet and make new strategies to fight each other. We may never know that. 

After so many years, we don't even know if Kuzusus and Shidachis are the ones with whom we interact everyday at our work. They are dressed up as our colleagues and friends in our industry. We don't even know if we are the channel or follower of one of these groups. Obviously, it is nearly impossible to identify or differentiate between a modern day Kuzusu and Shidachi, because they all appear like one, doing testing and wanting to do it better.  

We all want Shidachi group to win. Even Kuzusus would publicly speak about wanting the Shidachis to win and that is how they can remain camouflaged. Another twist to the story happened in July 2006 when a new group was formed by someone somewhere. This group didn't want to give themselves a name and remained anonymous from first. Their objective is; we don't care if good or bad testing is happening as long as we are getting paid, hikes and promotions as regularly as possible.

That's it. It has become all the more tough for the Shidachis to strike large success. You just can't say that you belong to the fourth group who doesn't believe in all this. You are already one among the three. The only problem is, it is hard to know which group you actually belong to. You might have thought while reading this that you belong to Shidachi and the people with whom you work with are Kuzusus. That's exactly how the Kuzusus want you to think because that's the way they do too. Finally, to an outsider, you and the people with whom you work might appear as the third group who doesn't care about good or bad testing as long as you are being paid.

So, the quest for the current and future generation software testers is not about identifying which group they belong to but to work with each other to win hearts. The fight of the brain is as important as the love of the heart. The first organ developed in a mother's womb is the heart and we shouldn't be ignoring it in our fight of the brains.

So, dear reader, whoever you are, if I have said things to you in the past either in this blog or in forum discussions that offended you or made you feel hurt, please forgive me. Help me to be of help to you in future. Its time we consider winning each others heart and brain. 

Also welcome to the Hridaya group!

Experience report of testing versus checking

Many thanks to my client in Bangalore who encouraged me to blog about the experience, value and outcomes we had in separating tests and checks. If you are reading this, it means my client has approved this for publishing.  

Thanks to Michael Bolton who posted a series of posts on his blog under Testing versus Checking and those who commented on the posts, whose ideas and thoughts helped me to think about delivering the value of tests and checks to my clients. Tests and checks series of posts has helped me communicate things better and organize my thoughts better. We were using the term "check" and "test" much before Michael blogged about it; however, he gave a cognitive structure to it in our brains.

The context

One fine evening, there came a client looking for me and the story goes like this... 

I was hired for a couple of weeks, to consult and test various products produced by a large organization whose headquarters is in Europe. The moment I was hired, there was a need of a hand to run through a few tests on a product. The product was used for analysis of something across many country deployments. It has an engine loaded with business rules, lots of history data in a database and report viewer application. How do we know if the business rule engine didn't violate any rule? The reports had to comply with complex business rules that at the first sight appeared to me as contradicting one another. The core development of the product was being done by a vendor in the US of A, who was reputed within my client's office for pumping in poor builds, as frequent as they could. I wish I could tell you more than that.

Video test cases instead of lame text

On starting off, I realized that the learning curve on the project appeared to be steep and I couldn't cope with it in the time I had. A demo was given to me from a colleague who had been running tests ever since he was hired. Does a demo suffice? No way. 

Building a little credibility with that colleague helped me convince him to record videos of the demo and tests with the help of Windows Media Encoder 9.0 for Windows XP. So rather than following dumb scripted tests, I was watching a video on one monitor and running tests on the other. Video test cases were more reliable than documented ones. Based on how quickly I picked up things, the videos are now being planned as training material instead of asking someone to go through test cases. 

Recognizing checks being called as tests

On running them, I realized, what I had been running were checks than tests. To quote Michael Bolton, Checks are machine dependable and tests require sapience. I was checking if the report complied with all business rules. While I was checking, I instantly knew that these checks could be automated.

Testability, Test Coverage Issues & Stakeholder Interests

I noticed a couple of bugs in the software were preventing me to run other kinds of tests as fast as I thought I could. Anything that blocks me or slows down my testing is a serious issue for me and to those who have hired me. With the proposal to test for various quality criteria being accepted, I made a presentation to the stakeholders of how the quality can be improved at least with respect to testability and other quality criteria (such as Usability, Performance, Security.. ) accompanied by a test report.

They instantly liked the improvement of testability idea because to check if the business rules are not violated for every installation, they had been taking 2 - 3 days. They needed rapid feedback to take better decisions.

Imagine looking for a specific value across many excel sheets. By the time you are on the fifth, you might almost forget what you saw on the third and where you saw it. However other kinds of quality criteria didn't matter to the stakeholders for their current context. I wish they had considered it but that's a business decision. My bug advocacy was strong though. I had stakeholders wanting to automate the checking part of our testing. That was a winner.

Partnering with developers

I was sitting in an organization whose policies and people are new to me. I was proposing things that required lot of dependencies on other people in functions such as development, business analysts and on-site personnel with of course the management. The time frame I required to achieve this was really short. Writing emails in a manner that helped the people realize the importance of the value was a big winner for me. For instance, here is an email I sent to the development team that sits in my client's location who mostly do some integration and writing wrappers.

Dear Developers,

Greetings!

I am glad to be interacting with you. As we are committed to reduce the time we take to analyze a release of XXX XXX, we realize that a tremendous saving to the organization can be contributed by making a few changes. We have made a few things from our end, such as, having a formula do the job for learning about XXX-XXX rule being satisfied for all installations. Creating a repository of possible combination of tests we need to do to be able to say we have exercised the business rules very well.

In that path, we realize that with your help, we can do a lot more. I understand that you might be busy with other things on the table but as I feel we are working together towards a common goal, it makes me request you to help us.

We request the following things from you:

• Currently we are spending about 3 – 4 hours manually to verify the XXXXXX business case for a small installation.  
• We have investigated and conjecture that if you help us have the XXXXXX ( such as XXX or XXX details ) to be available in a list associated with corresponding XXXXX and the XXXXX, it would take us less than 5 minutes for us to perform the business rule validation.  
• If these details are a part of the export of Weekly Report (highly preferable) or as a separate excel list (less preferable) it would help us achieve the goal.  
• Assuming a set of testers will be testing XXXXXX for all country implementation in future, you would be helping us bring down the cost of testing by a couple of thousand dollars ( Well, Euros, as we are European based )  
• Additionally, as we are doing it manually, we are not exceeding more than 10 categories per installation. With your help we would have the capability to go as much as the number of categories the system can support. Thereby, we test for larger samples and increase our data coverage.

We would be in a position to provide you any help you might need from us in achieving this task or would be glad to offer back any other help that you may require to make your work more effective.

Thanks & Regards,

-- Pradeep Soundararajan

30 minutes, no response to the email. At the 31st minute, a developer was at our desk. It happens that he was serving his last day for the organization and carved out time for us. I considered that as a great gesture by a developer to support test team and our test team brought a chocolate for the help he offered. I couldn't get a "Thank you" card; else I would have wished to have given that as well.


That developer helped us with stored procedures and SQL queries that we need to extract data out of the db. 


The Checker Tool Development & Testing

So, instead of looking through reports exported through the GUI, we now had the option to extract values directly from the database. We got one step closer, to what we wanted to achieve.

While I was planning to write a Perl script to extract the values and dump it to an excel sheet, I became aware of a developer in test who joined the organization and had not yet been assigned to a project. The good news for me was she hadn't got her computer to start working. I requested the manager to get her time for helping us write a script and gave her my PC for 2 days while I mostly roamed around saying "Hi" to people around and getting a hang of the culture and talking to people about their work. I could have done all by myself using Perl but I know how slow I'd be as compared to a developer in test.

I provided her with requirements and all necessary tools that she'd need to develop and test the script that could dump the values in an excel sheet in a format we wanted. It's a complex data structure and in order to perform an analysis we had to develop a report format. With the help of my colleague, we zeroed in on the report format and in 2 days time, she could help us see what we wanted in the report format.


TDD for writing Excel Formulas

Off to validating business rules through Formulas in Excel. I knew the power of Excel and its formulas but I didn't know its limitations. I remembered that I had received an email long back from someone containing  MS Excel formula dictionary as an attachment. I pulled that out to help myself.

Excel shouted a warning and error if we exceeded 7 nested loops. It didn't warn us when we had a typo in one of the formula, a star * symbol instead of 8. Copy pasting formulas from one sheet to another had reference to the previous sheet. I know lot more about excel than what I knew earlier. 


I wasn't doing a good job and I knew it very well. A month and a half back I attended a demo on TDD from Venkat Subramaniam. It struck me at the right time. Instead of continuing to do a mediocre job, I started collecting data, scenarios and information that should make the formula fail or pass. I started to write the excel formulas based on the data we had. My colleague who had been in this project for a while helped me get the set of data that had to result in pass and those that should result in fail. If you may permit me to say that writing formulas as, "coding"  then we were pair coding.


This was very helpful. After starting with that approach, we could find bugs in previously written code. Prior to the TDD and paid coding approach, I wrote a formula, loaded the data and checked to see if it failed or passed. For a condition in which it should pass, if it failed, I used to tweak the formula to see a Pass. That was dumb, especially, after I knew that making a Pass could impact other things that were supposed to fail. 


With TDD in, we achieved faster progress on our business validation formulas. We were now at a speed lane. We cracked the puzzle too. We could check for business rules violation in a matter of few seconds.


Comparing humans and tools for speed

I am against the idea of comparing human testers with test automation. I still am against the idea but I saw myself in a situation where I was comparing automation with manual effort. Why did I do this? Should I remain silent about what I have done and continue telling people who attend my workshop that; comparing a human and a tool for speed is a bad idea? I was reminded of James Bach's blog post "Manual tests cannot be automated" and Jonathan Kohl's interview where they put a lot of things so well that I couldn't have put it then.

In self retrospection, a learning emerged. We didn't automate tests, we automated checks. As humans who were not supposed to be doing checks were doing it, I was forced to make that comparison. In fact, I should have said, "Please do not see this as a faster way to do things by comparing how testers were doing it earlier. They weren't even supposed to be doing that. I just wish there was more freedom and a little bit more authority for them to express it".
  
Getting our tool tested by domain experts

"Yipee! It works". That's not what we thought. When we developed this tool, we were sure that the users of this tool could be beyond the test team and how good our tool was to be decided by a "domain expert". When we showed some of our unit test results to a business analyst in house, he was excited at how simple this would be for him to use when he goes to assess a country installation in the future. He vowed to support all that we needed. He offered help writing us a few formulas that we needed. I tweaked it to suit our needs. 


He had a logic that I wouldn't have touched upon. So, that really helped. We didn't need to be a domain expert all by ourselves to be able to do this. A testing brain and a domain expert brain coupled together yielded a lot of value.

Added to that, he was a key stakeholder we had in house. He acted like a customer in house. He suggested the changes we needed to make it more suitable to other stakeholders' usage. Having been on site, he could bring the perspectives of people who could use this tool and their contexts. We needed lot of clarification on the business rules. Imagine a set of rules contradicting each other but still plausible under certain contexts which had to be shown as pass and not fail. Whoof! I liked it :)




Live Testing of Business Rules Validations Checker Tool V 1.0

So, our in house customer was happy with it. We showed a demo to the test management and they were excited. The test manager was excited enough that he pulled out the product manager out of his busy schedule to show our tool to him. The product manager was about to be in a meeting and he asked us, "How long this would take?", because he knew the business rules was taking a few days. When he asked us that question, we initiated the script, took a deep breath and said, "It's over". We showed him the report and there was a smile that our test manager was hoping to see. The only skepticism he had is about business analyst's approval of this tool. A bug in this tool would really misinform the decision makers.

So, the tool was shipped to people in Europe who tested it on the live system. They took about 2 days to get back. It wasn't a very nervous moment at all for me. I was just impatient. They had to take two days to approve this tool because they had to compare the results of the tool against their manual checks. While we were waiting for the reply, we continued testing our tool to see if we could spot problems. We ran it for several test installations and we made notes on how we could improve but no bugs found. 


The climax

On the next day when I was traveling to my client's location by bus, I get a call from a person who heads the testing for the group which I was hired. "Are you in the bus? Can you get down? I want to talk to you. I will come there soon". Before I could ask anything he disconnected the call.

That's really crazy. I suspected something must have gone wrong. I thought the Head of Testing wants to give me a bad news that my consulting contracted is terminated. He arrived at the place, I got in, and he showed me an email and said, "Congratulations man. This is great news". The tool was approved by the business analysts who were on site in country installations. They said it was accurate and also proposed some modifications to a dashboard we had created in the excel. It hit the target on the bulls eye. 


The Head of Testing and I together prepared a presentation of the whole story to circulate it throughout the organization. The Head of Testing currently hopes the concept of separating tests and checks is carried over across all other projects. 

At one side I was extremely happy for being such a value to my client and on the other hand, I was telling myself, "Consultants like me are made to look good for doing what everyone else is supposed to be doing". Had my colleagues at the client location been people who were active in learning, I wouldn't have sounded like an expert tester to them. I just feel pity and can offer the help I can. I have to move on and learn more things.


Preventing bad builds from sneaking in

The story doesn't end there. I had another proposal up my sleeve. I wanted to ship this tool to the vendor in US who was pumping in bad builds. We are planning to use the Business Rules Validation Checker Tool as acceptance checking of a build into the client's location. The idea of automated acceptance checking appears to be interesting to my client since they work with several vendors, delivering broken builds not as frequent as the vendor stated above :)


Return on Investment, in its true sense.

The ROI is not about the speed with which a certain task is done. The actual ROI is that the testing team time is now freed from checks to do exploratory testing, bettering the test coverage, faster feedback, better informed decisions, focusing on things that they were not focusing on AND preventing bad builds from entering the client's location.

I thank my colleagues (nicknamed) Rag and Tham for the support they offered me in the three weeks. Without a management providing freedom to a tester like me, this wouldn't have happened. So, thank you Mr. Head of Testing, Test Managers and Business Analysts for all the support and for approving this blog post.


The only thing pending is a party to celebrate. Mr. Head of Testing, are you reading this? :-)

Usability of usability & performance of performance

Hiya there. My name is Pusher V9.3.4 and I am a web application software living in thousands of computers worldwide. I am twenty four years old. I was born to help humans transfer data from one secured location to another and to provide them with a report of all transfer they have done in the past. I have traveled around the world and for every economic meltdown, I was moved from one country to another. I don't do anything different from what I am programmed to do. However, in several situations, humans using me make conjectures that I do things I am not supposed to.

For twenty four years, I have been silent, mostly because software applications like me need to listen to Windows XP and Internet Explorer for our survival. Today, I was transferring some data for one of the human and just peeped in to see what is he browsing. I hit upon this blog and read the words "Some birds aren't meant to be caged" and instantly decided that I belong to that category. I feel happy to have broken my silence.

This would also be the first time I am going to be doing something that I am not programmed to do. I started writing this post when Chandok from Turkey hit the "Generate Report" button. I am aware that I am running a risk of being kicked out of CoWAP (Community of Web Applications) for violating the rules. I care a damn, I want to be of value to you.

I don't plan to take a lot of your time. I want to tell you a story that I witnessed and was forcibly a part of. So, here it goes...

As the number of computers grew so did the number of users. To those who were developing and testing me, it meant they had to help me handle thousand times more data than I ever did an year back. Someone came up with a plan to test my limits and I could hear a term, "performance testing" being used very often. I don't have any physical feelings like what humans have. To me, if you do a load or functional or stress test, its all the same. I won't empathize with a user when he is stressed. I am not programmed to do that, mind it.

So, one fine day, the product manager came up with a list of response times that I should possess for a set of actions that humans are likely to perform on me. I was wanting to know how good I am programmed to handle complex things quickly. The tests were planned and executed. 5 seconds was the target for a report generation. At that moment, I could generate a report in a little more than 2 minutes. So, there had to be a lot of tuning done on me. After five months of effort, they got me to perform the report generation in 10 seconds. Considering my earlier performance, they thought they had achieved something phenomenal. After they made necessary changes to the hardware on which I was hosted, they could see a further drop and I was generating reports in 3 seconds. To test if I would scale for future, they added about 10,000 more concurrent users performing complex operations while reports were being generated. I was varying somewhere between 8-12 seconds. That night they had a party. Only I knew why they shouldn't have had the party.

About 7.2.1 versions later or in other words, 11 years later, how long do you think the user takes to generate a report? 12 minutes.

While you might start suspecting that the reason behind 12 minutes is due to a highly expanded user base, I still take 5 seconds to generate a report but the user takes 12 minutes.

"How can that be?", is that what you are asking, too?  The math that you have learned is of actually little help in such situations. Allow me to explain that to you.


Due to changing requirements, the report generation which was a one click action was turned to a multiple entry and then do one click action in V6.3.1. So, a human has to input several things before he can actually do that one click on me, to see his data transfer report. For instance, he needs to input the date range, the kind of report he wants, the format in which he wants the report, the email id to mail the report, a couple of check boxes, huh, half a page of mandatory fields and rest half of optional fields.

Here is the scenario of 12 minutes : Most users enter incorrect values. I don't have a problem. Its not my problem if I crash. I want to be of help but if I am programmed not to be of help, I just do what I have to. When a user enters incorrect values in some fields, he is shown an error only for the first field and then he goes makes a correction, hits the "Generate Report" button. I then show him that another field has an incorrect value. He corrects that and then hits the button again. I then show him that the third field has gone wrong and force him to correct it. Like this I do for one full page of fields and check boxes. When everything filled in there appears to be right, I am forced to show a warning that a huge file will be emailed to the mail id given and ask them if they still want to go ahead with it. Finally, the human gets to hit the "Generate Report" button and I get into the branch of code that takes just 5 seconds to generate the report.

"Is that a problem?", you may ask. Trust me, those who developed and tested me believes it is not because they are not aware that the users are taking so much of time to generate a report. "How come?", again, you may ask. Simple, I have a page whose title is "Report Problems / Feedback", many thousands of users click on it and see a couple of fields and check boxes. I just giggle when they close me after seeing that page. That's what you might want to call as "Checkmate" if you are chess player.

I just wish I could have told those humans developing or testing me that, not paying attention to usability and focusing on performance would make them poor performing software professionals. I also wish I could tell them that the metrics they were collecting and the way they were drawing inferences was the best way to fool themselves and the people around them.

Even if I said that, would anyone bother to listen to me? No way. Am I bothered that they wont listen to me? No way.


Yikes, time to go back as it looks like the Chandok has finally passed through all the fields and check boxes.  Bye!