First attempt at Test Consulting : An experience report

_ my post in software-testing@yahoogroups.com a group owned by James Bach and Cem Kaner _

A start up company in Bangalore who were impressed by my blog Tester Tested , invited me to test their products and train testers. I decided to do it in parallel.

Here goes some notes ...

I reached the place around 10:30 AM on Nov 4th, got a brief introduction to the company and the team. I went through the documents they gave me pertaining to the products I was supposed to test. The documents were not in detail but gave a general overview and yet I identified some key areas to test. I talked to the testers and developers to know the current state of the project and the need for calling me. That's like quickly getting to know the context.

I tried getting my mission as quick as possible and succeeded to get it at 11:15 AM. I wrote down the mission and asked them to check if I went wrong in understanding it.

Next move, I approached the tester and asked him to show me a demo of their product. I started digging for more info by asking questioning the testing. I found that the testers there had no clue what testing was and that is not surprising. Plus, they were sitting close to the developers that helped them to be biased.. "I saw it working on the developer's PC, so I did not test that" were the words from a tester. :)

The software is supposed to be sold to schools and colleges for the management to facilitate all the paper work to go online.

Some features of the software are

1. Admission process
2. Student attendance records
3. Library Management
4. Question paper generation

• I questioned the need of every feature , every radio button, every format, every assumption from the developers, everything that I could see, everything that I could not see too..
• The application was written in Java, and uses a Tomcat server to access the database ( MS Access ).

Two amongst many other interesting bugs I found helped me arrive at better informed decision about the product were.

1. The administrator was able to delete his own account details and whoosh.. There is no other power user than administrator, so the application became useless.

2. There was a "book search" option in Library Management, The field was not accepting more than a word, a developer's countered his assumption that it is a feature which makes the search simple for students to search for "java" , "testing". My counter counter argument was "If there is a book on testing which has a title "The craft of making good software", will a search on "testing" reveal this book?

• This provoked me to make the developer list out all assumptions he had made to develop the software. I started testing all his assumptions and without surprise, most of his assumptions deviated the software from its purpose.
• Every few minutes once, I kept saying to the testers there "Do not accept if I say something is wrong, I encourage you to argue with me on the point I say". ( The secret of getting better as a tester, learnt from James Bach)
• It was around 1:45 and we all dispersed as it was late for lunch. 
• 2:15, I am back, with a higher level of energy after lunch since I wanted to dig the software much deep.
• I do not know what discussions the developers had during lunch but there were more people joining to see me test and discuss the issues. It gave me more confidence, since I made an assumption they were interested to see " how is this guy able to dig all this?"( Sometimes it is better to motivate yourselves, as a tester, by saying this "wow, developers too are interested to listen to me" )
• I added great humour now and then and made the developer laugh at his own mistake than getting disappointed/frustrated in front of his seniors, internally felt "wow pradeep, you seem to be a good manager too"
• I also helped them see the product in a number of ways, they might have not thought. I developed a good eye to eye contact with the 10 people I was surrounded by and whenever someone said "yes", I was convinced that they said "yes" from heart since their eyes too had the same expression.
• Well as we were so involved with the discussion on issues and finding new bugs, we were surprised the bug count had gone beyond 50 or 70 after a while by around 4 O clock and out of them, many show stoppers.
• I showed them at least one database corruption, tomcat server crash, multiple application crashes, multiple cosmetic bugs, requirements missing, design defects, multiple wrong assumptions.
• At around 7 PM, I stopped by re-stating the mission I had and how much of it I felt, I finished. I also requested for feedback from them on their views on my results v/s mission objectives.
• I motivated all the developers to challenge me the next time I drop in. I motivated the testers to challenge me and not even let the developers say "I can challenge pradeep".
• At 7 25 PM: "Pradeep ... Pradeep" a voice came out from a guy who seemed to be chasing me from another bike. I stopped my bike to discover he was one of the developer who followed me for a distance. After we stopped he said, "I was impressed and never knew testing was so creative, I want to shift to testing"
• He asked for the books he has to read and I suggested the two books I feel are bible for starters. Of course, the two books are Learning Software Testing - Dr Cem Kaner, Jack Falck, Hung Ngyuen and Lessons learnt in software testing - Dr Cem Kaner, James Bach and Bret Pettichord.

I was very happy of having done a good enough job. I felt the following points could make my guru James Bach happy of having coached me for free ( I could not have afforded had he charged me :) ) 

I felt that 

• I asked good questions that reveal great information about the current state of the product, its quality. Got my mission. 
• Collected enough evidence to prove a bug. 
• Talked specific. 
• Recovered from my own mistakes rapidly. 
• Learnt the product at its breadth in a short while.
• Performed exploratory testing better than my previous attempts.
• And said to myself, "This is still the beginning...".

I got a message today morning (Nov 5th) from one of the developer "I am still wondering; how you were able to make the Tomcat disappear?" and the company has decided to re design the whole software and are going to invite me for another round of testing. ( Of course they would invite me, I did not charge them anything )

I am sure, I am not yet as good as most of you here are but I am happy that I am in the same road you traveled sometime back and I see you far far away. Miles to go before someone sees me as far as I am seeing you all today.

Many thanks to people like Dr Cem Kaner, Michael Bolton, Mike Kelly, James Bach, Johanna Rothman, Jerry Weinberg, Jon Bach, Simon Fields, and all of you, who have inspired me in one or many ways, to put in lot of hard work and develop myself as a better tester each day. I wish to work more hard and hence keep inspiring me.