"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Thursday, November 09, 2006

First attempt at Test Consulting : An experience report

_ my post in software-testing@yahoogroups.com a group owned by James Bach and Cem Kaner _

A start up company in Bangalore who were impressed by my blog
Tester Tested , invited me to test their products and train testers. I decided to do it in parallel.

Here goes some notes ...

I reached the place around 10:30 AM on Nov 4th, got a brief introduction to the company and the team. I went through the documents they gave me pertaining to the products I was supposed to test. The documents were not in detail but gave a general overview and yet I identified some key areas to test. I talked to the testers and developers to know the current state of the project and the need for calling me. That's like quickly getting to know the context.

I tried getting my mission as quick as possible and succeeded to get it at 11:15 AM. I wrote down the mission and asked them to check if I went wrong in understanding it.

Next move, I approached the tester and asked him to show me a demo of their product. I started digging for more info by asking questioning the testing. I found that the testers there had no clue what testing was and that is not surprising. Plus, they were sitting close to the developers that helped them to be biased.. "I saw it working on the developer's PC, so I did not test that" were the words from a tester. :)

The software is supposed to be sold to schools and colleges for the management to facilitate all the paper work to go online.

Some features of the software are

1. Admission process
2. Student attendance records
3. Library Management
4. Question paper generation


  • I questioned the need of every feature , every radio button, every format, every assumption from the developers, everything that I could see, everything that I could not see too..
  • The application was written in Java, and uses a Tomcat server to access the database ( MS Access ).

Two amongst many other interesting bugs I found helped me arrive at better informed decision about the product were.

1. The administrator was able to delete his own account details and whoosh.. There is no other power user than administrator, so the application became useless.

2. There was a "book search" option in Library Management, The field was not accepting more than a word, a developer's countered his assumption that it is a feature which makes the search simple for students to search for "java" , "testing". My counter counter argument was "If there is a book on testing which has a title "The craft of making good software", will a search on "testing" reveal this book?

  • This provoked me to make the developer list out all assumptions he had made to develop the software. I started testing all his assumptions and without surprise, most of his assumptions deviated the software from its purpose.
  • Every few minutes once, I kept saying to the testers there "Do not accept if I say something is wrong, I encourage you to argue with me on the point I say". ( The secret of getting better as a tester, learnt from James Bach)
  • It was around 1:45 and we all dispersed as it was late for lunch. 
  • 2:15, I am back, with a higher level of energy after lunch since I wanted to dig the software much deep.
  • I do not know what discussions the developers had during lunch but there were more people joining to see me test and discuss the issues. It gave me more confidence, since I made an assumption they were interested to see " how is this guy able to dig all this?"( Sometimes it is better to motivate yourselves, as a tester, by saying this "wow, developers too are interested to listen to me" )
  • I added great humour now and then and made the developer laugh at his own mistake than getting disappointed/frustrated in front of his seniors, internally felt "wow pradeep, you seem to be a good manager too"
  • I also helped them see the product in a number of ways, they might have not thought. I developed a good eye to eye contact with the 10 people I was surrounded by and whenever someone said "yes", I was convinced that they said "yes" from heart since their eyes too had the same expression.
  • Well as we were so involved with the discussion on issues and finding new bugs, we were surprised the bug count had gone beyond 50 or 70 after a while by around 4 O clock and out of them, many show stoppers.
  • I showed them at least one database corruption, tomcat server crash, multiple application crashes, multiple cosmetic bugs, requirements missing, design defects, multiple wrong assumptions.
  • At around 7 PM, I stopped by re-stating the mission I had and how much of it I felt, I finished. I also requested for feedback from them on their views on my results v/s mission objectives.
  • I motivated all the developers to challenge me the next time I drop in. I motivated the testers to challenge me and not even let the developers say "I can challenge pradeep".
  • At 7 25 PM: "Pradeep ... Pradeep" a voice came out from a guy who seemed to be chasing me from another bike. I stopped my bike to discover he was one of the developer who followed me for a distance. After we stopped he said, "I was impressed and never knew testing was so creative, I want to shift to testing"
  • He asked for the books he has to read and I suggested the two books I feel are bible for starters. Of course, the two books are Learning Software Testing - Dr Cem Kaner, Jack Falck, Hung Ngyuen and Lessons learnt in software testing - Dr Cem Kaner, James Bach and Bret Pettichord.

I was very happy of having done a good enough job. I felt the following points could make my guru James Bach happy of having coached me for free ( I could not have afforded had he charged me :) ) 


I felt that 

  • I asked good questions that reveal great information about the current state of the product, its quality. Got my mission. 
  • Collected enough evidence to prove a bug. 
  • Talked specific. 
  • Recovered from my own mistakes rapidly. 
  • Learnt the product at its breadth in a short while.
  • Performed exploratory testing better than my previous attempts.
  • And said to myself, "This is still the beginning...".
I got a message today morning (Nov 5th) from one of the developer "I am still wondering; how you were able to make the Tomcat disappear?" and the company has decided to re design the whole software and are going to invite me for another round of testing. ( Of course they would invite me, I did not charge them anything )

I am sure, I am not yet as good as most of you here are but I am happy that I am in the same road you traveled sometime back and I see you far far away. Miles to go before someone sees me as far as I am seeing you all today.

Many thanks to people like Dr Cem Kaner, Michael Bolton, Mike Kelly, James Bach, Johanna Rothman, Jerry Weinberg, Jon Bach, Simon Fields, and all of you, who have inspired me in one or many ways, to put in lot of hard work and develop myself as a better tester each day. I wish to work more hard and hence keep inspiring me.

16 comments:

Anonymous said...

Pradeep,

This is excellent work. This post and others is an inspiring material for many testers like me and thanks that an Indian is able to do all this.

I wish you many more success, no doubt you will face it.

Pradeep Soundararajan said...

Srinath,

Thanks for your encouragement. It is you people who keep playing a vital part of my confidence that keeps me going.

I wish you many more success, no doubt you will face it.

Thanks for your wishes but if I keep meeting success, when will I learn?

Not that success does not teach anything but we need to have failures too and learn how to convert failures to success stories and that is greater than success, right?

Anonymous said...

Thanks Pradeep. This kind of real life experiences really are useful to learn from. Excellent post!

Anonymous said...

The maturity shows. Congrats on the success journey.

Anonymous said...

Hi Pradeeep ,
Yes ..this is what we all want " attention " ..all these attention comes once you have the authority and power on the subject .The consulting work came to you due to your reputation and you deserve it.And yes I got time to watch the James Bach's Video "Becoming a Software Testing Expert" which made me use the word "Reputation".Thanks for the your informative posts in the group as well as your blog.
-Anil

Anonymous said...

Hi Pradeep,

I somehow came upon your blog just last week and let me tell you the posts are simply fabulous!
I have been selectively reading your posts from January and I wish I had seen this blog earlier.
Thanks for your efforts in putting down your thoughts on such interesting and useful topics.

Regards,
Prashanti

Anonymous said...

I showed them at least one database corruption, tomcat server crash, multiple application crashes, multiple cosmetic bugs, a requirement missing, design defects, multiple wrong assumptions.

Just awesome... great catch..

Siddhi said...

Wow. Awesome post! Great job.

Anonymous said...

This is the best thing I have ever read in my life as a tester so far.

Arent you a awesome tester?

Ur guru James Bach must be very happy for coaching you. I want to see more posts like this from u.

Anonymous said...

Hi Pradeep
I liked this blog thread. It contains good amount of technical details to inspire testers.

Niteen Yemul

Anonymous said...

Great work Pradeep. I was particularly interested in the 'motivating' part. On the testing part, no surprises, u were always good at it.

Anonymous said...

Pradeep...

Good Job, Sorry, I wont appreciate you because then at some time it may stop you somewhere, so you are utilizing your opportunity.
Keep continuing....

This is A kumar One of your student/Fan. Also a beginner in Testing.

Anonymous said...

Pradeep has the indian test community identifed u as 1 of the best tester we have?

ur hard work is visible thru ur blog and a gr8 source of inspiration to a lot of testers in my company

we read your blogs and we discuss on it. i thank u for inspiring us.

hemant

Sulaiman said...

I saw your blog only today and read only couple of your posts.

These real time experieices and other posts help beginers in gaining more confidence in software testing.

This particular post is very inspiring.

Good work.

Sulaiman

Pradeep Soundararajan said...

@Sulaiman,

Thank you! Also I am grateful to you for recommending the screen saver to your organization.

Unknown said...

Great Job Pradeep, Great stuff ....