Pradeep fails BBST & continues to seek the Holy Grail

Yesterday I spoke to Jon Joseph from Pondicherry, Tamil Nadu, India. He has at least 4 times more experience than I. He had owned a software development company prior to 2002 and now works as a Senior Test Engineer in a company that develops e-publishing software. He will be coming down to Bangalore to attend my upcoming session on 29th September.

It was a great experience to listen to his appreciation of my blog and the impact this blog has made to his life. He having developed software as a start up and has had experience as a Lead for Java based products switched to testing after reading my blog (not because I convey that message).

That's not my achievement, it's his own but maybe what I helped him and he said; "my work and thoughts have got a lot better and your blog is something that I worship".

Well, if that seems too hard for you to believe here is what Michael Bolton said last week: If a test passes in a forest, and no one sees it... and James Bach quoted that line that Michael liked from my blog in Nationwide Insurance Conference in the United States, the same day.

That makes Pradeep a great tester!

Now, here are things that helped Pradeep (that's me) realize he isn't a great tester and Pradeep also wanted to share it with his readers because -- after listening to people who have been impacted by his passion, coaching and blogging, he felt responsible to let all of them know that I fail and I struggle just as they do.

I took up the BBST Foundations online course offered by Dr Cem Kaner, Scott Barber and Jon Hagar from Association for Software Testing in the United States of America. I miserably failed the course, in my opinion, and thankfully the reviewers had the same opinion.

There were bunch of things that contributed to my failure: lack of time, improper time at which I worked the course, over confidence, lack of humility, under estimating the complexity, different method of teaching that was new to me, blindness towards the quality of my answers, lack of math skills, frustration of not performing to my expectation mid way through the course, a sense of embarrassment after looking at others answers, ...

but that doesn't mean I did not learn anything from the course. I think I had a very different and fantastic learning. I also learnt a lot more about testing that I did not know. I was glad that I was honest through the course and I admitted what I did not know and also admitted that I forgot a few things during the exams and exercises of the course, which fetched appreciation from another student who reviewed my answers.

There was another Indian on the course who did perform very well and I am very, very happy about that. My close friend Ben Simo too performed fantastic and would be the person who teaches the course the next time I take it and he deserves it.

I was worried if James Bach and Michael Bolton might be disappointed with my result but I was delighted to know they weren't. They helped me understand a lot more and James said, "I see this as a learning process and would be great for your teaching, to help your students know that you too struggled and faced the pain, which can make them happy that they aren't the alone in facing such problems" and then said, "I would be disappointed if you do not take this up again and pass through it".

I also had another worry: fear of losing credibility with Dr Cem Kaner and Scott Barber, which I have not dared to ask and it is better not to know certain things. However, I have great respect for them and their time and I thankful to everyone who spends time for my learning.

I would be curious to know what you learned from this post?

Here are some things that I decided to do:

I am going to put this failure of mine in the presentation that I am doing this weekend at Bangalore. It is important that people know that I talk about my failure when I introduce myself and helps me in remembering that I am not a great tester yet.

I might not be willing to approve any more comments on my blog that just appreciates me or the post that I write. I am young and I am not wise yet to take such an appreciation not deter my growth. Your true appreciation would be when you let me know that some ideas or all ideas helped you do better testing just as Jon Joseph. He has been my blog reader for more than an year and has never commented but got in touch with me to say my blog has impacted the way he does and think about testing over e-mail or phone.

I am sure this might make my competitors (if any) feel stronger and I don't mind that. All I want to do is to become a good tester.

I wanted to post this on my blog that I can visit any day in future to help me realize my true self.

I also am going to be a little away from testing for a couple of months because one thing that I need to do is to take a break. I haven't taken a break in the last 2 years, which was another contributor of my failure. Too much of anything is bad and to keep myself alive in this industry, I have to be a little away from it for sometime. That doesn't mean I wont blog but it means I will not do a lot of things that you might or might not know for sometime. That also doesn't mean you should not get in touch with me for help. I love to help, especially after knowing that I am not a great tester yet.

Why do you think you are seeing words, "great tester", too often in this post?

It is because I thought I was an upcoming one such and also because I discovered that it is too far away than what I imagined and here is a post to

Don't forget an important message that this post is saying: Pradeep fails + be careful if you are following his ideas and suggestions.

It is also important to note that I don't intend to be a person who can never fail but I am worried about the reasons of the failure. There were things under my control which I should have taken care!

Here is my view of perfection: It is an illusion or lack of experiments that can help in discovering failure.

I think I forgot to tell you that becoming a great tester is like seeking the Holy Grail and I wish I am not as the knights of round table of Camelot to get lost in the quest. Jon Joseph said he shall pray for me and I am sure there would be a handful to do that along with him.

No training course, to my knowledge in testing offered anywhere in India can teach wonderful things that James and Michael's Rapid Software Testing and James Bach and Cem Kaner's Black Box Software Testing (BBST) can teach but I am optimistic about my workshops as they are reflections of these courses.

Time for me to appreciate things as how they are without questioning!

-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton

Notes from spying test experts

This is a top secret post and be very careful with the information. I am not sure if I can expose my notes to my dear blog readers but I am taking a risk of doing it.

There are some secrets of test experts that not many know. I had a mission sometime back to spy about how test experts actually find many important problems, quickly.

Many people think ( that included me, too) that Test Experts do lot more tests and have more fantastic test ideas and that's why they find many important problems, quickly.

I was hoping that there is more information hiding than what is visible about test experts and spying helped to uncover those hidden secrets.

This post is all about the secrets that I found spying test experts and the secrets of how they find many bugs?

You can trust me about the information to an extent that I and many other testers have been practicing the secrets that has resulted to our success of finding many important problems, quickly.


They just do one test to find many bugs!

Now, I am sure you don't want to believe me because my finding says that they do just one test to find many bugs. I am sure you have seen testers finding one bug per tests that find a bug and that makes you not believe this information but... you must understand that the secrets are always unbelievable.

You can feel safe after knowing a fact that I too didn't trust the secret of one test finding many bugs but my spying mission was to find out more granular details about it.

Looking more carefully into it, you might discover that it is not a test that finds a bug but it is a human that finds a bug and a test plays a role in helping the human find it. So there is something with humans that is helping to find bugs and not completely with the tests.

When I collected this much of information over months of spying, I was invited to be a part of the round table conference of experts as I had posed myself as a test expert from India (I knew I wasn't a test expert and even today I am too conscious that I am not one such) but you know... I am a spy and had to pose that way.

Over the round table conference, test experts started discussion about ORACLES* and I didn't know their terminology and just took notes. That evening, when I returned to my hotel, I was trying to connect ORACLES and the spying mission I had in hand.

* Oracles - principle or mechanism by which we (humans) identify problems

I was perhaps lucky that James Bach and Michael Bolton were staying in the same hotel that I was put up and we caught up at the dinner table.

I had an idea that I hope it worked. I was hoping that James Bach is a kind of person who would spill off the secrets in a context to prove me wrong and help me learn the secrets if I talked nonsense about ORACLES, because his passion to the craft of software testing is something that I am trying to match. Michael Bolton, the Birbal of North American software testing was a person whom I had to handle very carefully as he is more likely to identify me as a spy. At the end, I assumed that I was able to outsmart Michael in not letting him know that I am spy.

Going back to the hotel room, I started writing my discoveries about the secrets of test experts and the granular details of how they find important problems,quickly.

Before I publish a report to end the spying mission, I was sure that I had to practice the details I collected. I did practice and was excited that those ideas really worked and I found more bugs by executing one test.

Why many testers might not be able to practice the secrets that I am unlocking here is because they have a fixed expected result that is derived from a specification document that the tester thinks is The Holy Bible for the project.

Specification Document - is one single oracle and hence you might find one bug per test you execute.

The important thing is experts define a bug as "anything that threatens the value of the product" (-- James Bach)

Here are a list of Oracles that experts use with every test they do:

Consistency with the Image: When I execute a test, I try comparing and contrasting it with the image that my company or stake holders have been projecting about the product or the company.

For instance, if my company has an image in the market as people who produce software whose performance is good, and I execute a test whose documented expected result is "This functionality should work", I ask question, "Well it works but didn't it take too long to perform the operation?" and then I find a bug and say, "Although this operation goes through the time it took to complete the operation exceeds beyond the image the market has about us"

Consistency with Similar Product(s): When I execute a test the functionality might work as expected but I ask a question, "Well it works but doesn't it seem to perform in a way that users who are used to similar products might expect it to happen?" and then I find an important problem, quickly.

Consistency within the Product: When I execute a test that appears to have passed, I ask a question, "Well it works but doesn't it work in a different way than other areas of the same product?" and then I find an important problem, quickly.

Consistency with Claims: Marketing personnel and my manager are making certain claims about the product. When I execute a test and the test appears to have passed, I ask a question, "Well it works but does the way it works go against the claim that they make?" and then I find a bug.

Consistency with Statutes: When I execute a test that appears to have passed, I ask a question, "Well it works but does the way it works go against a law that an organization or a certifying authority might have?" and then I find a bug.

Consistency with User's Expectation: When I execute a test that appears to have passed, I ask a question, "Well it works but does it work in a way that the users whom I am aware of might not want it to happen?" and then I find a bug.

Consistency with History: When I execute a test that appears to have passed, I ask a question, "Well it works but does it work in the way the previous releases of this product work?" and then I find a bug.

While writing this post, I got an e-mail from James and Michael that reads...

Dear Pradeep,

We knew that you were spying us for a long time. We also saw the way in which you took notes, asked questions and drew inferences from what you could capture and that impressed us to help you learn more about testing because those are some important skills that a tester needs to have, to add good value for the cost customers pay for. We consider to hire you and we recommend you to quit spying profession and take up testing where you might get more laurels for your skills. Here is a link to know more about more secrets : www.satisfice.com/rst.pdf

Good luck!

-- James Bach & Michael Bolton

I was excited and I did quit spying, to take up testing as a full time activity. I thought I shouldn't post this in my blog because I didn't want those secrets to be given away for free and saved a draft of this post. I am afraid if Blogger has a bug that publishes drafts that are not intended to be published.


-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton


Update : Michael Bolton, blessed me with a post on his blog for this article of mine. I was jumping in joy for at least half an hour reading that and I am pleased to share it with you: http://www.developsense.com/2007/09/if-test-passes-in-forest-and-no-one.html

Mother Nature - A teacher of all good testers

When I started to test, I wasn't aware that I entered a fantastic field that demands me to look at other professions to do a good job as a tester. A little later I realized that people in a specific profession have lots to learn from other professions to perform better at what they are doing.

Everyone, in my opinion does that but the question is: How many do it consciously?

Don't worry, you aren't left behind and here is your opportunity to unlock what you could learn from other professions. Here is an example of how Michael Bolton and Ben Simo talked about "How doctors think and the learning of a tester from that thought process" . During Michael's previous visit to India, we did talk about how his experience of cooking and theater that helped him in his testing and I was very glad to hear those stories.

I intend to pull out my notes of how people in different professions think that can help testers in their testing activity:


Doctors


I had mentioned this earlier and I would like to re-iterate. Doctors ask a lot of questions as a part of treating you. When a patient says "I am having a back pain", I have observed doctors asking questions about the hand and legs and they carefully observe emotions of the patient when they try to press the back, different places in leg and hands.

Software and human body are very complex systems. The "back pain" might be a symptom of a bigger problem and hence doctors ask questions about other parts of the body. So, as a tester if I encounter a bug, I get to think that this bug might be a symptom of another big bug that is hiding and ask questions that help me figure out the big bug.

When a patient says, "stomach ache", I have observed doctors prescribe blood tests and various other tests to take an informed decision. The management needs as good information as possible to take better informed decisions and testers need to supply the information. By prescribing blood and other tests, the doctors are looking for coverage and so, we testers need to look for coverage than to find xxxxxx number of functionality bugs (unless that is the mission).

When a patient is in a critical state and needs to be operated, there are diversified set of doctors who are in the operation theater. For instance, an Anesthesia Specialist, General Surgeon, Neuro Surgeon, Heart Specialist... and NOT all General Surgeons or all Neuro Surgeons do the job. That is a fantastic example of diversity and value addition to the operation's success. A testing team needs to be diversified. Not all testers who know to run tools like QTP, WR, LR ( toolsmith's I mean ) can add value to the project and successfully achieve the mission. Look at my FAQ's for more information on diversity of testing teams and its benefits.


The Indian cobbler ( I don't know of any other country cobblers)


An expensive shoe that we purchase might have been manufactured by a top company with state of the art machines but when it is torn or needs a fix for the sole, we do not go the manufacturing unit but to a cobbler nearby.

The Indian cobbler is a pretty simple guy. He uses the tools that are not state of the art but yet does a fantastic job whenever I or my friends have gone to him to get a problem fixed. He doesn't intend to use a tool because other cobblers think it is state of the art because it doesn't suit his context. Many companies buy tools from vendors who market it as state of the art, and later discover that it isn't suiting their context well but force the testers to use the tool to see some value of the money spent on it and lose the value that could have actually been delivered (if the tools were not put to use).

There are some cobblers who move on road and they dont carry tools that are hard to carry or need electricity to operate and yet complete the mission assigned to them. They do a manual activity and harness the potential in the tools that they carry to maximum extent. No testing is completely manual and no testing is completely automated. Those who think they are testing something completely manual are as much wrong as those who think their testing is completely automated. Those testers who know to carry the tools that suit their context are smarter and will add great value as compared to those testers who carry tools with them because someone said "That's the future".


Police ( What movies has shown me)


Catching criminals is as interesting and challenging as catching bugs. Police look for clues during investigation of a crime, to nab or zero down on criminals. They do not just look at obvious places but non obvious places, too. For instance an investigator looks at a dustbin while investigating a murder, finds a cigar in the dustbin and draws inferences about the criminal. Testers usually do not look at clues that surround a crash or hang and miss the actual criminal that many a times is caught by the end user. Log files is one such clue that has helped me nab several other criminals.

Police personnel ask the same question in multiple ways to the same person at different situations and look for consistency with the answer. Anything inconsistent helps them to get fishy and are one step closer to catching the criminal. If a test passes that doesn't mean it really is a "pass". The same test might fail when executed in a different time frame, different input, different tester executing it, different PC, different network, different hardware... If you observe carefully policemen are using consistency oracles to find the culprit.

I am thrilled that I am looking and learning from many other professions, too and I am glad that I am a software tester who is gifted to see the beauty of testing. I must thank God for his blessings. Not all testers are gifted but they can become one such when they start learning from all possible situations, people, things, objects, happening...

-- -- -- -- -- -- --

When you set your mission to become a wonderful tester, the nature will take care of your learning and all you need to do it to keep all your senses wide open. Mother Nature is the best teacher and you wont realize that by reading this sentence unless you experience it.

James Bach, today's leading test expert and testing legend, whose work has affected the entire testing community, is a strong example of someone who forced Mother Nature to teach him by becoming a self drop out of school during 8th grade ( or Standard ). His relationship and thought process is a gift that Mother Nature bestowed on me when I cried to Mother Nature seeking help for learning to test better.

So, set your mission and cry aloud, SHE will help you. SHE would test your passion under turbulent situations but once you pass HER test, you will get more tests and will start learning and enjoying the experience.


Reminder: Registration for the 500 rupees half day testing mania, is still open, so book your seats to experience such wonderful stuff. For details look at the left hand links section of this blog or simply link click maadi.

Learning & On demand lectures & Challenging the challengers

India celebrates September 5th as Teacher's day marking the birth anniversary of the great Dr Radhakrishnan and so today I started getting calls from my students wishing me on the occasion.

I replied, "Happy Teacher's day, to you too"
and then a student said, "No, I am not a teacher".
"Well, haven't you taught yourself?"
... after a pause ,"Oh yes!",
"Well then you are a teacher and maybe you didn't realize it"

The list of people who have taught me things is huge. Not many readers of this blog know that they too teach me when they ask questions as comments and come back and clarify their points or correct me when I have been wrong.

James Bach and Michael Bolton are two big names many think as my only two teachers. I think I am a gifted student to have learned testing from these two people and there are many other people, too. While others help me and teach me to become a better tester, James and Michael goes steps beyond and help me teach other testers.

Why is my teacher list huge?

Its because James and Michael taught me an important thing: How to learn from others although they might not be consciously teaching me?

Without that skill, I just would be less confident about my testing because testing is an activity of learning.
So, I help testers how to learn and that makes them smarter than other testers who don't know how to learn.

On demand from a few students who said I should hold a lecture that is inexpensive as compared to my workshop, here is the announcement on this special day.

The coming September 29th (Saturday), I might be holding a 4 hour session, which of course consists of one of the most exciting testing exercise for a mere 500 rupees in Bangalore. I usually spill a lot of secrets of good testing during my talk and hence you might want to get your seat booked.

In case you wish to attend this session of mine, you could send me an e-mail at pradeep.srajan@gmail.com with a subject line "September 29th - Testing Lecture Registration" and I shall send you the details.

You could pass this on to your colleagues and friends, too, to ensure they join you for this session. It would not put me in loss if there are 50 people attending this session and so if you are planning to register, passing it to your colleagues and friends might be a good idea to increase the chances of not putting me in a loss :)

_ Challenging the challengers _

"Take a look at this, "Today it is no longer important to be a good tester, or have the right technical skills to manage the plethora of problems that you encounter in your day-to-day job as the testing professionals. Increasingly, today software testers require people-oriented skills to survive what can often be a lose-lose relationship with developers,managers and clients." Take a look at this link .

What the hell?

Someone is saying it is no longer important to be a good tester! So here is the e-mail that I sent them for which there is no response, so far:



Hi Accenture Testing Challenge creator,

Please have a closer look at what you guys have written : "Managing people in testing projects - Building, supporting and adding value to your team - Today it is no longer important to be a good tester, or have the right technical skills to manage the plethora of problems that you encounter in your day-to-day job as the testing professionals. Increasingly, today software testers require people-oriented skills to survive what can often be a lose-lose relationship with developers,managers and clients."

Are you sure that is what you wanted to mean?

I strongly feel that you guys in the context of creating interest among people should not misguide them with such words. What would a fresh starter in this field think if he reads such a thing that it is no longer important to be a good tester?

• However, is that condition true in all contexts? Who said that? Can you quote the person who said that?
• Communication, is a part of testing skills, I am wondering how someone wasn't aware of that?
  
• Is that the philosophy of testing prevailing in Accenture?

I am expecting things to change and a reply to this e-mail.

-- Pradeep Soundararajan



Neither did they re-phrase nor did they reply to my e-mail.

The thing that irritates me is the comments section in the link where people don't question it and just keep adding comments. I thought of entering their testing challenge contest but now feel I am not sure of their judgment.

"Oh God! Please help the testing community from people who misguide testers".

There are testers who misguide other testers in all companies and hence it doesn't mean I am accusing Accenture for this but the people in Accenture who are misguiding the community with such a statement.

Claimer: I claim that all opinions that I share in this blog is mine and I am independent enough to think and it is none of my employer or client views.

-- Pradeep Soundararajan - pradeep.srajan@gmail.com - +91-98451-76817

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton