Evolving styles & small "e" exploratory testing

When I first heard about the small "a" agile and the big "A" agile, I was intrigued. I was wondering if it really did exist. Later when I interacted with those who were agile and those who claimed to be agile but were not, I started believing in it.

Similarly, I am going to be talking about the small "e" exploratory testing style.

Every tester I have interacted, tested in pairs, watched them test... have their own style to testing while being exploratory. My own style has changed over the years and its evolving. To detail it out, there are several dimensions.

First dimension

• I Did exploratory testing without knowing it was called that way. To...
• Did learn the term exploratory testing but didn't know if that's what I was doing. To...
• Did exploratory testing based on my own ideas of how it should be done. To...
• Did learn that others have their own style to exploratory testing. To...
• Did practice other testers style of exploratory testing combined with my own style. Plus...
• Learnt that to become a decent exploratory tester, I need to develop various kinds of skills. Plus...
• Started to ask for mission, charter, objectives before exploratory testing. Plus...
• Learned about SBTM and practiced exploratory testing with SBTM (and failed at my first trials). Plus...
• Learned to modify SBTM to my style and my context needs (current state). Plus...
• < To be updated >
• ...

Here is the second one.

• Was clicking on every visible GUI stuff to see if a bug dances out. To...
• Thought I was doing good exploratory testing if I was finding a lot of crashes. To...
• Thought good exploratory testing should always result in finding a lot of bugs. To...
• Learnt that bug investigation is purely an exploratory process. To...
• Learnt that test designing skill is important and learnt & practiced it. Plus... (note the plus?)
• Learnt being conscious of the test coverage is important. Plus...
• Learnt that making notes of all my observations and inferences is important. Plus...
• Learnt that briefing and debriefing is not just like any other discussion about testing. Plus...
• Practicing to debrief. Plus
• < To be updated > 

After second, usually its the third dimension

• Thought of exploratory testing as a technique. To...
• Learnt exploratory testing is an approach. Plus...
• Practiced applying several techniques in exploratory approach. Plus..
• < To be updated >

The fourth

• Unable to explain what I did after a few hours of testing. To...
• Being able to verbally explain what I did. To..
• Being able to explain and document what I did and why I did that. To...
• Being able to do that much better over practice. Plus..
• < To be updated >

Then comes the fifth

• Being able to do all that and trying to keep it secret because I wanted to be a hero. To...
• Realizing that I can better only if I share with others. Plus
• Started to coach testers. Plus
• Started to learn from them. Plus
• I recognized that if I had kept it as a secret, I would have been zero and not hero. Plus Plus,
• < To be updated >

Semi finally, the sixth

• Aping what others are saying. Plus
• Trying out my own ideas. To
• To forming my own style of doing it. Plus
• Trying out different attack (not the attack you know) as a starting point to my testing. Plus
• < To be updated >

Se7enth

• Thinking of all exploratory testing sessions as freestyle. To...
• Learnt that there are loosely scripted & checklists types. Plus...
• Practicing loosely scripted, freestyle, checklists types. Plus
• Learnt that a good exploratory tester cannot be determined in the first few hours of testing but what comes after that. Plus
• < to be updated >

Value

If you were wondering why I moved from one style to the other, I want you to be informed that it was for being more value to my customers and not because someone said so. If you are doing anything because someone said so, without knowing if it suits your context or what value it adds, you are most likely doing the big "E" exploratory testing. Also, if you don't know what many serious testers who are practicing to be good in exploratory testing are talking / practicing, you still are doing the big "E" exploratory Testing. If your style of exploratory testing hasn't evolved over the years, you are most likely to be doing the big "E" exploratory testing. Nothing wrong in being the big "E" Elephant. Everybody started there. Everybody were stuck there. Not everybody remained there for a long time.

Also, you may ask if the points listed above indicate anything about the style. In case you did ask, I want to thank you for that because I plan to write about it here. I mean, here --> As we learn and unlearn, the way we test, changes. This influences the style. 

For example, imagine me thinking of all good exploratory testing to be yielding tons of bugs, I'd grow to be a manager who'd question someone why they didn't find a lot of bugs over the last one hour of exploratory testing and hence making myself stupid.

If you were to take another example: Learnt being conscious of the test coverage is important : Everybody in their first few years of testing, at some point, lose the sense of test coverage they are achieving. They get carried away by bugs and as a matter of fact, the client is happy too because they are seeing important bugs being found. Currently my central theme of exploratory testing is test coverage. I ask myself, "OK, I found these bugs and they are great but what is happening to my test coverage". The client may not know to ask for it but it is my duty to help them with that information.

This one, Yo! Being able to explain and document what I did and why I did that. -  I personally know many testers who seem to do good hands on exploratory testing but being unable to narrate a story of what they did, makes them look bad. The senior management appears to just look at the bugs they report and hence the good exploratory tester looks like a "not so good" exploratory tester. Today, I can tell an engaging story of why I did what I did and what value it adds to them and also why I should be given more freedom and hence more responsibility.

Try this! Practicing loosely scripted, freestyle, checklists types. Plus


Based on how accountable I need to be, I choose which type of exploratory testing is needed. That only comes with practice.

Recently, I was asked to explain my test coverage & strategy. Why did that arise in between the consulting assignment? When I started my first round of testing, I was finding bugs as I learnt the application, then I stopped finding bugs because I was doing some confirmatory tests and during that time, the client was wondering, "Is that all about Pradeep?". A very good question to have asked. So, I presented them with my strategy and coverage plan. Here is snippet from that. (Taking out all confidential information). Level 1.


- Toolbar options
- Mouse hover / clicks / right click / left click
- Keyboard shortcuts
- URL / Path editing
- Menu options coverage
- Claims based testing
- Questioning the purpose of existence, position, alignment and meaningfulness of all UI
- Consistency within the application
- Consistency with Windows standards
- Confirmatory testing
- Process monitoring & control
- Security vulnerability
- Consistency of text, font, sizes, options
- Links / Broken links / paths
- Reliability of user login / logout
- Violating policies - Firewall, System, Third party software
- Hidden details & options
- Scenario based tests -
- Boundary tests
- Hardware based scenarios - Interrupt
- Timing based coverage
- Error message coverage
- Tools / Utilities
- Independent components
- Extended usage
- Multiple connect / disconnect
- Reboot / Sleep  / Hibernate
- Close abrupt / Kill
- User coverage
- Usability standards comparison
- Functional checking
- Supportability testing
- Accessibility
- Navigating between zones / desktops / sessions
- Platforms
- Devices coverage - Bluetooth, Printer, Memory, Ext HDD
- Authentication / Authorization
- Splash screens
- Obvious / non Obvious things for a user
- Brand image versus product behavior
- Comparable products / benchmarking
-  Testability tour
-  Behavior patterns, user expectations and consistency
_snip_


At Level 2, I had a checklist of what tests I did.

Here is an example: http://support.microsoft.com/kb/301583 : I ran through each one of them. I could tell what shortcuts cause what problems and an investigation about the problems. They were highly convinced why they were paying me as much as they did. I just stood up to the scrutiny.

At last, Trying out different attack (not the attack you know) as a starting point to my testing. Plus


Most testers use Functionality / Security as their first few possible ways to start attacking (start testing) a product. I was practicing to try out different things trying to determine how useful it is for short and long term projects. Testability attack has yielded great results for me than any other form of attack. Now, I might have learnt a lesson that the world already knows but what matters is, I did learn it. Hurrah!


If you were to determine how skilled an exploratory tester is, its not the first few hours, its what comes after the first few hours. That's what determines the the big "E", exploratory testing style and the small "e" exploratory testing style.

Also, our testing industry is so damn confused that they might end up asking you about the big E and small e exploratory testing in interviews by those who might happen to read this blog post half way and run to a meeting. Tell them that Pradeep used the small e and big E to explain styles and not to define or redefine anything about exploratory testing. Also, tell them that they didn't read this post completely. 

Traditional Software Testing Process - The cash cow demystified

So, I don't need to tell you that most of the software testing in the world is outsourced. I live in Bangalore and I assume a considerable amount of the world's outsourced testing projects are here. I meet testers from most of these organizations and hear the same process of testing software although called by different names their organization executives named them.

Most testers I meet lament about the process (or sucked within it for survival ) and approach they are forced to follow and try talking to their management about better ideas and better ways to test. Those executives don't seem to listen much nor willing to change.

Wondering why they wouldn't?

They have hit a cash cow.

Mark Crowther and I were discussing about Code Coverage & Requirements Based Testing in Test Republic and some how drove me to write about the way most businessmen are fooling their customers through the process they love to follow.

So, here it goes with editing and expansion. Assuming Mark from UK or maybe even you outsourced a testing project to me:

• I would spend a couple of days analyzing your requirement document and bill you for X hours per person involved in my team.
• I would spend a couple of days writing a test plan document ( but not refer to it ) and bill you for 2X hours per person involved in my team for preparing it.
• I would spend a month or two writing test case document ( and refer only to it ) and bill you for 10X hours per person involved in my team for preparing it.
• I would then again create a traceability matrix ( just to fool you and your boss about our coverage ) and bill you for 5X hours per person involved in my team for preparing it.
• So far, total of 18X hours per person involved in the team is the billing.
• Assuming X is 50 hours and there are about 10 members in my team, that's 18 * 50 * 10 = 9000 hours of billing with no single bug found yet. ( Mark wrote about it, too )
  
• If you are paying $20 an hour per person on an average, you would have actually given me a business of $180,000 without me or my team finding any bug yet.
• So after investing $1,80,000 on me and my team, you would want some benefits of that. So, you wouldn't pull the project out or move it to another vendor because more or less he would do the same and you would end up paying another $180,000
  
• Then comes the test case execution cycles for our documented 10,000 tests out of a possible hundred million tests
  
• For every new bug that you find out of the releases I make, my team would spend documenting the new test case, getting it reviewed and resulting in slower testing for you and more money for me.
• So assuming running 10,000 tests take 2 weeks for a team of 10 members to execute. Also assuming least 50 cycles of testing, you would have paid me about $140,000 for a coverage whose value might be not worth of the money.
• Of course there is additional documentation of missed test cases and other template filling activities that will be billed to you.
• To fool you further, I would instruct my team to use some expensive license based tools ( what else will I do with the money you are pumping in ) to give you a sense of faster testing ( by foolishly comparing it with human speed of testing ) and call it "Automation Testing". It turns out that these tools could have helped me find bugs that are of not a great value to you but hey, we want to see test case pass more than fail.
• Your coverage isn't improving much because we have converted manual tests to automated tests ( although its not the same test ) but to show you the speed of our tools.
  
• So think about adding another $25,000 and giving you an illusion of an ROI of $100,000 while pumping multiples of $100,000 from your bank account to mine.
  
• The CMM, TMM, Six Sigma, ISTQB scams are built around this eco-system to enable more money flow for hardly any value. Who knows there could be a cut for the people who know all this and yet do it.
  

Why wouldn't a businessman be glad about the traditional approaches to test software?

Are you asking about what happens to the users of the product?

• Lets bother about the users of your product later during our maintenance billing phase. Don't you know SDLC ends in Maintenance Phase? If we do everything right in the previous stages then how do we prove we follow SDLC when there is hardly any work in maintenance phase?
  

While you are reading this, you shouldn't be thinking of this happening only in India but in most parts of the world and even within places like United States and Europe. There are smart businessmen everywhere. At one end they pay us but at the other end use us to make more money. We need money and they need us to make that. Don't make smart businessmen exclusive to India and leave your own country out of it.

I hope those who outsource start pushing for services that doesn't fool them. Who would actually listen to this argument is testers turned businessmen and testers turned outsourcing heads and testers turned business leaders.

Whenever I visit a testing services company and see a testimonial of a customer who talks about the great ROI they got and faster testing, I wonder what a heavy price they paid to believe so.

Exploratory Testing ++ , Context Driven Testing ++ , Rapid Software Testing ++ or else YourMoney --

Don't want to be fooled by outsourcing software testing? One of those who could be of help to you among many folks I know, is myself.

Collection of Notes & Experiences: Bangalore Workshop on Software Testing 1

Approved for Publication by all Participants

Theme: Changing the way people test and think about testing
May 2^nd, 2009
Venue Sponsor: Edista Testing Institute & Test Republic


Participants displaying their K - cards
For more pics, click here

Participants ( not in the o: Ajay Balamurugadas, Aishwarya D Shukla, Guruprasad, Manjunath, Rahul Verma, Rahul Mirakhur, Shrini Kulkarni, Manoj Nair, Ravisurya, Santhosh Tuppad, Shikhar Singh, Raghu Sahay, Sharath Byregowda and Pradeep Soundararajan

Bangalore Workshop on Software Testing officially started with the announcement on my blog on April 12, 2009. We had about 15 people rushing in their registration and abstracts for presentation. When you see the list above, you would know who finally made it.

We tried following the LAWST style peer conference and we think we did. I played the role of host and facilitator. I am sure the next time someone else will play the facilitation. I learned facilitation by observing Paul Holland do it in TWST and CAST. He is an amazing tester and facilitator. We used K-cards and that really worked. You would see in photograph, people holding K cards.

We had a check in and introduction (as there were a few faces who did not know each other) at 9:30 AM followed by Ajay kicking off the first presentation.

Ajay’s presentation: Rapid Software Testing

Ajay talked about the change he had after attending my workshop on Exploratory Testing – A Rapid Software Testing Approach and how it helped him log several hundreds of bugs when he started doing a structured exploratory testing. He explained how he struggled with the templates that he had to adhere and how those test cases filled in templates failed to yield coverage and value.

Ajay brought in an interesting thing: “When I started to log more bugs, people were skeptical of how I was doing it. They used to think I was stocking them and releasing it as required”. In this case, Ajay funded himself for the training.

It surprises me that some organizations get their resources trained to help changes to occur to their work and grow skeptical when they get to witness the change.

Before he finished his presentation, he mentioned how the management started seeing Ajay’s work output as a different value than what they expected out of him. What they expected from Ajay was to run as many test cases as possible and what he ended up producing was finding defects and reporting them in their bug tracking system. That is an interesting one – test cases are there, wherever they are, to help testers find bugs and some managers might be misusing it to show progress of testing. I have heard it happens everywhere in the world and not just in India.

During the facilitated discussion that happened after Ajay’s presentation, Shrini mentioned, “The value depends on what stakeholders defines it as”. However, sometimes some stakeholders might need education from a tester to have more meaningful value definition.

Rahul Mirakhur (RM) brought in the idea of a tester wearing different hats and being able to think the value that different stakeholders might want from a tester.

When he interacted with Support teams in an earlier organization, it opened up a completely new world of information source and that, changed the way he did testing thereafter. It also gave a glimpse on how different functions on a project bring in value and in the case of Support teams case, value is more "after" product release.

As the discussion drifted to bugs, Rahul Verma said, “I see bugs as questions we can ask”

Manoj Nair’s presentation on An attempt for better testing

Manoj talked about the opportunity he got in his project where time crunch and amount of testing to be done, enabled his manager to bestow him with a freedom to do free style exploratory testing (without much documentation) for an important feature in his project. The defects and the information he came up by performing free style exploratory testing made the management to postpone the release realizing the danger. He felt happy with his effort, until he was required to explain the details of test coverage across the feature and provide test case document for the tests he conducted for those features. The test case document he prepared were subset of the entire range of tests he performed because he did not have time to document all those tests. He realized the value of tests he did could possibly diminish through that activity. Looking back at it he felt, if he had used Session Based Test Management, he could had done a much better job in providing test ideas document.

During the discussion, I talked about the idea of a Wiki for changes to the product open to all team members and the value it could have for the test team.

Aishwarya, an undergraduate student asked a striking question, “How much information would be an overflow of information?”

The discussion then digressed to – When is the right time to get testers on the project for which all participants had different ideas and experiences. Many organizations have tried bringing in testers early and late and most of them are not yet sure when to bring them in.

I argued that it depends on the kind of a tester an organization is trying to bring in and other factors. Not all testers are the same. I have witnessed organizations that actively ignore testers in requirements review meeting as they have had a bad experience of bringing in a tester. It makes me think that if I lose credibility, it affects other team members in my test group. That is how important it is for each team member to have and build credibility.

Manjunath’s Presentation of Review of Review of Review of Bug Reports

He talked about his experience in one of his previous organization in the context of bug report reviews conducted to help testers log better and credible bug reports. This exercise was undertaken based on the success of improved bug report quality, post a review of each bug reported from the previous project.

Interestingly, in the second project there was another level of review that happened over the review comments of bug reports. The tester, whose bug reports were review had an option to appeal for escalation if he found the review comments did not fit his (her) bug report. It got many testers to defend each of the review comment of the bug reports they logged. Thus spending more time on a single bug report, instead of finding more bugs. That's opportunity cost. That’s like review of review of reviewed bug reports. Yes, we did have a good laugh!

This process was then declared a failure for the second project as its implementation went wrong and 
ceased to exist thereafter, maybe. 

The process can become so worse that it might eat so much time for so little value. On probing, Manjunath revealed that the review process was actually initiated to help testers write better bug reports. All participants were convinced that the way in which the mission was executed might have been poor and what works in one context does not fit another. I am sure we are going to remember Review of Review of Review of Bug Reports experience report.

Doesn’t it help us understand that the best practices do not work for another project within the same organization because the context keeps changing?

During the discussion, we discussed ways of credible bug reporting and shared our experiences of reporting bugs through Video recording of the bug and the value it has had for people who experimented that. It cuts across communication barriers and I am surprised why this practice is not widely used. Oh! 

Someone hasn't talked about it as "best practice" yet? It then digressed to how much English should testers know and then RM talked about practicing English as a daily activity as opposed to a one time upgrade of English skills.

He provided inferences that it’s (relatively) harder to learn language (or command over it) when we get old compared to when we are young. Shrini mentioned that it is not a "rule" by citing his personal experience of how despite being relatively old (compared to most of the audience), he is constantly on the lookout to learn new things.

We got back to reviewing and then discussed on reviewing as a skill and review effectiveness.

Lunch Time folks!

At about 1:15 PM, we walked to a nearby hotel down the street and carried over facilitation-less discussions over lunch table. A heavy lunch

Then when we returned, something helped us get the heat back… Rahul Verma’s presentation on Confessions of a Fallible Tester

This was a very energetic presentation we had for that day and thanks to Rahul Verma, he did it right at the time it was needed ( post lunch )

He made confessions of the mistakes he did right from the start of the career where he thought testing was not his cup of tea. He just did not confess but also explained how he learned from those mistakes and lived better after each big mistake.

One of his confession sounded very cool - “When senior testers in the team said XXXX XXXXXX was the best tool, I believed it” and then he explained how the project suffered because of that tool or the idea of thinking that as THE BEST Tool.

He explained how he cleared that trap and ended up writing a customized plug in to work with XXXX XXXXXX to make it suitable for his testing context.

He then confessed on another mistake of how two blank lines that caused the test environment to fail to enable the product to be tested over it. He or his team members make turned the case by defocusing the customer to something else when the customer observed it. The message was – Test your Test Environment. I think those who do that today are the ones who have realized the value of it. Many testers just setup and start running their test cases without bothering if test environment is what they intended it to be.

The discussion kicked off and we started discussing about tools. The topics varied from Automation Readiness to Vendors bribing managers to sell their tools to Good code versus Measurable Code.

From Rahul Verma to Rahul Mirakhur’s presentation on My career and experiences in testing

He started with a cool question, "When people say WHY will you want to be a software tester, I ask WHY NOT" and I think that was exciting. RM had some interesting observations about IT people in India – “By the time people realize what they are good at they would be in some other domain and would have beefed up living a luxurious life that doesn’t allow them to shift to something they want to do”

He asked another question “Anyone can be trained on software BUT what can they really do?” and discussed that every human is a born explorer and testing is an innate thing and can be developed by proper training.

He then talked about his experience of witnessing developers who moved to testing because they were interested in testing. He then said based on his experience, “Jump into development first if you are serious about becoming a tester”. He shared the experience and value of hiring a person who earlier worked in a support job.

The discussion post this presentation helped in understanding how many different ways are there to get into testing and the value of taking each way. We also talked about the value of learning testing from other fields and that reminded me of CAST2008 conference theme and presentations from there.

RM went back from BWST and started a blog and has promised to keep posting. Another credible blog from an Indian tester on the way - check out.

Sharath Byregowda’s experience of Session Based Test Management

Sharath talked about his experience of experimenting SBTM on the project he worked for and how he influenced other testers, the management and customers with it. He has detailed it in his blog post and I am sure you'd read go there and read it or maybe even learn from it. His team won an award for achieving great results in his organization that has thousands of testers. In addition, Sharath got another award from Michael Bolton.

We had an interesting comment that - By the time an organization implements some new practice as a "best practice" and penetrate every team, the idea is outdated. It reminded me of Ted Neward’s presentation at Oredev08 where he talks about the time difference of programming language birth to becoming an industry usage.

Shrini Kulkarni’s experience of test automation consulting

Shrini talked about his experience of becoming a tester and then about how he grew from being a fresher in software testing to wherever he currently is. He then talked about an interesting experience of Test Automation Readiness Assessment that he was supposed to do for a client in North America and the challenges he had. Shrini talked about his confidence in doing it although his colleagues thought of it as an impossible task. Shrini detailed the value of collaborating with people like Michael Bolton, James Bach, Cem Kaner and others that have helped him evolve to a Test Automation Expert.

 Shrini also talked about his consulting experience and mentioned that almost all test consulting assignments he witnessed, the client was interested in knowing some "universal" or industry standard benchmark related to testing - be it dev to test ratio or % of project cost that could be associated to testing. His experience was that he rarely witnessed anyone challenging those industry standards. In his opinion - That is hindrance for our profession.

Discussions on Test Automation, mindset and skill set followed Shrini’s presentation until the bell rang at 5:30 PM

We had a quick check out and got on to the socializing evening and facilitation-less discussions. We decided on a Food Court in Koramangala and spent about 3 hours there munching food and discussions on Cricket, Movies, Testing, Food, Beer and Wine.

What a fantastic learning to all of us. I must thank Shikhar Singh and Raghu Sahay for coming down from Mumbai and Pune to attend BWST-1. It was fantastic to note that Aishwarya Shukla, an undergraduate student, wanting to do a software testing project as his final year project after attending BWST along with Santhosh Tuppad who completed Practical Software Testing Training at Edista and has been working on several testing assignments as a freelancer. Every participant asked questions that helped other people learn and discover more about themselves and the work they do.

Once again, special thanks to Mohan Panguluri, Pradeep Chennavajhula and Rachna for providing us the space @ Edista Testing. I wish India has more leaders like them who facilitate knowledge sharing and learning in software testing without just being business minded.

Shikhar went back to Mumbai and wrote an e-mail to me regarding Mumbai Workshop on Software Testing that he is planning. Watch out India!

Here is the list of Post It Notes that all participants contributed as an additional reference points to the discussions we had:

• Only testers can talk about Testability
• Book suggestion: Perfect Software and other Illusions about Testing – Jerry Weinberg
• “Every bug is a duplicate of a master bug that the product fails to do something that is important to work for someone” – if that sounds bad - that's still me.
• Philosophy in Golf & Testing – Adam Goucher
• Bug Advocacy – Cem Kaner
• How to investigate Intermittent Problems – James Bach blog post
• “The best tester is the one who gets the most the right bugs fixed” – Cem Kaner in Bug Advocacy
• Do confess when you screw up
• Turning Numbers into Knowledge – A book by Jonathan Koomey
• Lessons Learned in Software Testing – Kaner, Bach, Pettichord
• “Automation could be a dangerous word for a tester” – Rahul Verma
• “If you have failed big, it indicates you rose to a great height” – Rahul Verma
• Value of Checklists – Cem Kaner’s presentation in CAST 08
• Video Record Your Bugs
• The intake I have had from BWST-1, has questioned what I have learned so far. It has helped me to unlearn and learn consistently - Ravisurya
  
• "When I spoke to support teams, it changed the way I tested, forever" - Rahul Mirakhur “
• After I attended Rapid Software Testing Workshop, my testing improved” – Ajay Balamurugadas
• “The value you add as a tester is like NAV of an investment. It changes every day” – Shrini
• General Systems Thinking – A book by Jerry Weinberg

Checkmate heuristic :: A security testing attack

It has been five times over the last six months that someone considered hiring me before making a release decision or after getting skeptical about scripted tests.

Amidst recessionary times, someone considered outsourcing some testing work to me from United States. An advanced version of the product was slated for a release in the next couple of days. The United States company had outsourced development and testing work to a Structured Fancy Name Process Following Disciples company who had ran through thousands of tests over it and had achieved >98% test case pass, a week before today.

Someone in the United States company thought they'd like some Exploratory Testing and I got the opportunity to lay my hands on it to perform a Rapid Testing on it. The charter for me was to report any security related threats and usability problems.

I found about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems:

• I could reset the password of any account by tweaking the variables that the client was using to interact with the server.

• I could stop auto e-mailers reaching any registered mail account in a similar manner as above.
  

I then tried to reset the password of the dummy account I was using. No e-mail reached me. I then thought, "How about reset of admin account?" and then did the same.

The password was reset and no e-mail reached the admin, as auto e-mailers were stopped. So, I asked the admin of the UnitedStates company to login from his credentials and the response was a pleasent, "What did you do and How did you do that?"

Subsequently, all other users were blocked. Only the admin could release the lock but the admin could not log in to the system. 2 hours of outage till someone got into the database to recover the admin account.

You write a lengthy email, hit the submit button and the application prompts for your Username and Password. You enter them and it says, "Incorrect Username or Password". You attempt to reset your own password but the email does not reach you.

Checkmate!

Change in hiring and interviewing process in India for software testing and software testers

I claim to be one of the most experienced and most affected tester in the context of interview in India and here ( in this link ) is more information about it. After going through that link or this video you would know that I have been struggling to not see another Pradeep Soundararajan in the job market who is frustrated with this industry's idea of interviewing. Edista Testing Institute ( my client and partner ) is constantly pushing towards seeing a better testing community and that's why I chose to work with them.

I consider the following as one of my biggest contribution to the change the industry needs. Most training institutes in India (even the so called highly reputed ones) have people who dont know to test, teaching testing by running a thousand slides. I think they have so many slides that if you run 25 slides a second to watch it like a movie, it still runs for about 2 hours. Such slides have always caused an avalanche slide of many victims career, knowledge and skills.

Edista started to redefine things by hiring me and then allowing me to hire Manoj and then Sharath and then now more people. The last I heard from Manoj is that testers who were interacting with him are now excited about what Manoj is doing and are enquiring what it takes to be able to get skilled in testing.

You might also discover that Manoj has started to publish his practice sessions on testing in his blog.

We sit to gether, test, learn from each other, create exercises, practice pair testing, discover new tools, debate on ideas, think about more heuristics, brainstorm test ideas, discuss about bugs, run a test club ( like the movie Fight Club ), teach people how to learn and then how to test. We are never away from testing and we are never away from anything about testing.

We aren't skeptical about the fact that there could be more people like us and who knows they might be interested to join us. So here goes the job posting for the same: ( and I have posted this in LinkedIn and other communities - please feel free to share this job posting to all other Indian testers )

Consultants in Software Testing and Software Test Education :: Bangalore

Profile:

Edista Testing Institute ( www.edistatesting.com ) has a couple of openings for Consultants in Software Testing and Test Education.

It wouldn't be wrong if I say, they are looking for people who have the urge to be heroes in software testing. This role demands you to train ( yourself and others ), collaborate with the on going research activities, test products, learn and innovate.

This role also demands you to grow to an extent to be able to contribute valuable things to the testing community and work for its betterment.

About your co-workers:

You would work with skilled testers and brains who constantly engage in learning activities, blogging, discussions, teaching, mentoring, challenge and argue online in testing forums ( like www.testrepublic.com ), offline and reinvent the art of reinventing things in testing.

If you think you wanted to be a hero ( or hero-in ) in software testing and never got the opportunity, here it is.

Eligibility:

We'd be glad if you have worked as a tester for a while ( at least 2 - 5 years ) and also be glad if you are willing to travel within India (or abroad ) on short term assignments.

We would prefer you have a degree in Engineering or Science however if you dont have them but have a demonstrated ability of good thinking, we'd be fine.

Interview Process:

Our interview process is cut above all other organizations that neighbor us. We put you in the testers seat, give you time to test a product and have a discussion of your testing based on the report you produce.

We aren't too bothered if you dont know the difference between Sanity testing and Smoke testing because we believe, knowing the difference ( even if it exists ) doesn't make a huge difference.

We aren't bothered if you have a certification in Software Testing or not as long as you are passionate, skilled in testing, and have the fire and fuel to take you a long way. In simpler words, it *doesn't matter* if you dont have ISTQB, ISEB or CSTE certifications.

About Edista:

www.edistatesting.com

Time to join:

Immediate is preferred. A little delay is fine if you are stuck somewhere.

Contact:

Send your profile to : resume@edistatesting.com

If you can crunch your profile in one page, we'd silently thank you for that.

The results so far:

• We invited about 7 candidates so far who claimed to have energy, passion and demonstrable testing skills.
  
• We are seeing great benefits of this approach.
  
• It makes us spend 3 minutes ( after a person has finished the test and generated a report ) to know about the claims a tester has made in his profile are fale and a little bit about the organization that said, "Yeah, he can test".
• We spoke only to one person in depth as his report was quite interesting.
• We know that we can filter more candidates with lesser time we have and get better ones to work with.
• It would be dangerous to get a person who cant test and report credibly into any organization that wants to hire testers.
• Those who fake testing experience fear to even apply or even if they do and by our overlooking we invite them for the testing session, we dont need to spend time beyond 3 minutes post their test.
  

Testers have to be tested on their testing skills and not on memorization skill. The best test ( based on the current situation ) you could give a tester during an interview is to make him sit on a computer and asking him to test a piece of software by giving a meaningful mission and time to do it.

Side note: Are you planning to be in the supporters list?

Audio Podcast :: What Software Testing Could Be?

If you haven't noticed that I have been doing audio [and video] podcast series Testing Stories from India, no problem. You have known it now.

Here is my latest podcast: What Software Testing Could Be? [ 3.18 MB ]


Summary:

• There could be about 28,000 definitions of software testing
• What? Are there rules of software testing?
• Many testers don't test their own definitions of software testing?
• Certified testers hardly speak about the definition they learnt to get certified.
  
• What is Pradeep Soundararajan's definition of software testing?
• What tests did I do on the definitions I subscribe to?
• What is your definition of software testing?
• Have you tested the definitions you subscribe to?

Previous Podcasts:

• The word "TEST"
• Test Plans Aren't Ice creams
• Testing Lessons from Beer
• Indian Testers and Fake Experience

Software Testing Videos:

• http://www.viddler.com/explore/testertested

If you can't download these because your organization firewall doesn't appear to have bugs that let you download these, write to me, I shall send you a copy in an e-mail.

Progress Report 2008 :: Pradeep Soundararajan

Click on the image below to download the full verison.



Alternatively, you may right click on this link and save the file to your desktop.

Jony Jony, Yes Papa! Following Process, Yes Papa! Telling Lies, Ha ha ha!

I am 100% sure that this article would make no difference to the world because there have been better insightful articles than this about the topic that I have started to write, which didn't make much difference to the part of world I think I am living in.

I feel it is important for you to ask yourself, "Why would I want to read an article that the author is certain that it wouldn't make things around me better?' because by asking that question and continuing reading ensures that you are wasting your time because you chose to.

I conducted my public workshop on December 1st through Edista Testing - a QAI venture in Bangalore on December 1st. A couple of months ago, I had expressed my dream to go around places in India other than Bangalore and conduct my workshop on human skills of testing titled Exercises for a Testers mind - A Rapid Software Testing Approach and the dream is coming true as Edista plans to bring this workshop to your city wherever you are in India. Do not worry about the cost as long as your company can sponsor you for a day.

I challenge testing minds and I learn from them and some of the curious minds who attend the workshop largely benefit from the workshop. All audience are thrilled about the things they learn from the workshop but what makes them sad is that they have to go back and follow the process their company mandates them to follow - which at the end of the workshop they know that it doesn't add as much value as they witnessed the value that Rapid Testing adds.

How do you think they witnessed, the value that Rapid Testing adds V/S the value that their process ( IXX, CXX - Level X, SICK6 SXXXX, XXXXX, Centre of Excellence, Test Factory) adds?

This time I felt terribly challenged at the question of process and I revealed something that shocked the audience and were not willing to pose further challenge. Its exactly an year old secret that was lying in my inbox and I had shared it with very few.

Rapid Testers stand up to scrutiny and so I stand up to scrutiny in case the following information you might read, appears to be an exaggarated or made up to prove a point. It might appear so because those who dwell with those processes can never achieve it and it is close to impossible for one who believes in those to believe the following.

Here is an e-mail that I received last December from my Supervisor from a large Products + Services company of India on the last day of my job at that company.

---- Forwarded by Pradeep Soundararajan on 12/27/2006 04:29 PM -----
All the best, Pradeep.

There are learnings for all of us from the way Pradeep has conducted himself in this tenure in the team. The confidence he has shown and willingness to help others and constantly exploring for new ideas are some of the highlights. Fortunately I was also part of the team which Pradeep was associated.

One more thing I would wish to share with you all is that, he was handling XXXX (product name masked) releases for XXXX (A multi billion dollar customer name masked - who also had a test team at his end who tested the releases we made before they released to their customers ) and with proud I can say that there isn't any bug which customer found apart from what we or Pradeep found here.

On behalf of the team and on a personal note I wish him all the best in all his future endeavour .

Regards
XXXXXXXXXXXX ( Supervisor's name masked )

__ end of e-mail excerpt __

Some very important points to think and remember:

• To remind you, this company too, is one such who believed in the above mentioned process till I helped at least some teams realize that they could achieve bigger success if they could come out of the trap they fell into and add more value to customers. No customer would say - I don't want you to add more value but I am paying you to follow XXXX process. If a customer says that, either the customer needs to be educated on testing or it is a customer who deserve to be off the business list unless the customer is willing to pay a huge price.
• Some of you might think this post as my self marketing and might fail to learn some important lessons that your customer might want you to learn. Also if I wanted to use the above as my marketing, I wouldn't have waited to reveal it an year later.
  
• Some of you might think this happened somewhere too far away from India, which is not true, because this happened in Bangalore, India.
• Some of you might think I did *complete* testing but I admit that I know no one can completely test anything, so I did not do complete testing.
• It was no one man Pradeep show, it was a team effort and it was achieved because I practiced the skills that James Bach and Michael Bolton helped me gain on the product I tested with the skills that I already claimed to have + the skills of the team + Exploratory Testing mixed with very little scripted testing.
• This achievement for the company, the team and me didn't happen because we intended that to happen at the start of the project BUT we had realistic goal as a test team "To find important problems, quickly, and gather as much information as possible through our SKILLS and present them in a useful manner to help the management take informed better decision" AND NO STUPID GOAL SUCH AS "LETS MAKE A BUG FREE PRODUCT".
• By the customer not finding any bugs other than what we found, although there was a test team at his end, doesn't say that the product was bug free. What customers looked at is value for the money he paid us, and I think he was sure, as the project progressed that he was getting a bonus. At least it helped him understand that he had to hire a better test team than us to find bugs out of what we had found.
• It is hard for me to know what is the state of the product now but I can safely say that the 8 month duration I stayed there, there was nothing that our customer or ever his customer reported on the releases we made that we didn't know.
• I'd like to add another important point that in order to achieve what you read above, I and the team had to break certain things that the process mandated to achieve it.
  

Don't ask your management or customers to read this post because both of them might start to demand more from you, which might take away your coziness you have been enjoying in testing with the process you are asked to follow. In case you are curious to share with them, be prepared to learn and practice human skills of testing and better insight into testing, out of which some of them are taught by James Bach and Michael Bolton's Rapid Software Testing and Cem Kaner and James Bach's BBST courses.

"I would say that a process is the way things happen. The Earth in orbit around the Sun is a process. In that sense, yes Rapid Testing is a process.

What Rapid Testing isn't is a set of instructions to be followed without understanding. It is not a collection of physical behaviors. It isn't even really a set of techniques, although it does feature some of those. To say
it's a skill set and mindset is to locate RST within the mind of a tester. It's a process of making sense of testing problems and reacting to them differently, as the situation demands." -- James Bach


"One of the hallmarks of Rapid Testing is that your work can suck less. The pointy-haired bosses can tell you what to do to some degree, but they can't supervise you every minute of every day, and they can't tell you how to think. In the "free" time that you have--those little micromoments of disposable time, for which you won't be punished because they can't watch you every moment--you can perform a quicktest, find a bug, write another line of the test tool you're working on, sneak a look at the specification you managed to photocopy, check in on your mission, befriend a developer, have a chat with another tester about where the bugs might be, cover a product just a little more deeply with just one more test... And after you've leaked a little of what you've discovered and have a few successes under your belt, maybe you can start to train your manager into expecting nuggets like that." -- Michael Bolton

Here is another evidence of high value addition that Rapid Software Testing produced to Michael Bolton's client

Here is one of the great and very insightful quote I have read about achieving the mission, "Who cares if you followed the instructions if it doesn't work when you're done?" - Scott Barber's Dad

All I have done in this post is to complicate that quote in many words which the wise man said it in one sentence. I apologize for that.

-- Pradeep Soundararajan - http://testertested.blogspot.com - +91-98451-76817 - pradeep.srajan@gmail.com

"Pradeep's first language is not English--his first language appears to be testing." -- Michael Bolton