Evolving styles & small "e" exploratory testing

When I first heard about the small "a" agile and the big "A" agile, I was intrigued. I was wondering if it really did exist. Later when I interacted with those who were agile and those who claimed to be agile but were not, I started believing in it.

Similarly, I am going to be talking about the small "e" exploratory testing style.

Every tester I have interacted, tested in pairs, watched them test... have their own style to testing while being exploratory. My own style has changed over the years and its evolving. To detail it out, there are several dimensions.

First dimension

• I Did exploratory testing without knowing it was called that way. To...
• Did learn the term exploratory testing but didn't know if that's what I was doing. To...
• Did exploratory testing based on my own ideas of how it should be done. To...
• Did learn that others have their own style to exploratory testing. To...
• Did practice other testers style of exploratory testing combined with my own style. Plus...
• Learnt that to become a decent exploratory tester, I need to develop various kinds of skills. Plus...
• Started to ask for mission, charter, objectives before exploratory testing. Plus...
• Learned about SBTM and practiced exploratory testing with SBTM (and failed at my first trials). Plus...
• Learned to modify SBTM to my style and my context needs (current state). Plus...
• < To be updated >
• ...

Here is the second one.

• Was clicking on every visible GUI stuff to see if a bug dances out. To...
• Thought I was doing good exploratory testing if I was finding a lot of crashes. To...
• Thought good exploratory testing should always result in finding a lot of bugs. To...
• Learnt that bug investigation is purely an exploratory process. To...
• Learnt that test designing skill is important and learnt & practiced it. Plus... (note the plus?)
• Learnt being conscious of the test coverage is important. Plus...
• Learnt that making notes of all my observations and inferences is important. Plus...
• Learnt that briefing and debriefing is not just like any other discussion about testing. Plus...
• Practicing to debrief. Plus
• < To be updated > 

After second, usually its the third dimension

• Thought of exploratory testing as a technique. To...
• Learnt exploratory testing is an approach. Plus...
• Practiced applying several techniques in exploratory approach. Plus..
• < To be updated >

The fourth

• Unable to explain what I did after a few hours of testing. To...
• Being able to verbally explain what I did. To..
• Being able to explain and document what I did and why I did that. To...
• Being able to do that much better over practice. Plus..
• < To be updated >

Then comes the fifth

• Being able to do all that and trying to keep it secret because I wanted to be a hero. To...
• Realizing that I can better only if I share with others. Plus
• Started to coach testers. Plus
• Started to learn from them. Plus
• I recognized that if I had kept it as a secret, I would have been zero and not hero. Plus Plus,
• < To be updated >

Semi finally, the sixth

• Aping what others are saying. Plus
• Trying out my own ideas. To
• To forming my own style of doing it. Plus
• Trying out different attack (not the attack you know) as a starting point to my testing. Plus
• < To be updated >

Se7enth

• Thinking of all exploratory testing sessions as freestyle. To...
• Learnt that there are loosely scripted & checklists types. Plus...
• Practicing loosely scripted, freestyle, checklists types. Plus
• Learnt that a good exploratory tester cannot be determined in the first few hours of testing but what comes after that. Plus
• < to be updated >

Value

If you were wondering why I moved from one style to the other, I want you to be informed that it was for being more value to my customers and not because someone said so. If you are doing anything because someone said so, without knowing if it suits your context or what value it adds, you are most likely doing the big "E" exploratory testing. Also, if you don't know what many serious testers who are practicing to be good in exploratory testing are talking / practicing, you still are doing the big "E" exploratory Testing. If your style of exploratory testing hasn't evolved over the years, you are most likely to be doing the big "E" exploratory testing. Nothing wrong in being the big "E" Elephant. Everybody started there. Everybody were stuck there. Not everybody remained there for a long time.

Also, you may ask if the points listed above indicate anything about the style. In case you did ask, I want to thank you for that because I plan to write about it here. I mean, here --> As we learn and unlearn, the way we test, changes. This influences the style. 

For example, imagine me thinking of all good exploratory testing to be yielding tons of bugs, I'd grow to be a manager who'd question someone why they didn't find a lot of bugs over the last one hour of exploratory testing and hence making myself stupid.

If you were to take another example: Learnt being conscious of the test coverage is important : Everybody in their first few years of testing, at some point, lose the sense of test coverage they are achieving. They get carried away by bugs and as a matter of fact, the client is happy too because they are seeing important bugs being found. Currently my central theme of exploratory testing is test coverage. I ask myself, "OK, I found these bugs and they are great but what is happening to my test coverage". The client may not know to ask for it but it is my duty to help them with that information.

This one, Yo! Being able to explain and document what I did and why I did that. -  I personally know many testers who seem to do good hands on exploratory testing but being unable to narrate a story of what they did, makes them look bad. The senior management appears to just look at the bugs they report and hence the good exploratory tester looks like a "not so good" exploratory tester. Today, I can tell an engaging story of why I did what I did and what value it adds to them and also why I should be given more freedom and hence more responsibility.

Try this! Practicing loosely scripted, freestyle, checklists types. Plus


Based on how accountable I need to be, I choose which type of exploratory testing is needed. That only comes with practice.

Recently, I was asked to explain my test coverage & strategy. Why did that arise in between the consulting assignment? When I started my first round of testing, I was finding bugs as I learnt the application, then I stopped finding bugs because I was doing some confirmatory tests and during that time, the client was wondering, "Is that all about Pradeep?". A very good question to have asked. So, I presented them with my strategy and coverage plan. Here is snippet from that. (Taking out all confidential information). Level 1.


- Toolbar options
- Mouse hover / clicks / right click / left click
- Keyboard shortcuts
- URL / Path editing
- Menu options coverage
- Claims based testing
- Questioning the purpose of existence, position, alignment and meaningfulness of all UI
- Consistency within the application
- Consistency with Windows standards
- Confirmatory testing
- Process monitoring & control
- Security vulnerability
- Consistency of text, font, sizes, options
- Links / Broken links / paths
- Reliability of user login / logout
- Violating policies - Firewall, System, Third party software
- Hidden details & options
- Scenario based tests -
- Boundary tests
- Hardware based scenarios - Interrupt
- Timing based coverage
- Error message coverage
- Tools / Utilities
- Independent components
- Extended usage
- Multiple connect / disconnect
- Reboot / Sleep  / Hibernate
- Close abrupt / Kill
- User coverage
- Usability standards comparison
- Functional checking
- Supportability testing
- Accessibility
- Navigating between zones / desktops / sessions
- Platforms
- Devices coverage - Bluetooth, Printer, Memory, Ext HDD
- Authentication / Authorization
- Splash screens
- Obvious / non Obvious things for a user
- Brand image versus product behavior
- Comparable products / benchmarking
-  Testability tour
-  Behavior patterns, user expectations and consistency
_snip_


At Level 2, I had a checklist of what tests I did.

Here is an example: http://support.microsoft.com/kb/301583 : I ran through each one of them. I could tell what shortcuts cause what problems and an investigation about the problems. They were highly convinced why they were paying me as much as they did. I just stood up to the scrutiny.

At last, Trying out different attack (not the attack you know) as a starting point to my testing. Plus


Most testers use Functionality / Security as their first few possible ways to start attacking (start testing) a product. I was practicing to try out different things trying to determine how useful it is for short and long term projects. Testability attack has yielded great results for me than any other form of attack. Now, I might have learnt a lesson that the world already knows but what matters is, I did learn it. Hurrah!


If you were to determine how skilled an exploratory tester is, its not the first few hours, its what comes after the first few hours. That's what determines the the big "E", exploratory testing style and the small "e" exploratory testing style.

Also, our testing industry is so damn confused that they might end up asking you about the big E and small e exploratory testing in interviews by those who might happen to read this blog post half way and run to a meeting. Tell them that Pradeep used the small e and big E to explain styles and not to define or redefine anything about exploratory testing. Also, tell them that they didn't read this post completely.