"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.
Showing posts with label rapid software testing. Show all posts
Showing posts with label rapid software testing. Show all posts

Tuesday, January 03, 2012

How Pradeep teaches software testing - Part 4

If you didn't know how Part 4 came up? Here is how it came. I first wrote Part 1, then Part 2 and then Part 3 ;-) So, logically here is Part 4 of the series How Pradeep teaches software testing.

"I know how to do certain types of testing but if someone asks me to explain what I did, I struggle to explain" was a problem statement I heard from a tester today. I replied in a confident tone, "I know why that happens. It doesn't just appear to be your problem but I know a lot of testers who appear to have the same problem." 

I continued, "You should consciously practice explaining what you did, first to yourself and then to your colleagues although they probably know how and why you did it. Why? I am going to explain how your brain works (or how I think it does). It has two nodes. One that contains what you know and the other that controls your explanation of what you know. When you make a conscious effort, you are forcing a connection between these nodes in your brain. When you force your brain to connect those two nodes too often, at some point it will judge the need for a permanent connection and create it for you. After that you have a free flow of what you know and your explanation of what you know." 

After I said the above, I could see a smile in the face that reflects, "Yes, I now know how to solve this problem". I read a person's understanding not by their head nodding or when I hear, "I get it", but by the emotions and expressions on their face and the body. 

James Bach identified that I was a metacog. I didn't know I was one. After that, it has been very helpful for me to understand why I do things the way I do. I guess I turned myself into a metacog because I thought it suits the kind of testing I wanted to. It's not a special status, it is just a way of life.

I don't even know if the brain works the way I explained it to be but I guess you can understand why the above explanation makes sense. I make sure I tell people that I am not trying to misinform them about anything when I use such examples. I am just helping the tester imagine why there is a problem and how to solve it. A lot of my coaching is consulting.

Examples like the one you read above govern a major part of my coaching. I observe a lot. I practiced consciously making connections between what I have seen, heard, thought, experienced and know to being explain it when the context demands.

Analogies and Examples are powerful approaches to teaching. I need to know what connects to my audience very well. Although all my audience are testers, I can't use the same analogies and examples, it just doesn't work. Bangalore testers need a different example than those in Pune. In Pune, I would talk about Raj Thackery and in Bangalore I would talk about Vattal Nagraj, in Chennai about Goundamani and not about Vattal Nagraj. Now, for those of my readers from United States or Europe, you wouldn't know Goundamani or Vattal Nagraj and hence I would use examples of Chuck Norris, Sarah Palin, Julian Assange. If you are a F1 fan, I'd talk about testing through specific GP incidents. That's how the examples need to adopt based on audience. I also use a lot of examples from what I think the world connects to. For instance, I closely read and watch Air Crash Investigation in Nat Geo channel. OMG! There is so much to learn from the way NTSB (National Transportation Safety Board) deals with it. So, basically as a coach, I have to keep connecting to various thoughts and happenings to be able to connect with my audience. 

Similes, Metaphor or if you choose to call them Analogies are interesting and tough piece of cake, if you are the one providing it. People tend to take it in their own interpretation than what you intend to. It is good in a way, I get to learn when not to use what type of analogies to what kind of people :)

I use Fishing for explaining test coverage. I know a few other people have already used fishing as an example for teaching testing, so I didn't invent it but I know how to explain different concepts of testing with the fishing example. I think that most learning happens when you are interacting with the audience and not when you are explaining. That's where I do better with the fishing analogy.

I start drawing different types of fish, from guppies to clowns and from star fish to sharks. I then draw a size and shape of a specific type of net to catch fish. I give them a goal of catching as many different fish as possible in a limited amount of time. Some audience come back and tell me, "Ah with this net the guppies will escape"... what's your immediate thought? You would tell them to build a net that has smaller squares to catch guppies?  (Why say it? They know it) I would probably be Pradeep and say, "Fantastic. A lot of time spent celebrating the success of finding one good bug steals away the opportunity to find more good bugs. How do you want to celebrate? 3 minutes left."

So, after they come out with strategy and different nets, I tell people that just because they have nets (tools) to catch a specific type of fish it doesn't mean they can really catch a plenty of it. I then tell a story and examples from my life. One of them: I consulted for an organization who had bought an expensive automation tool hoping that they would now be able to find more bugs. They spent all their energy to set up tests on it and found fewer bugs over the year. I drive points like: Tools don't help you unless you know how to help the tools to do what you think they can.

For a while people think it is possible to catch a shark from a net. It is possible, maybe. I explain how a powerful shark can bring their boat down if they try to catch it through a net or a fishing rod and how a harpoon is better than a net. Sometimes people try to think of similar approaches to solve many different problems. The example of shark, net and harpoons are cool for people to relate to something they have done in the past that shouldn't have been done the way they did it. I then equate different types of fish to different quality criteria. I ask my audience what type of fish are they mostly catching and I hear a shout, "Functionality". 

After a lot of back and forth between me and my audience, we all discover what good test coverage could mean. I then start probing into their projects and figure out how much of fish they are missing that they shouldn't be and try to help them understand why they shouldn't be catching too much of the same fish. Sometimes it upsets the food chain :) 

While there is fishing in my class, there is also plenty of room for Sine and Cosine for Test Techniques, Brian Marick's Minefield Analogy for Regression Test Strategy, Tom and Jerry examples for How Scripted Testing is dangerous, what lessons can we learn from Saurav Ganguly's come back, how to read Sachin and Kambli's career graphs, farming... It must be fun to sit in my class. I don't know, I have never been able to.

Happy New Year!

Tuesday, January 25, 2011

Story of how a hang becomes a crash (because testers love reporting crashes)

I am doing my first exploratory testing workshop for 2011 and first from Moolya Software Testing Pvt Ltd. and I am excited about it. You can find the details by clicking on this link . If you are in India and plan to attend it, please do it quickly. I usually don't take a lot of participants although I know I'd gain more money out of more participants. Ever since I announced it yesterday, there are 4 seats that got filled within 5 hours.


Now, I am going to tell you a story that I had been hiding for quite a while. Not intentionally hiding but was planning to blog about it and the time is right now :)


I provide many applications to test in my workshop. At the end of a testing session, I check with the participants if they found any crashes with the application. Not to be surprised, a room full of testers, they do report crashes.


I ask them to reproduce it with me watching what they do and here is a shocking information; most of what they report as crash weren't crashes at all. What you are about to read is also a great example of confirmation bias that I have witnessed.


Here is what they are doing: They are performing an input constraint attack or providing a large set of characters as an input to a field and hits the "Submit" button. They wait for 10 seconds to see what the application does and on seeing the application in not responding state, kill the application and report that as a crash. 


Awesome! Isn't it?


Now, I got conscious of the fact that testers seem to be reporting a hang as a crash and was keen in looking at live projects during my consulting assignments. I get access to the bug tracking system during my consulting (wow) and I see patterns of such reports. 


I go filter out some of the crash reports from the bug tracking system and try to attempt what the tester did to report that type of a crash and bang, that's a hang.


I have made this point to testers who attend my workshop in order to help them be more conscious of what they are seeing versus what they are reporting versus what they are eager to report versus what they should be reporting. 


It seems to me that testers have an anxiety to report crashes. There's nothing wrong in it but it goes horribly wrong when you report a crash by fooling yourselves and the people around you. I recently blogged about the Obsessive Checking if being mentioned disorder that I was suffering from and here is a relevant excerpt from it, "Months later, I started replacing every word by my name till I saw my name on others post. So, I may have read a few posts without actually learning anything from it because all I saw is "Pradeep" & "Tester Tested" on those posts."


Now, why is that relevant to this post? I see that the testers I have witnessed who report hangs as crashes also appear to have a similar problem of obsession towards reporting crashes that they appear to not see a hang but see a crash. I would love if you ask if these testers ever report a hang? Yes, they do. If an application recovers before they kill, that's a hang.


I understand that an application might have hanged because it has crashed but how do you know? Oh yeah, allowing the GUI to fool you and me?


Here is an excerpt ( and tweaked to hide confidential information ) from an issue that I reported on Sep 30, 2010 @ 12: 13 PM IST (thank you Jira) while testing a new OS that is soon to be launched. No, not Chrome OS.


"I opened Media Player application and made an attempt to subscribe to a podcast. 
The application appeared to hang and it also did not allow me to close using the X mark on top right corner.


In an attempt to kill it, I navigated to XXXXX (product name masked) and tried closing it from there. However, the XXXX is open and Media player still seems to be remaining in the hung state.


From that point, I might be forced to reboot the system to get out of it or can escape by waiting for a longer time that I waited for Media Player to recover (>5 mins). No user to my knowledge would wait for more than 5 mins to see the Media Player function. Even a reboot appears to be a faster option" 

Now, if that was the Description, here is the summary :

"Unable to kill a XXXX when the application in it appears to hang or is busy"

  • Why I posted the above? Is it because I wanted people to note the usage of the word "appears"?
  • Why did I make a few words in it bold? Is it to highlight it?


Long ago, well, not so long ago, I blogged about how teaching testing is helping me test better. The above post and the bug report I posted, is another example of how teaching testers has helped me do it a lil better.


Now if you are going to be sitting at my workshop and be worried if you are going to make mistakes that I am going to highlight, please be informed that you are paying to my workshop to get into the safest environment to fail. No managers watching you. No clients frowning at you. No appraisals being done. Just your own dream of personal excellence aching you for not being there yet but being happy that you know why you are not there. Its a wonderful feeling.


Now, for some marketing again :) You can find the details of my upcoming exploratory testing workshop titled "Accountable & Manageable Exploratory Testing" by clicking on this link 


Happy Republic Day folks!

Monday, January 10, 2011

Evolving styles & small "e" exploratory testing

When I first heard about the small "a" agile and the big "A" agile, I was intrigued. I was wondering if it really did exist. Later when I interacted with those who were agile and those who claimed to be agile but were not, I started believing in it.

Similarly, I am going to be talking about the small "e" exploratory testing style.

Every tester I have interacted, tested in pairs, watched them test... have their own style to testing while being exploratory. My own style has changed over the years and its evolving. To detail it out, there are several dimensions.

First dimension

  • I Did exploratory testing without knowing it was called that way. To...
  • Did learn the term exploratory testing but didn't know if that's what I was doing. To...
  • Did exploratory testing based on my own ideas of how it should be done. To...
  • Did learn that others have their own style to exploratory testing. To...
  • Did practice other testers style of exploratory testing combined with my own style. Plus...
  • Learnt that to become a decent exploratory tester, I need to develop various kinds of skills. Plus...
  • Started to ask for mission, charter, objectives before exploratory testing. Plus...
  • Learned about SBTM and practiced exploratory testing with SBTM (and failed at my first trials). Plus...
  • Learned to modify SBTM to my style and my context needs (current state). Plus...
  • < To be updated >
  • ...
Here is the second one.
  • Was clicking on every visible GUI stuff to see if a bug dances out. To...
  • Thought I was doing good exploratory testing if I was finding a lot of crashes. To...
  • Thought good exploratory testing should always result in finding a lot of bugs. To...
  • Learnt that bug investigation is purely an exploratory process. To...
  • Learnt that test designing skill is important and learnt & practiced it. Plus... (note the plus?)
  • Learnt being conscious of the test coverage is important. Plus...
  • Learnt that making notes of all my observations and inferences is important. Plus...
  • Learnt that briefing and debriefing is not just like any other discussion about testing. Plus...
  • Practicing to debrief. Plus
  • < To be updated > 
After second, usually its the third dimension
  • Thought of exploratory testing as a technique. To...
  • Learnt exploratory testing is an approach. Plus...
  • Practiced applying several techniques in exploratory approach. Plus..
  • < To be updated >
The fourth
  • Unable to explain what I did after a few hours of testing. To...
  • Being able to verbally explain what I did. To..
  • Being able to explain and document what I did and why I did that. To...
  • Being able to do that much better over practice. Plus..
  • < To be updated >
Then comes the fifth
  • Being able to do all that and trying to keep it secret because I wanted to be a hero. To...
  • Realizing that I can better only if I share with others. Plus
  • Started to coach testers. Plus
  • Started to learn from them. Plus
  • I recognized that if I had kept it as a secret, I would have been zero and not hero. Plus Plus,
  • < To be updated >
Semi finally, the sixth
  • Aping what others are saying. Plus
  • Trying out my own ideas. To
  • To forming my own style of doing it. Plus
  • Trying out different attack (not the attack you know) as a starting point to my testing. Plus
  • < To be updated >
Se7enth
  • Thinking of all exploratory testing sessions as freestyle. To...
  • Learnt that there are loosely scripted & checklists types. Plus...
  • Practicing loosely scripted, freestyle, checklists types. Plus
  • Learnt that a good exploratory tester cannot be determined in the first few hours of testing but what comes after that. Plus
  • < to be updated >
Value

If you were wondering why I moved from one style to the other, I want you to be informed that it was for being more value to my customers and not because someone said so. If you are doing anything because someone said so, without knowing if it suits your context or what value it adds, you are most likely doing the big "E" exploratory testing. Also, if you don't know what many serious testers who are practicing to be good in exploratory testing are talking / practicing, you still are doing the big "E" exploratory Testing. If your style of exploratory testing hasn't evolved over the years, you are most likely to be doing the big "E" exploratory testing. Nothing wrong in being the big "E" Elephant. Everybody started there. Everybody were stuck there. Not everybody remained there for a long time.

Also, you may ask if the points listed above indicate anything about the style. In case you did ask, I want to thank you for that because I plan to write about it here. I mean, here --> As we learn and unlearn, the way we test, changes. This influences the style. 

For example, imagine me thinking of all good exploratory testing to be yielding tons of bugs, I'd grow to be a manager who'd question someone why they didn't find a lot of bugs over the last one hour of exploratory testing and hence making myself stupid.

If you were to take another example: Learnt being conscious of the test coverage is important : Everybody in their first few years of testing, at some point, lose the sense of test coverage they are achieving. They get carried away by bugs and as a matter of fact, the client is happy too because they are seeing important bugs being found. Currently my central theme of exploratory testing is test coverage. I ask myself, "OK, I found these bugs and they are great but what is happening to my test coverage". The client may not know to ask for it but it is my duty to help them with that information.

This one, Yo! Being able to explain and document what I did and why I did that. -  I personally know many testers who seem to do good hands on exploratory testing but being unable to narrate a story of what they did, makes them look bad. The senior management appears to just look at the bugs they report and hence the good exploratory tester looks like a "not so good" exploratory tester. Today, I can tell an engaging story of why I did what I did and what value it adds to them and also why I should be given more freedom and hence more responsibility.


Try this! Practicing loosely scripted, freestyle, checklists types. Plus


Based on how accountable I need to be, I choose which type of exploratory testing is needed. That only comes with practice.

Recently, I was asked to explain my test coverage & strategy. Why did that arise in between the consulting assignment? When I started my first round of testing, I was finding bugs as I learnt the application, then I stopped finding bugs because I was doing some confirmatory tests and during that time, the client was wondering, "Is that all about Pradeep?". A very good question to have asked. So, I presented them with my strategy and coverage plan. Here is snippet from that. (Taking out all confidential information). Level 1.


- Toolbar options
- Mouse hover / clicks / right click / left click
- Keyboard shortcuts
- URL / Path editing
- Menu options coverage
- Claims based testing
- Questioning the purpose of existence, position, alignment and meaningfulness of all UI
- Consistency within the application
- Consistency with Windows standards
- Confirmatory testing
- Process monitoring & control
- Security vulnerability
- Consistency of text, font, sizes, options
- Links / Broken links / paths
- Reliability of user login / logout
- Violating policies - Firewall, System, Third party software
- Hidden details & options
- Scenario based tests -
- Boundary tests
- Hardware based scenarios - Interrupt
- Timing based coverage
- Error message coverage
- Tools / Utilities
- Independent components
- Extended usage
- Multiple connect / disconnect
- Reboot / Sleep  / Hibernate
- Close abrupt / Kill
- User coverage
- Usability standards comparison
- Functional checking
- Supportability testing
- Accessibility
- Navigating between zones / desktops / sessions
- Platforms
- Devices coverage - Bluetooth, Printer, Memory, Ext HDD
- Authentication / Authorization
- Splash screens
- Obvious / non Obvious things for a user
- Brand image versus product behavior
- Comparable products / benchmarking
-  Testability tour
-  Behavior patterns, user expectations and consistency
_snip_


At Level 2, I had a checklist of what tests I did.

Here is an example: http://support.microsoft.com/kb/301583 : I ran through each one of them. I could tell what shortcuts cause what problems and an investigation about the problems. They were highly convinced why they were paying me as much as they did. I just stood up to the scrutiny.

At last, Trying out different attack (not the attack you know) as a starting point to my testing. Plus


Most testers use Functionality / Security as their first few possible ways to start attacking (start testing) a product. I was practicing to try out different things trying to determine how useful it is for short and long term projects. Testability attack has yielded great results for me than any other form of attack. Now, I might have learnt a lesson that the world already knows but what matters is, I did learn it. Hurrah!


If you were to determine how skilled an exploratory tester is, its not the first few hours, its what comes after the first few hours. That's what determines the the big "E", exploratory testing style and the small "e" exploratory testing style.

Also, our testing industry is so damn confused that they might end up asking you about the big E and small e exploratory testing in interviews by those who might happen to read this blog post half way and run to a meeting. Tell them that Pradeep used the small e and big E to explain styles and not to define or redefine anything about exploratory testing. Also, tell them that they didn't read this post completely. 

Friday, December 03, 2010

Future of Indian software testing looks safe

I have spoken bad about testers in India in the past. I received an applause for it from Americans, Europeans & even Indians. I didn't consider it a sin. There is an equal (no, wait, "much more") applause if an American was complaining about testers in India being bad in testing. I just didn't mind that. That was a few years ago.


2 weeks ago, I had an opportunity to speak at STC2010 conference in Bangalore. The turnout was awesome. About 400+ people. For the first time, I witnessed latecomers looking out for chairs that were vacant. You could simply call it house full.


Critical thinking demonstration


For me, software testing conferences in India is more about meeting interesting people (who don't have a public presence, yet do lots of good work) and less about the keynote speakers or their content. Some talks, in my humble-less opinion were pretty bad but that's why the good ones amidst them were shining and bright. 


Ajay Jain, from Adobe, Noida in particular gave a talk that I stood up and clapped for what he said. He said things like, "If something takes time, the immediate instinct is to automate but how about letting certain things take its own sweet time because there could be a value in it."


There appears to be hungry vultures waiting to find an opportunity to automate something that is taking time and amidst those people, Ajay Jain, is a star because he is thinking critical. I didn't know him before that talk. Later, I went up to him and talked a good deal. There are many undiscovered Ajay Jain's in India.


ET & SBTM experience reports


Moving to a poster presentation by Shaham Yusuf and his lead Vivekanand Suman from Delloite, Mumbai where they published an experience report of exploratory testing & session based test management. They talked about how their exploratory testing influenced the developers and in turn how its doing good to their product. The most important thing about these guys is that they had sought permission from their organization to present some of their actual reports in a conference and allowed more people to know that exploratory testing and session based test management is put in use at large organizations and in large projects. 


Heading the certification campaign by opposing it


I then picked up a conversation with a tester at the conference lobby on certifications. She works for a large organization and is responsible for certifications in her organization other than her usual work to find bugs and report them. I assumed she was certified too but it turned out that she wasn't. She doesn't believe in certifications and had the guts to say, "Don't enforce a certification on me. You want me to be a good tester, I can prove it to you at work. You are welcome anytime to my project and I am willing to answer your questions about my work" to the senior management. When I was thinking that I know of all testers in India who are as bold as me, she proved me wrong. 


So, how is she leading the certification responsibility? She is helping people choose a certification that suits their mindset. When she identifies a tester wanting to improve the skill and not for the sake of getting one, she is making those people aware of BBST course. She said, "If I didn't take up the responsibility of leading the certifications group, then I am not sure if the other person would have suggested BBST for a few to whom I did". 


Fantastic. We would imagine girls in India to be the shy types and say, "I want a 9 to 5 job. Got to take care of my in-laws" but then someone like this (and of course Parimala, Meeta, Jassi, Krishnaveni...) are a blessing to India and its future in software testing.


Testing in Testing Institute, not Certifications


Then I met a person who is running a testing institute. I had perused his website sometime back and after seeing ads on certification, I thought, "Yet another testing institute wooing people with certification" but talking to him changed that perception. He said, "Ah, you believe everything that is there on a website? Shouldn't be doing that" and continued, "Our institute focuses on trying to help people develop testing and thinking skills. We don't stress on certifications or their content but still if they want, we don't say No".


These people are like soldiers in the border of your country guarding you, whose names you don't know. In the above two cases, I feel, you shouldn't know their names. They are doing a fantastic job. I think they should come out and speak in public what they spoke to me after they have achieved some more great success.


For more...


Our own test automation power


I was glad to meet Narayan Raman, the developer and product owner of Sahi, a web application testing tool. We had met more than a couple of times in the past. He was a very special invitee at Google Test Automation Conference 2010. Over the last two meetings, I realize how much important is Narayan Raman for India. He has the zeal, skill and enthusiasm to put India on a higher scale. He is giving a run for tools like Selenium and with his tool starting to support Flex from next couple of months, I think Sahi is a rock star. Check out the comparison between Sahi and Selenium 


Weekend Testing & Weeknight Testing


If you don't know about Weekend Testing, you should visit http://weekendtesting.com and spend enough time there to learn this big revolution started in India and now the whole world seems to be catching up. Americans are extremely happy & excited of having their own chapter. Europeans are enjoying it. It was also featured in Eurostar and got a standing ovation. James Bach had a dream of seeing Weekend Testing becoming Weekday Testing. His dream got closer to reality during London Testers Gathering where Mike Scott proposed the idea and Sharath joined the bandwagon with some other good people to kick off Weeknight Testing. I didn't believe till then that it's only in United Kingdom that (K)nighthood is bestowed to people. The knights didn't wait for the queen though.


Peer conferences


Bangalore Workshop on Software Testing is a peer conference inspired by my own experience with Toronto Workshop on Software Testing. We have been running it over the last two years and the next one is coming up in Feb / March 2011. We are planning 2 days instead of just one by looking at how many more people want to join and how much they are enjoying it. 


Exploratory Testing & Rapid Software Testing


The interest for Exploratory Testing & Rapid Software Testing has grown to a great extent in the last few years. I myself have trained about 1000 testers on it ever since I started doing Exploratory & Rapid Software Testing Workshops. You are seeing a lot more testers demanding freedom and ready to take up that additional responsibility that accompanies freedom because they are working on their skills.


Hands on Testing Coaching for College Graduates


I don't know if you have gone through this report on excerpts of work done by participants of hands on testing training. There were businessmen in India who volunteered to allow me to experiment a complete hands on testing training with hardly one hundred slides for one month of training.


The outcome of of doing this with one batch is, we have Santhosh Tuppad as the multiple bug battle competitions winner who is giving a run for other country testers (and even other testers from India) a real hard run to be able to win the bug battles he is competing. The other people who chose not to write and work as public as Santhosh Tuppad are doing excellent and their employers are way too happy to pay them well.


Good blogs from India


When you were thinking a lot of testing blogs from India are horrible copy paste and plagiarized stuff, you also saw the rise of some good bloggers. Parimala Shankaraiah stands as one who started blogging less than two years ago and people like Lisa Crispin who is the author of the book Agile Testing and has been writing for long, considers Parimala as her  hero. We have many other testers like Dhanasekar, Nandagopal, Vipul Kocher, Rahul Verma, Ajoy Singha, Santhosh Tuppad... joining to the band of good testing bloggers. The last few recently started blogs never had a copy paste but original content. Testers have started to write down their experiences. 


So, So, So, So, So, So, So, So, So?


A couple of years ago, if you heard someone talk bad things about all testers in India and you laughed at what they said because you thought they were speaking truth, you did the right thing. We were bad.


Henceforth, dear other country folks, if you hear someone talk bad about all Indian testers, I still encourage you to laugh but for a different reason that they don't know or are ignorant about what is really happening here.

A note to Indian testers: On a second thought, I wouldn't encourage you to laugh at them if they are from America or Europe or elsewhere because some Heads of Testing in India themselves don't know about all these. Silently giggle if you are working in one such company and get on because you are the future. Ensure, the future generations don't giggle at you because you are not going to know that even if they did. Is your Head of Testing aware you giggled right now? If giggling isn't your types, then go educate them.


Remember, you are the future and work even more harder and smarter. 


Jai Hind!

Wednesday, November 03, 2010

Software testing black swan bites cause pain

This post is about an experience I had recently. An experience that proves to me that there are more hard working people than me and I shouldn't feel too proud of what I have achieved so far. 


I was at a conference recently. I was at several conferences, so don't get confused with what is shown in my Events page. This conference was not listed and doesn't have a conference page either. It happened in Bangalore on 2nd Nov, 2010, right after my return from Google Test Automation Conference - Hyderabad, India.


Due to my popularity, I was hoping people would come, recognize me and talk to me about their testing problems. It just didn't happen. Some other person was getting all attention and there were people surrounding him and asking questions. There were so many people around him that I couldn't get a chance to see the man who was getting all this attention. I was wondering who is this guy getting an edge over me in India?


I then thought it must be someone who traveled to India from a western country. Only then people forget their local guy. Nothing new to me. I was just waiting for my chance to shake hands with that guy and talk to him about testing. Well, I just wanted to know what makes him so special that he is getting all attention in India. 


Check at my stats, I am supposed to be the popular guy out here. If you are seeing the popularity hungry Pradeep now, I must say; I too saw him.  The difference between us is, you might feel ashamed of having known Pradeep and I don't.


An hour passed by and still I could not get to meet that guy. I saw a tester coming out and asked her, 
"What's about the guy in center?" 
"Oh, you don't know him? He is an expert in test estimation"
"Everybody in here is an expert, din't you know that? So which country is he from?" 
"India" she said. 


What? an Indian? and I don't know him yet? I know everybody who blogs from India. At least, everybody who blogs from India knows me. How come I don't know about this guy who seems to be more popular than me? Maybe he doesn't blog but even then I should have known him.


All this was driving me crazy. Added to that were some of the talks I wanted to attend and it had started. I just pushed myself into one of the track hoping I could catch hold of that guy at one of the lunch tables. I have never waited for lunch so much, not even when I was very hungry. I wanted to meet this guy. He was a challenge I wanted to face.


At lunch, same bloody thing that happened in the morning repeated. People surrounded by him and I just cant get to meet him. My ego hurts me a lot if I have to go introduce myself to him amidst other testers who might think that I am not as popular as they thought. So, I picked up a plate and tried to eat alone. Fortunately, some testers who couldn't get to talk to him saw me and approached to have a conversation. My mind was somewhere else. I guess I don't know if I did answer the questions those testers asked me. Maybe they would have stopped reading my blog as I don't know how crazy my answers were. I just wanted to meet that guy.


Finally the moment arrived. The only way I could corner him was in the washroom. I was waiting for him there adjusting my shirt and trousers making it look to other people as though I care too much about my how neatly my shirt is tucked in. There he came. I didn't mind if his hands were wet but just put my hands forward and said, "Hi, I am Pradeep Soundararajan". He shook hands with confidence and said, "Oh, I know you. I read your blog and follow your work closely". 


At one end, I felt happy that the man who was sought much more than me follows my work but it was still aching as to how this guy managed to be the center of attraction amidst my presence. I took courage and asked him, "How come I don't know you. What's so special about you that these people are flogging you?"


"I have learned to help people estimate their work in a way that makes them feel successful following my advice" he said that with a soft and gentle tone. I put a step towards the door closeby and turned to him and said, "Why don't we discuss this off the washroom?"


My intention was to steal the idea. After several years of hard work, I can't allow someone to steal away the limelight I have been enjoying. When I say I wanted to steal, I mean, I wanted to know what his education was. How different was it from mine? 


We sat on a couch and I asked a question that was designed for deception or to learn about what he has learned


"So, what's your source of learning?"
He had a smile on his face before he said, "I read Bach, Bolton, Kaner, Jerry and you"
"Sounds interesting. I do the same too but how come you seem to be doing better than me?"
"I don't know"
I was pissed off but couldn't let it out because I still hadn't got the secret out of him. 
"So, you are suggesting that you learn something more from them than me?"
"No, I haven't met them Pradeep and they don't know about me"
"Pretty sure because if they did know, they would have let me know about you"
"Yeah"
"So, let me stop beating around the bush. How do you help people with their estimation problems?"
"I do it ............................................................................... this way"
"Wow. That's cool"
"Where did you learn that?"
"You are so humble Pradeep. You have read it, too. I picked up ideas from Michael Bolton's Test Estimation & Black Swan series of posts, experimented with them, made my own notes, refined them for a while to arrive at this point" and then he walked away saying he had to deliver a talk and it was getting late.


If you had been to the conference, you would have seen me crying on the couch post that meeting. I wasn't crying because someone gained an edge over me but was crying that I learnt the cost of not reading those lengthy posts just because it was lengthy. 


I finished crying and went around looking for that person to thank him for the lesson he offered. He had already left. The series of posts from Michael Bolton on Test Estimation & Black Swans had been lying there on his blog and I just kept feeling lazy to not read those lengthy posts. I am probably in the Twitter Era. I want people to say anything great or stupid in 140 characters and I also hope they say that around my timeline. 


Walked around with disappointment. I decided to go out of the conference venue. I wanted to go home, have a drink and get a tight sleep to forget all this. I thought I was reading everything by Michael Bolton. When he posted about estimation, I thought I had already read enough of estimation from him and he was packaging the same stuff. Just then, the conference was getting over and the final lightening talk was mine. I was called to the stage. I went on the stage with tears still dropping at 1 drop every 10 seconds, forgot about my talk and asked the audience in a shrill voice, "Have you people read the series of post on Test Estimation & Black Swans from Michael Bolton?". 


The responses were, "Its lengthy", "We didn't find time to go through it", "I got a call in between and almost forgot to continue", "I was too busy with my project". Almost everyone were saying the same thing, "No, I didn't read it because it was lengthy". I laughed out loud and walked away as though I had seen myself in hundreds of mirrors placed in front of me. There are so many Pradeep's in our industry. Some Pradeep might not even have got this far on this post because he might have thought, "Oh, this is lengthy". 


If you don't look like Pradeep when you stand in the mirror, here are the posts:


Project Estimation and Black Swans (Part 1) 
Project Estimation and Black Swans (Part 2)
Project Estimation and Black Swans (Part 3)
Project Estimation and Black Swans (Part 4)
Project Estimation and Black Swans (Part 5): Test Estimation


The next time you don't want to read a post just because its lengthy, remind yourself that if you miss spotting the Black Swan, it doesn't mean Black Swans don't exist.  They bite hard to remind you that they existed and you didn't pay attention to them.

Sunday, October 03, 2010

The domain particle

The inspiration


The title is inspired from Angels & Demons where they talk about the "God particle". The topic is  inspired by so many testers asking or answering to, "How much important is domain knowledge?", "Can we test without domain knowledge?", "What happens if we test a product or technology without domain knowledge?" ... 


Every person who writes in public and in forums have been asked these questions about the importance of domain knowledge to test a product. Going through those forums, you'd know that there are answers like : Yes, domain knowledge is very important to test a product OR Oh yeah, you know without domain knowledge I couldn't have found the show stopper I found a few days back...and others saying, "I did manage to learn quickly, so it doesn't matter as long as you too can", "I think it depends on what you are testing"... 


Passing time by failing to respect others time


The question I have to most testers out there is. If we, who reply to all those questions of yours, say that it is very important that you should be an expert of the domain to test the product, are you really going to become one or even try to? 


You are aware that bug investigation skills are important, without we having to say that, what have you done about that? or would you go pick up the test framing skills?


I am convinced that there are many hundreds and thousands of testers who are a trap not just to themselves but to others wanting to help testers. Every testing related forum is infected with people asking questions who actually don't intend to do anything about it despite unintentionally (or maybe intentionally) wasting a lot of passionate testers time & energy. There are genuine people in there but in a rare occurrence. Most genuine people aren't asking questions but answering them and not all who answer questions are genuine. You may want to consider me among those who are answering questions but not genuine, if that pleases your ego.


I have respect for CDT mailing list and Software Testing Club. There are serious people, asking, answering and watching. I just hope there are a few more forums like that.


Why do we have labels?


Some testers are in a trap of calling themselves "Telecom testers", "BFSI testers", "Web App testers", "Mobile App Testers", and what not? The question I have is, should it matter? BTW, I call myself an exploratory tester because its an approach I follow to test any software & not a domain.


As how programming is a mindset and a good programmer wouldn't call themselves "Java only programmer", testing is a mindset too. Oh, there is a skillset along with the mindset.


Henrik Andersson of Sweden is going to do a Webinar on October 12 : Do we need labels? Are we all not testers? I read the abstract of the webinar and was excited. I envy Henrik a lot. He is the guy who is blessed to be in the happening places - be it PSL, AYE or CAST. Other than that he is the energetic test consultant & partner at a testing consulting firm based in Sweden. I think you all should register and listen to what Henrik has to say. 


Henrik made me ask these questions: Why am I calling myself an exploratory tester, rapid tester, and whatever fancy stuff I had been calling myself when I know all testing is to some degree exploratory? What is the need for me to do so?


I need a label to differentiate myself or to communicate to people what I specialize at. My label helps my potential clients to gain interest to check about my services based on their needs. My label acts as a filter. People know what questions to shoot me. So far I have received very few emails from people asking if I can help them record and play QTP scripts. Most emails are about seeking help on how to develop themselves, think & test. I love to talk to them because they teach me things I want to learn.


So, I see label as a filter but I am against the labels that filter my own opportunities to learn. For instance, if I call myself a "Multimedia Tester", I would have blocked my learning on many technologies and contexts that I have experienced. I am starting to use "Brainual tester" these days. I am also giving a lightning / lightening talk in GTAC this year about it. James calls it Sapient and I call it Brainual. We both worship the same God but we call them in different names. I prefer the word "Brainual" because it is quicker to replace the word "Manual" to those who see "Manual" as a hand activity of testers than brain. People are used to saying, "Lets automate those manual test cases" but I expect the reluctance to set it to say, "Lets automate the brainual work thats being done now".  Their own ego would hurt them. They don't want to be seen as a fool.




Filter-less


Being a consultant and added to that the twist and turns I have had in my testing life has got me to test software ranging from wireless, mobile applications, medical devices, multimedia, Retail, CRM, desktop applications, dating applications, billing solutions, video surveillance, stock market, auction systems, kiosks, cloud computing, testing tools, games and what not. I have never bothered what domain each of them belong to as long as I have 
  • An understanding of general principles of how software works (which I constantly refine)
  • A skill to quickly learn and convert the learning to tests & churn more learning out of it.
  • Ability to build models of learning to speed up my learning.
I try understanding how people who invented things might have thought. I fancy thinking that all these technologies have emerged out of observing something. Some of the key observations about human being and living things have led to many inventions and discoveries. Should I tell you that birds were an inspiration for Wright Brothers?


The human connection to technology


# 1 DHCP as to how it has evolved out of human behavior : When we go to the theater to watch a movie, we ask the personnel at the ticket counter to check if there are tickets available for a specific movie (sending request from a client to the server) of what we want to watch and our preferred time (details about client to see if server has anything for us). 


The ticket counter personnel in the theater gives us a ticket based on availability (assigning an IP address from the available pool) with the seat numbers to watch the movie.

In the ticket, there exists a seat number ( IP address ), duration of validity ( lease period ). If you prefer to watch the movie again and in the same seats ( Static IP ) you need to advance book or renew your tickets  
( ipconfig / renew )


# 2 Why do connectors have a male pin & a female pin? How did gender come in to technology without having observed humans and how they interact?


I have a huge list of things that computers/ technologies do which are based on how humans communicate or how humans could have. Think of SIP protocol or a client server host architecture and relate it to human communication. You'd probably enjoy as much as I do.


Simplification


In Rapid Software Testing class of  Michael Bolton & James Bach, there is an interesting dice exercise. I didn't crack it the first time but on learning the meta pattern, I am likely to crack it for any pattern or will get closer to solving it.


I have my own version of the dice game but with a different learning objective - simplicity. Based on running the exercise on thousands of testers in India (and a few folks outside India), I am making a conjecture that I see people struggling to cope up with simplicity. 


The first set of thoughts that comes to a human mind is not necessarily simple and the next set of thoughts are usually more complex than the previous ones. So, I think humans keep building on the already existing complexity of their previous thought.


Unless you train your brain to think simple, you are unlikely to crack many things you can.


The reason I am telling this here is because those who set out to learn a new domain forget that there are simpler sub systems of the domain they already know either because they also exist in other domains they have tested or have used such products extensively. You may want to figure out the meta pattern of software & how they are supposed to work. 




Practice to be fit


Quoting Parimala, "What you know is not as important as what you can do with what you know". 


I run an exercise in my workshop in which I allow people to do freestyle exploratory testing. At the end I ask them the techniques they consciously used while testing. Although many of them know many techniques to test, at least the first 10 minutes of most testers freestyle exploratory testing is, "Clicking here and there to see if a bug dances out on the screen" If a few bugs do dance, "Wow, you see I did exploratory testing". I think of that as an inferior self standard setting to understanding exploratory testing.  


How do we use what we already know to achieve better results? PRACTICE!


The biggest shame for ISTQB / CSTE certification bodies doesn't come from those who oppose it but from those who are certified & don't show traces of having gained anything from it. I have consulted at least for a few organizations here in India who hire these certified testers. Not a single test case document that these certified testers have produced has anything related to what they might have learned from the certification. 


There is a scripted test design technique that is a best practice and the most widely used one although it doesn't have a name - copy paste a sentence of requirement document into expected results column and then write the test steps accordingly. This is the most successful trick ever invented in testing. 


As a side note: Do you play any outdoor sport? Have you taken a long break from it and then went back to it? You might have been good at it at some point but when you get back after a break, you no longer feel the same comfort you had. If I were to speak to Indian testers alone for a moment, how does it feel to hold a cricket bat and face a fast bowler after taking a break from cricket? 


Bits & Pieces


I admire ants. They have a way to deal with problems. They don't say, "Oh my God! That cake is about 1000 times bigger than my size. How can I eat it?". They break the huge cake into bits that they can process, carry it home and then come back for the next bit. Doing it bit by bit helps them to achieve the goal of moving the entire cake into their colony.


While testing a product whose domain / technology that I don't know, I try to remember the ants. I learn one bit, use that bit to frame tests. The tests I perform with the bit I learned, help me learn about more bits of the system and I choose to eat bit by bit or byte by byte if my mouth has got bigger.


I may know nothing about a system when I start but at the end I can learn so much about it that it would amaze me or my audience if I tell what I discovered going bit by bit. I demonstrated this in one of the workshops where I learnt the user base of the website, traced the kind of users, identified why those users might be coming to the site, what kind of problems such users might be facing, what kind of tests to be run, what could be the most important problems that is making the audience to not give more sales + found 14 issues + 10 questions to the developers - all in one hour. I had enough feedback to the development team that they got busy working on a few things that gave me the time I wanted to go learn what I wanted about the product.


Live oracles


What are the business analysts doing? Are your sales and marketing folks so busy that they don't have time for you? How much do you interact with them? Have you invited them for a paired testing? Have you talked to them about the importance of they being with you for an hour in a month while you are testing?


If that's one set of questions, here is another set: Why in the world do you want to know everything about a domain when you know its not possible for any human being to do that?


Fooled by foolishness


Some organizations who hire testers only because of their domain knowledge, are fooled in other ways such as, those testers probably know little about testing to be called as testers. Its opportunity cost. I have spent all my time trying to be a better tester, you ask me to troubleshoot a network router, I may end up testing it or learning about it by testing it. Ask a network admin to test a router, she may end up troubleshooting or reconfiguring it.


Some organizations who continue to think that domain knowledge is the most important skill for a tester to even apply or be interviewed by them are blind to the fact that most issues that their customers are reporting has probably got nothing to do with having testers with extreme domain knowledge in the team.


There are tons of testing problems these organizations are not paying attention to, while they see their problems are because they don't have enough testers with enough domain knowledge.


Everyone of us choose to be not fooled by something and that exposes us to be fooled by something else.


The domain particle


Do you have a domain particle that makes you think "domain knowledge" is the important thing for a tester to perform well? 


I don't want to take it out. I want it to be there so that I can help you mutate it. If you could help yourself mutate it, and make the domain "learning" instead of "BFSI", "Telecom", "Multimedia", "Whatever"... I think you would have cracked what you are likely to in 20 years from now. That's an opportunity to be wise without needing to age for that.


Did you say that?


Those who speak about exploratory testing are asked, "Are you saying there is no value for scripted testing?". Those who speak about using brains to test are asked, "Are you saying there is no value in automating tests?". Those who speak about Check Automation are asked, "Are you saying checks are not tests?" while the speaker/author didn't mean any of that. Those who speak against certification are asked, "Are you saying people shouldn't get certified at all?". Those who speak about wasteful documentation are asked, "Are you saying documenting is a bad idea?". Reading some of the above paragraphs, some of you might have had a question, "Are you saying all ISTQB/CSTE testers don't know test design?"


So, whatever anyone says, there is a group of testers aggressively waiting to ask such questions. They are not doing anything wrong. They are just being themselves. Some representatives of the group are likely to ask me, "Are you saying domain knowledge is not needed to test the product?" and I am going to punish them by asking them to re-read this big post.


BTW, I am looking to hire testers of Banking / Web Application domain testers. Please send me your resume as soon as possible. 


Also, if you are a tester from Hyderabad and want to meet Rahul Verma, Dhanshekar and yours truly on 27th evening, email me. We wont mutate your particles! We will be attending Google Test Automation Conference in Hyderabad. 


Post bio for my self reference: Took 5 days to write the first draft, changed the style and contents 3 times, 1 external review, 3 self review, 5 hours of editing and finally publishing it. Published from my cousin's place in Delhi. Laptop: IBM Thinkpad