That's all folks!

I have decided to end this blog. There is no emotion or event behind this decision. I have to move on and I am moving on. I have had a great time writing for myself and to you through this blog. I am not bringing it down so whatever I have posted will stay. I am not adding anything more here either.     

That's all folks!

A question on exploratory testing from Quora

How exploratory testing differs with other testing? What is the common problems faced by a tester when performing exploratory testing?

.... 
And my answer:

You can cook pizza by following instructions or trying it out on your own. When you try it out on your own, you learn a lot more than merely following instructions. You don't get pizza a couple of times but once you get it, you would have exhausted many mistakes. The next time you have to cook any food, you will be aware of the kind of mistakes you made with pizza and would then ask a bunch of questions to avoid doing those mistakes. You may ask for the instructions but you won't treat the instructions the same way you would have treated had you not done your own mistakes. You have prepared yourselves to cook any food in this exploratory cooking process. 

However, you get a pizza faster when you follow instructions given to you. The goal is to not produce one pizza but thousands and if you need to keep following the instructions every time, the first pizza is faster but to produce thousands is slower. Accidentally if the instructions are wrong, you perfect the imperfect pizza. The next time someone asks you to cook pasta instead of pizza, you are asking instructions, follow it and pray God that the instruction given to you is perfect. 

There are competitions like Master Chef and what we observe is - the participants need to cook food and there are some standard procedures to follow to cook the same food. However, some people do it more awesome than others. Why? Their experience of doing mistakes with the dish, their skill and passion for doing it better and their skill to make the right decisions at the right time helps them win. 

Be a master chef. If you want to be one. Be one from today because nobody becomes somebody all of a sudden.

http://www.quora.com/How-exploratory-testing-differs-with-other-testing-What-is-the-common-problems-faced-by-a-tester-when-performing-exploratory-testing

Misfits of the software world

I just turned in notice rather than watch the exploratory test job I love turn into writing test cases to reassure PMs that things "work."

— Natalie Bennett (@bennett_nj) May 6, 2014

Natalie isn't just talking about exploratory testing, she is talking about value. However, the value she is talking and what the PMs want seem to be different. I know there are people like her across the globe. I am one of them. Welcome to the world of misfits.

Misfits don't fit into many organization who want lesser value than what these misfits could bring in. So how do people like Natalie fit in this world? They don't and they won't.

I am now a CEO of Moolya, the company I co-founded. A bunch of things I am doing (and loving) is not what I imagined I would do when I started this. What impacts Moolya's growth is how we can fit into this world. There is plenty of money if we fit into this world. So much that I will be super duper rich if I can fit myself in. I can be a multi millionare in an year. The same is the story with other misfits like me. The world is telling them; fit to us and you will be fine. However, for us, fitting into such a world, kills us. This world is like a black hole. It keeps sucking whatever it comes across and once something is sucked, it sucks the next available thing.

I am not having a pessimistic look at the world. There is a small little world of misfits too. The House of Test, Barclays GTC, Doran Jones, Moolya and there are hundreds of my friends in the testing community who are a part of this world of misfits. We give each other hope but we all fight wars at different locations with different people who think like the PMs Natalie mentioned. We don't give up. Natalie didn't give up. Look at the conversation following her tweet. She repeatedly tried helping those PMs see things but they had an optical and sound filter. They will only hear and see what they want to, and what they think is good.

At Moolya, we received an RFP (Request for Proposal) with questions for us to answer about how we test. The questions were in such a way that it kills testing. Some humble examples from RFP,

• What will be the speed of your test case execution? How many test cases per hour?
• What are your SLA's for defect leakage? Can you assure all critical S1 defects are found before release?
• Can you assure at least 96% of requirements coverage during your regression testing? 

This RFP was sent to other services companies but this customer only had trouble understanding our response to this RFP. We challenged every thing there and also proposed what they really want to know and how we have helped our existing customers do well. If we don't get the work (which is my pessimistic and at the same time optimistic view) they see us as misfits to their context. They are going to work with another service company and their revenues shall grow by millions of dollars.

Is this the end of the world for misfits?

No way. You read that right. The fight I am in is to make the current fits the misfits. The fight Natalie and people like Keith, Henrik Anderson, Huib Schoots are in is the same. I took to be a businessman because I thought this gave me more power to change things faster. It is helping great deal but the momentum isn't yet good enough for me to oppose the force that is pulling testing down.

  

I think the world is waiting for a tipping point. I see it coming. Every little thing the misfits do is helping improve the chances of reaching tipping point. Till then, the kind of chaos we are seeing with people saying testing is dead or want test cases instead of testing will continue. When the world changes, we will be kind to them and forgive them for their past life sins. We aren't Gods, we are humans. Humans are fallible. That is exactly what we have been telling and that is why we are misfits in many occasions. The misfits are reverse-fallible. We have been failing usually with these PMs, we need a chance of win and we don't want an occasional win. We want a win that stays. The tipping point, we are working towards you. 

-- An arrogant courageous misfit

PerScholas, Keith Klain, Doran Jones and how this industry is changing

I hadn't heard about PerScholas till 2013. I guess Keith Klain introduced PerScholas to the testing world through his leadership activities during his awesome stint in Barclays. The idea was definitely exciting but it needed more than an idea. Every idea needs awesome people to execute. 

I don't exactly know how Keith Klain found Doran Jones Inc to join as its COO but it was the best thing to happen to our industry, when I look back. At a situation where every country's economy is not looking good, Keith Klain and PerScholas seems to have struck the deal with software testing to improve America's economy. Ever since I met Paul Holland in 2008, I have become his fan. He is one of the most awesome guys I have met who has great balance and abundance of humor and charm. Paul Holland, James Bach and Michael Bolton have supported PerScholas in its training software testers and I know for sure for those who got trained, they are left with enough inspiration from these cool guys. Now, Paul Holland is also a part of Doran Jones and that is great news for the testers there.  

Read this news: 150 Software Testing Jobs Brought to Nation’s Poorest Congressional District Bringing Millions in Economic Revenue 

This is massive. As a businessman, tester and someone who cares for the community and industry of software testing, this news is one of the biggest I have seen in terms of bringing in a change. For Doran Jones Inc, this builds great respect. For PerScholas leadership team, this is what they dreamed to see, I guess.

This news brings hope to our industry that good people exist, and some good people have courage too and they act, not just speak. To the person of the year and years to come, I salute you.

Bangalore Workshop on Software Testing - 4

The BWST is back. There was a gap of 3 years after BWST 3 to get to 4. All because I was doing things that took my time away from community activities to build Moolya. Moolya and I are doing great enough for me to resume my community activities. So, as BWST is back so is Pradeep back to the Indian software testing community activities. I am not an easy guy. I have never been one. I have grown more uneasy over the last three years and hence the community would find great benefits out of my uneasiness :) . Let time speak about it.

Theme of BWST 4: The secret ingredient to great testing 

June 21st, Bangalore

I have secret ingredients with me when it comes to testing. I am the big fat Panda who believes there are no secret ingredients but however what I mean by secret ingredient is - something that already exist but has not yet become obvious to a lot of people. That also means - some people know it - some people practice it - some people are experts at it. I would like to invite people to come and present their secret ingredients to great testing that they practice. 

I have never presented at any of the BWST so far but I'd be presenting my secret ingredients in this one. Just to share with you here, one of the many secret ingredients I am going to be presenting. I was fooling myself that learning testing is a never ending process till I discovered the "meta" to learning testing. The meta to learning testing helped me understand testing in very simple ways. I can see people struggling to learn testing despite 100 years of doing it because they didn't work on building their "meta". If you go to the meta - things become simple and when you bring the meta to your life, things become obvious. 

Just because I have the meta - I am able to pass through all the noise people are creating to themselves and see things clear. This also helps me advice people well on testing. My posts on Quora are a big evidence to what I am speaking. My posts anywhere is a big evidence to what I speak. So is my work.

Do you have your secret ingredients to present? Please email your thoughts to pradeep.srajan at the famous mail service that starts with g with subject Speaking at BWST4. I am looking at experience reports. Your experience reports. There will only be 4 speakers over the day and hence giving so much more time for discussions. 

If you wish to be a participant but not speak at BWST4, then email me with subject Participating at BWST4. I'd like to hear why I should allow you into being a participant at BWST4? Tell me something really crazy or tell me how you are going to change the world if I allow you to be a participant. 

Some pointers about BWST

• This is a pay for yourselves (max 750 rupees) peer facilitated conference/workshop. 
• The max number of participants would be 30
• This is a full day event : 900 AM to 5:30 PM
• We will have a social drinks and dinner evening after the event (again, pay for yourselves)
• We will have plenty of discussion about testing.
• If you are a speaker you would have 30-45 minutes to speak and 45 minutes open house facilitated discussions around your topic and other topics that crop up.
• Venue will be communicated only to confirmed participants.
• Last date to send speaker submissions is May 11th, 2014
• Participant entries are first come first served.
• There would be a wait list usually if the max number is reached before you send your request.

All testing done should be...

All testing done should be...

• Risk based : Caring to not  lose users, money, name, market share, brand value, customers, people
• Business driven : Focused on goals that business wants to achieve
• With care for users : really caring for them and not masturbating about caring for them
• To improve sales :  focusing on conversion, funnel, lesser cost per customer acquisition
• To increase profitability : to be more efficient while making the same money
• To make customers happy : by just allowing them to do what they intend to with the software

Does your testing fit the above pointers?

I think tests, heuristics or your lovely test cases should be classified according to the above categories other than your favorite little bunnies of functional, sanity, regression.

If, all your testing does, is to test for functional stability and not care for the above pointers - your testing is of extremely low value and someday nobody would care for you and your test cases. Not that you don't know.

It was intentional to use the word "should be" because the pointers I gave are meta layers to why testers exist or need to exist. 

Fake Agile - Test Consultant

I call myself a Fake Agile - Test Consultant. All consulting I have done in the space of Agile never exposed to me what some people over the internet seem to be talking about.

When I first read the Agile Manifesto I was excited. It sounded a lot like Context Driven Testing to me. I was hoping that tons of problems I faced in my career with poor software engineering practices would go away and people would be more happy developing and testing software. I saw more stressed faces or more irresponsible faces.

In reality, all places I have been exposed to in India is running Fake Agile. Consulting such teams, it would not be true if I call myself experienced in Agile. After working with those teams, I realize some Waterfall model teams I worked with had better maturity in being Agile and agile although they didn't call it that way. Even Natural Agile teams suffer with people wanting them to be Fake Agile.

I believe in collaboration. Even before I read Agile Manifesto. I believed in People over Process and Fake Agile makes me even more believe People over Process.

As a part of my consulting Fake Agile Teams, I get them to a point where they recognize and say "Yeah, this ain't agile". I end up creating a good foundation for the Real Agile Consultant to come and do the job they are good at. 

Pregnancy Analogies - Part 2

We look at test case pass and fail to make shipping decisions

There are pregnancy tests whose results turn out to be false positives and false negatives. In the pregnancy context, it would be very disappointing if the result is false positive or false negative, especially if the expected was an opposite. In India, I have seen and heard women who cry out of pain when the result turns negative when they want it to be positive. Similarly, teenagers (and those who think they have had enough kids) expecting a negative and false positives could drive them crazy causing a psychological stress.

A test case pass or fail can be a false positive or false negative.

Skilled software testers are aware of the existence of false positives and false negatives hence they investigate every bug they find or if software "works" despite the test case pass.

When I do strategy for testing, my strategy involves - doing tests to determine false positives and false negatives. Building heuristics (test ideas) and oracles (or expected results) that cross check results without causing an over head to my test progress is an approach that I use for my testing.

For instance, in Moolya when we were designing a strategy for testing a mobile app for a banking application - we decided to put tests in a specific sequence (that I cannot reveal) to test for false positives. We were able to test for data accuracy while it could look like simple functional tests.

A test passes or fails through what?

pass¹

pɑːs/

verb

1. 1.

   move or cause to move in a specified direction.

   "he passed through towns and villages"

   synonyms:

   go, proceed, move, progress, make one's way, travel, drive, fly; More
2. 2.

   go past or across; leave behind or on one side in proceeding.

   "on the way to the station she passed a cinema"

noun

1. 1.

   an act or instance of moving past or through something.

   "repeated passes with the swipe card"
2. 2.

   a success in an examination, test, or course.

   "an A-level pass in Music"

If your meaning of Pass is Noun(2) then your notion of Pass in Software Testing maybe weak. If you were to think about Pass in Software Testing as Verb(2) then you are giving yourselves an awesome power to your testing.



Frequently asked questions

Q: How many tests passed?
Q: How many tests failed?
Q: Did that test pass?
Q: Did that test fail?

Not so frequently asked questions

Q: What does a test need to pass through to be called as pass or fail?
A: A test needs an expected result or an oracle to pass through to determine pass or fail.

Q: How does Pradeep think when he exploratory tests?
A: He constructs a test (idea - a heuristic) and runs them across several oracles allowing himself to find more problems per test (heuristic) run.

Q: How does this help Pradeep achieve better test coverage?
A: If he were to run the test with just one oracle (or expected result, you dummy) he would achieve limited coverage and limit himself from finding the bugs he can per test run.

Q: Can you give an example?
A: Sure.

Let us take a test that needs to be run to determine if the GUI writes to the database when a Submit button is pressed. The most obvious oracles is - the values provided in the GUI by the user should be written to database. This can be considered a Requirements or Functional Oracle. The time it takes to write can be another oracle. My test may pass through Requirements and Functional but may not pass through Time Oracle. If it fails the Time oracle then I need to investigate if this is about my internet connection speeds or about the code or about the server. That investigation can help me to make recommendations of what speeds of internet are required to make the write faster. Now, if there is a time out implementation to the write command then I could try delaying it and seeing what happens to the data on the GUI considering that the user has spent 20 minutes typing into several forms before hitting the submit button (which we always do. Always. Let me repeat - ALWAYS). If that fails, then we have tested the test against Recovery / Error / Data Retention / User oracles.

All this takes hardly any noticeable time. I pass through a test through so many oracles to make my Pass a Power Pass. Scripted tests are Power-less tests because there are one (or maybe two) oracles at max people have to pass it through to determine Pass/Fail.

My tests passes or fails but not the way you think it passes but the way I see it pass through my powerful construct of oracles.