A test passes or fails through what?

pass¹

pɑːs/

verb

1. 1.

   move or cause to move in a specified direction.

   "he passed through towns and villages"

   synonyms:

   go, proceed, move, progress, make one's way, travel, drive, fly; More
2. 2.

   go past or across; leave behind or on one side in proceeding.

   "on the way to the station she passed a cinema"

noun

1. 1.

   an act or instance of moving past or through something.

   "repeated passes with the swipe card"
2. 2.

   a success in an examination, test, or course.

   "an A-level pass in Music"

If your meaning of Pass is Noun(2) then your notion of Pass in Software Testing maybe weak. If you were to think about Pass in Software Testing as Verb(2) then you are giving yourselves an awesome power to your testing.



Frequently asked questions

Q: How many tests passed?
Q: How many tests failed?
Q: Did that test pass?
Q: Did that test fail?

Not so frequently asked questions

Q: What does a test need to pass through to be called as pass or fail?
A: A test needs an expected result or an oracle to pass through to determine pass or fail.

Q: How does Pradeep think when he exploratory tests?
A: He constructs a test (idea - a heuristic) and runs them across several oracles allowing himself to find more problems per test (heuristic) run.

Q: How does this help Pradeep achieve better test coverage?
A: If he were to run the test with just one oracle (or expected result, you dummy) he would achieve limited coverage and limit himself from finding the bugs he can per test run.

Q: Can you give an example?
A: Sure.

Let us take a test that needs to be run to determine if the GUI writes to the database when a Submit button is pressed. The most obvious oracles is - the values provided in the GUI by the user should be written to database. This can be considered a Requirements or Functional Oracle. The time it takes to write can be another oracle. My test may pass through Requirements and Functional but may not pass through Time Oracle. If it fails the Time oracle then I need to investigate if this is about my internet connection speeds or about the code or about the server. That investigation can help me to make recommendations of what speeds of internet are required to make the write faster. Now, if there is a time out implementation to the write command then I could try delaying it and seeing what happens to the data on the GUI considering that the user has spent 20 minutes typing into several forms before hitting the submit button (which we always do. Always. Let me repeat - ALWAYS). If that fails, then we have tested the test against Recovery / Error / Data Retention / User oracles.

All this takes hardly any noticeable time. I pass through a test through so many oracles to make my Pass a Power Pass. Scripted tests are Power-less tests because there are one (or maybe two) oracles at max people have to pass it through to determine Pass/Fail.

My tests passes or fails but not the way you think it passes but the way I see it pass through my powerful construct of oracles.

Pregnancy Analogies - Part 1

Everybody is pregnant with ideas all the time. Women are gifted with a capability to be pregnant beyond ideas.

In 2010, I had the divine opportunity to learn about pregnancy through my wife who was carrying our first child. I couldn't stop my mind from co-relating pregnancy to testing software. I have been trying to crystallize my ideas for a long time (our child is more than two and half years old now) and this blog post was lying in draft version since early 2013. A long journey to get here.

When I attended Anne and Rob Saborin's talk in CAST 2008 - Testing Lessons from Labor Room, I had no way to co-relate what they were talking because I did not know about pregnancy or labor pain as much as I know after I experienced it through my wife. I wish, I could listen to that talk again.

Here is my version of ideas about pregnancy analogies for software testing. I want to structure it as questions (or situations) and answers (or responses) to it. Software testing is very simple to understand and this is why the world doesn't get it fucking right.

We are not ready for testing!

Sure. Not ready for what kind of testing. Not ready for testing the heart rate of the baby or its movements? Oh, you mean, you are in initial stages of pregnancy?

What about tests for healthiness of pregnancy? You maybe pregnant however sometimes there could be pregnancies that could lead to health risks. Ectopic Pregnancy is one such example. If you do not test for this (and that should be done much before your baby starts to take shape) , you may end up risking the mother and the child. You want to know about it earlier than later.

All tests are green. Can we push this code to production?

A doctor does not just look at a specific scan result to make a decision for C-section.She analyzes the patient history, current condition, determines what is safe for the child and the mother to make a decision for C-section.

It is a human who assesses all angles to make a decision and not just test results. Test results support the human to have more information when making decisions. Tests that bring in many kinds of information is important to good decision making. Having just one type of test keeps a different risk unaware.

When that successful company doesn't hire testers, why should I?

Delivering a baby is a life and death experience. There are many people who deliver babies at home or in situations where they are far away from hospital. The human race has not vanished because there were no hospitals or advanced technologies during the early days of civilization. Hospital and having doctors around has increased the survival rates, safety and health. Some babies have jaundice after birth and some babies develop fever post birth. Some babies like me don't cry the moment they are born. Having specialist doctors by side makes you focus on your baby than worry about what next.

Want to contribute your pregnancy analogy for testing/development/software? Write to me to my email id pradeep.srajan at the famous mountain view based gmail provider dot com and I shall mention your name and give the due credit if I like your idea in the upcoming parts. 

2013 - the year of change

Happy New Year! For a very long time, I couldn't blog here (or anywhere). This has been the longest gap I have had in my blogging life. I thought I would not get back to this blog but thanks to Jokin Aspiazu, Mauri Edo, Issi Hassan and my fellow Hindustanis, Rahul Gupta and Sharath Byregowda to re-inspire me to blog. 2013 was THE YEAR OF MY LIFE so far. It made me change a lot of things. Personally and professionally. I have had a great start to 2014. If I am a changed man, it shall reflect on the way I think, blog and execute ideas. I have done a lot of good things that is keeping me busy in ways that is exciting to me. I am all set to really do what I have been saying I will do. Not the same way I did things in the past. 

I have disabled comments on this blog. I don't have time for it anymore. I just want to write my thoughts and leave it. I have benefited a lot from responding to comments but I am selectively choosing how I spend my time. I would love you to talk to me though. I am going to be at some conferences and you are free to drop me a line.

I am hoping to make my posts to be shorter than how it were in the past. Unless the context demands me to go deeper into something, I will try to keep it short. Thank you all for coming back and reading this. I shall continue to appreciate the time you spend here and I shall as always work hard and sincere to make it a good experience.

I would like you to follow me on Twitter, Quora and Linkedin, if you haven't yet done that. My passion to write, remains. My passion to help, remains. That won't change.