If a batsman has to score four or six runs in a cricket sport, he needs to drive or hook the ball to the boundary. He can do that since he sees the boundary. As a tester have you ever seen a boundary?
I have never seen a boundary!
What I mean by that is, I have never seen a boundary as a batsman in cricket sport does.
A batsman is able to score four or a six because he knows where the boundary is. If the boundary kept changing, will the batsman ever be confident of scoring a four or six?
Here is a chat excerpt between me and a tester who is preparing himself for ISTQB!
CT: Pradeep, I have one doubt in testing.
CT: BVA for 4 to 10
CT: the values 3 4 5 9 10 11
CT: is it correct?
me: Man if it is this easy to do a boundary testing, anyone can do testing.
Also, have a look at a questions about boundary testing from a sample ISTQB exam paper:
7. Boundary value testing
a. Is the same as equivalence partitioning tests
b. Test boundary conditions on, below and above the edges of input and
output equivalence classes.
c. Tests combinations of input circumstances.
d. Is used in white box testing.
1. An input field takes the year of birth between 1900 and 2004
The boundary values for testing this field are
a. 0,1900,2004,2005
b. 1900, 2004
c. 1899,1900,2004,2005
d. 1899, 1900, 1901,2003,2004,2005
In my opinion, a tester should be taught that there is no fixed boundary in software!
Here is my story of boundary testing of a multimedia product:
I was testing an audio decoder which was supposed to play any mp3 file ranging from 24 kbps to 196 kbps.
So is boundary testing of such a product something like playing 23 kbps, 24 kbps, 25 kbps, 195 kbps, 196 kbps, 197 kbps?
Huh! The version of software I got to test crashed when I played 96 kbps file and I also found that 128 kbps crashed the application, too.
Yes, I agree that it was supposed to play any value from 24 kbps to 196 kbps ( as per the specification). So does that mean, I did not do a boundary testing on the product because I did not use the value 195, 196 and 197?
Jeez! In my opinion, I did a a boundary testing. I did *explore* the program for it's boundary and found that the version of software I got had a boundary of 84 kbps and any value above which the application crashes ( smoothly) .
Despite 96 kbps and 128 kbps file crashed the application, had 196 kbps file played, does this mean there are more boundaries than you imagined?
Isn't it something like saying Pluto is the last planet in our Solar System and NASA coming out with photographs of a planet beyond Pluto that is a part of our solar system. (Wow! General Systems Thinking, is helping me a great deal)
The next version had a fix for the crash and it appeared to me that a 96 kbps file did not crash the application.
Now that's a story of how I conjecture that boundary is not a static in software. James Bach spoke about Boundary Testing in Becoming a Software Testing Expert Google Video and it gave me more new dimensions about boundary testing and a lot of other things, too. If you haven't watched it, you don't know the secrets of becoming a software testing expert :)
As a tester, I am never *sure* about something that I have observed or claim to know about testing but I am very sure that after reading this post, many testers would spark off with their experience and jump out to say, "I have seen a boundary" or " I did boundary testing the way you said is an incorrect way of doing it".
If I were you, here is what I might want to say, "Pradeep, I have done boundary testing in a way that you pointed out as incorrect. I might have failed to notice the thing that you have mentioned, probably because of a tunnel vision imparted to me. Let me do a few more experiments with it and get back to you".
That sounds great!
Have you ever wondered whether you can see James Bach and Mike Kelly exploring boundary of a software and testing it?
No more wonders, it is a reality! Download this video
The boundary does not exist in the software but it does, with your thoughts!
Pradeep's first language is not English--his first language appears to be testing. -- Michael Bolton
Update: BJ Rollison has made a post on this and left a comment in my blog too. I have replied to BJ Rollison in the comments section here.
11 comments:
Of course, I can't test boundaries on true / false parameters either - but this certainly doesn't mean that I don't (or shouldn't) do boundary testing.
A good tester knows what type of testing to apply to a specific situation and certainly doesn't try to apply every technique they know to every situation. That's the same as trying to build an entire house with just a hammer.
Given the commonality of "off by one" errors in famous software defects, applying boundary value analysis to inputs and ouputs where it makes sense to do so is the sensible and intelligent thing to do.
Your assumption that boundary values are not static is obviously based on the fact that you don't know what boundary values are.
Boundary values don't always exist at the extreme ranges of data types or other variables. Occasionally, there are boundary values/conditions within the minimum and maximum physical ranges of a variable. (That's specifically why ISTBQ suggests boundary testing as boundary conditions on, immediately below and immediately above the edges of input and output equivalence class subsets. Of course, that assumes you know how to decompose data into equivalence class subsets.
Also, your analysis of boundary values of MP3 file bit rates is simply incorrect. The bit rates for an MP3 using mpeg2.0 encoding are 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160 kb/s, and the bit rates using mpeg-1 layer 3 are 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256 and 320 kb/s.
Therefore, since no encoding exists for 23, 25, 195, and 197 kb/s encodings these are simply impossible. Boundary analysis implies testing on, below and above the boundary condition using actual values, not something arbitrary or made up. If the requirements you were working with indiated it supported mp3 files using a minimum bit rate of 24 kb/s the min - 1 value is 16 kb/s, not 23.
Your analogy of planets is rediculous because we are discovering new things about outer space and much of it is speculation. Conversely, I don't know too many people who are reinventing or arbitrarily changing the range of a 32-bit int data type for example.
This is why you should read books on software testing and not on epistemology and other psycho-babble.
@Anonymous,
Many thanks for your comment.
A good tester knows what type of testing to apply to a specific situation and certainly doesn't try to apply every technique they know to every situation.
Absolutely, a *good tester* knows the context in which he needs to use a type of testing.
Given the commonality of "off by one" errors in famous software defects, applying boundary value analysis to inputs and ouputs where it makes sense to do so is the sensible and intelligent thing to do.
Absolutely, I see these words from a mature enough hands. Despite such mature people like you, testers are not taught about the context in which they need to apply . I would rather say, these low cost tests yield great software errors but that doesn't mean, a tester should be trained about boundary testing without stressing the importance of context and cost v/s value associated with it.
Pluto is the last planet in our Solar System.
From http://en.wikipedia.org/wiki/Pluto
On August 24, 2006 the IAU defined the term "planet" for the first time. This definition excluded Pluto.
Hey Praddep,
Thanks for the nice post on BVA.
It's bad to see that even now many testers thinking about BVA at unit level.
The examples explained in the post are the tests need to be done by the developer at very basic level.
Think of this algo
Read Int a,b,c
assign c = a+b
print c
In the above example, people need to see that weather a+b fits into the integer range in case if they are close to the boundary values.
These are the simple boundary checks that need to be done a coding level & the focus of the tester wrt to the boundary need to bring in more important issues wrt boundaries of the application / features / it's benefits which are more valuable to the product / customer.
In the context of Cricket as you described, For me the batmen is a developer who see the boundary. Like our developers it's simple for them at a unit level.
James used to say that Developer most of the time not aware of the boundaries. As per my knowledge what he means is like boudaries of the app / feature but not at very simple units.
@Zeljko,
Thank you!
http://en.wikipedia.org/wiki/List_of_planetary_bodies says Eris is also another dwarf planet. Techniques and technology to find celestial objects have improved and hence Eris might have been discovered. Similarly as thought process and techniques improve we discover new boundaries.
@ BJ Rollison,
Your assumption that boundary values are not static is obviously based on the fact that you don't know what boundary values are.
Yes, I don't know what they are and it makes me curious to explore the boundary. I have an opportunity to learn more about the boundary than someone who fixes the value and test for it. I respect you as a tester and hence I offered this explanation.
Also, your analysis of boundary values of MP3 file bit rates is simply incorrect. The bit rates for an MP3 using mpeg2.0 encoding are 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160 kb/s, and the bit rates using mpeg-1 layer 3 are 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256 and 320 kb/s.
Pooh! I am not sure why you don't know that bit rates can be 33, 41, 57 or any number you want to generate. I recommend you to go through some multimedia test content generation tools like ffmpeg which gives a multimedia tester an edge to generate test content of his choice.
A Test Architect of Microsoft speaking like this might not sound good to the community or to my blog readers (who are from Microsoft, too).
Therefore, since no encoding exists for 23, 25, 195, and 197 kb/s encodings these are simply impossible
For those who don't know tools like ffmpeg, it is impossible. I suggest you explore the boundary of your education on multimedia. I saw Microsoft ad which says, “Impossible is nothing”. Did you look at that ad or it’s a marketing thing by your company?
Your analogy of planets is rediculous because we are discovering new things about outer space and much of it is speculation. Conversely, I don't know too many people who are reinventing or arbitrarily changing the range of a 32-bit int data type for example.
As you didn't like my idea of boundary not being static, you obviously would find my analogy ridiculous. I am happy that you are consistent.
This is why you should read books on software testing and not on epistemology and other psycho-babble.
I honor that you look at it in a different way that makes you a credible tester in the community you are and I respect you for your views but your concept of a tester seems to be a person who reads software testing and nothing else. I see that many many testers get test ideas and new thoughts on reading books like General Systems Thinking and Epistemology. I perceive testing as a mental martial art and that is helping me test better.
Testers from Microsoft have got in touch with me appreciating the posts and thought process I have put in here and they are trying to adopt a few things from here. If you think they are going in the wrong lane, please announce to Microsoft Testers to not go through my blog as a part of your training services to them.
I don't want to attack anyone's work and I feel, in the time I attack someone's work, I can learn a few good point that exists with that person. Are there any good points I can learn from you?
I didn't attack ISTQB certification; I was bothered about my fellow Indians who look at a question having only one answer and no existence of possible other contexts.
Definitions and terminologies about testing have been redefined and restated by many experts after their research on it and there is no standard!
I am happy that I have tried to been as kind and polite as possible under the given context in replying to your comment even despite my testing guru, James Bach does not respect you as a tester.
I would like to continue respecting you hoping that you are open to all views from different parts of the world. Help me doing that!
Thanks!
[pradeep] a tester should be taught that there is no fixed boundary in software!
How about a rephrase of above on the lines of --
"Real and interesting boundaries for a testing problem are too tough to be expressed in simple 4-6 values. You will not find them easily in specifictions or by asking the developer.Most of the the time real challenge in boundary testing would be to identify the variables of the problem - leave alone their boundaries. In this context, traditional boundary value analysis looks like very basic and rudimentary technique."
Shrini
@ Shrini,
Excellent re-phrase!
I would like to add something here:
I was bothered about my fellow Indians who look at a question having only one answer and no existence of possible other contexts.
The boundary does not exist in the software but it does, with your thoughts!
This certainly is in other countries also true. And I think this separates some testers from many others.
As von Oech tells:
Where do you learn how to set your mental channel? One important source is your formal education. There you learn what is appropriate and what is not. You learn many of the questions you use to probe your surroundings. You learn where to search for information, which ideas to pay attention to, and how to think about these ideas. Your educational training gives you many of the concepts you use to order and understand the world.
…
Much of our educational system is geared toward teaching people to find “the right answer.” By the time the average person finishes college, he or she will have taken over 2,600 tests, quizzes, and exams … The “right answer” approach becomes deeply ingrained in our thinking. This may be fine for some mathematical problems where there is in fact only one right answer. The difficulty is that most of life isn’t this way. Life is ambiguous; there are many right answers – all depending on what you are looking for. But if you think there is only one right answer, then you’ll stop looking as soon as you find one.”
E.g. many written exams focus on one “right answer” only - but the exam is not real life. I would suggest anyone, to go through different forums, to see themselves, what Pradeep is talking about. People perhaps seem to be afraid to think and/or too lazy (do not kill me now for this sentence). This mind issue seems to be persistent in India, some examples:
How can it be, that people ask the same questions over and over, which were answered already before, instead of using the search button? Or they ask a one sentence question without giving any other context info? (And really expect somebody helps them.)
Testing is asking, true – but how about asking the right questions? Similar with the boundaries – should I use this and that technique right now in my given context? Is this the only boundary? How can I find an important one?
We should welcome every input, which let’s us rethink, what we seem to know.
I would not say, that Pradeep does attack people with ISTQB certificates (I have passed it – please notice, I say: I passed) – see it more as: Pradeep critiques people who do not want to increase their skills. See it as a friendly challenge to improve – to get out of your limits. The interactions with Pradeep made me rethink some things - and I am still digesting and learning. I must say, in my personal development, the interactions with Pradeep help. He is someone who really takes the time and helps you. Think about it, how many people really would help others like Pradeep does? When was the last time, YOU helped someone?
Dhanyavad,
Erkan Yilmaz
Hi
I am discussing this on sqaforums.
Would you like to contribute:
http://www.sqaforums.com/showflat.php?Cat=0&Number=533656&an=0&page=0#Post533656
Regards
Gaurav Pandey
PS - I am not affiliated with sqaforums. I am just another member who is learning and sharing about software testing
Post a Comment