"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Wednesday, December 02, 2009

Why testers need to learn to write code that works?

One of the things you thought I don't do often as a tester is - to write code. You are not completely wrong. I immersed myself all these days trying to be a tester who is /mostly/ black-boxish and interacts with the software through GUI. Did you read it as, "A black box tester doesn't need to write code?". Stop reading it that way! Read a sentence the way it is written and not the way you expect someone else to make a mistake that you like them to do.

You shouldn't be surprised to know that in the past I have written little tools, utilities and batch scripts that helped me or a test team I worked with. At the worst case I used to edit those scripts written by others to suit my needs or the mission. No matter how small they were, the value was the key.

Perl has been my favorite (you think I explored others enough, nah) since I first discovered it being used to automate checks in my first job. A couple of weeks back I decided to focus on shaping myself to be a tester who can write code in Perl for automating checks for most kinds of software I test. Irrespective of whether I am hired to do so or not, I'd want to be equipped.

In order to practice stuff that I learn in Perl, I decided to create exercises, puzzles and games for testers. That way I am trying to have more fun learning Perl. Perl itself is fun in its true nature and imagine adding more fun to it.

So wanna check out the puzzle I have for you? Hold on, don't be in a hurry.



You might want to consider reading what I experienced this time when I wrote code, found bugs in it, investigated them, fixed them, rejected a bug or marked it for future release.

Tester writing code

As a tester, writing code is such a beautiful experience. It gives an insight into why the programmer was reasonable (some times) when he wrote the code that way although it might have sounded unreasonable to you while you were testing it. It gives an understanding that his intention might not have been to write such a buggy code but there were things beyond his control such as a computer software that changed the behavior of the code when it was being executed. If you are lucky, you would understand that he is learning to code better as you are learning to test better.

I was using Komodo Edit for writing and editing code and I like it for its simplicity. I saved my code giving them a version number as and when I thought I made significant improvements to it. In one specific version I made a mistake trying to experiment a new idea and the code failed to even say "Hello World!". I had no track of the changes that went in to the current version as compared to the previous. I did a diff but then I realized I needed something more than a diff. So, a Tortoise has entered my system. It will start moving from my next experiment onwards. It took a bumpy ride to get a Tortoise in. Did I not know the value of Version Control? I have complained and whined about it in one of the previous organizations I worked for to maintain my test case documents. (Yeah, Pradeep wrote test cases and executed them a thousand times. Keep that aside.).

Writing code is a humbling experience for a tester. Go, write, code. (yeah, three, commas, in there)

Tester testing his own code

After being a tester who doesn't code much and then writing code and then ( and then: forgive my innovation in English ) testing it was a very interesting experience. I discovered that changing hats from a programmer to a tester isn't easy. When I tried testing the code I wrote, I was looking for basic flow failure of code. When I found a problem or two with the basic flow my thought process shifted, from finding more problems to fixing the ones that I had found. I was worried that something fundamental that was supposed to be working isn't appearing to work. I grew more curious to ensure basic work flows are fine. So bad. Being a tester I know what is basic to one might not be basic to others. I was contended that my code worked when those basic flows worked well. It was high time I needed a tester to test my code.

The target audience for my code was testers (yeah, you). I passed a version of it to Michael Bolton who was online (was busy) and he did a Q approach ( QPRADEEP - Quick - Practical - Deep ) on it. He suggested me things that would make the program more valuable that led me to write more code to it.

When I was testing my own code, I wasn't bothered about test coverage at all. When I am consulting or testing, "coverage" is always on the top of my head. I am known to question about coverage to almost every tester I talk to. Look at how beautiful my brain worked when I was testing my own code.

If you read the above as "Is he saying that this happens with every programmer testing his code?", I once again suggest you to read it the way it is written and not the way you expect someone else to make that mistake that you would like them to do. Don't be offended if you did read that way, most of us have a disability in reading and understanding. Its fixable.

Tester fixing bugs while investigating them


This is another important experience a tester should have. It helps in understanding that programmers might have made real attempts to fix a bug and yet it escaped in a different way. It also helps in understanding that standard test cases won't help in capturing the escaped-in-a-different-way bug and they need exploratory testing for investigating if-the-bug-really-got-fixed. If testers report symptoms of a problem and those symptoms are fixed, the problem still remains. So, it helps a tester to be careful in asking if he is reporting a symptom or a problem.

Let me give you a simple example from my code:

 if (($x == "\n") || ($x == "\t") || ($x eq ""))
                {
                  print "That's not valid according to me! Please try a number";
                  &ask_for_a_number_or_reject_input ;  # sub routine call to seek a number from the user
                }

What do you think I wanted to do by this?

If a user is providing a blank input or a new line or a tab, ( I am OK with some text but not blank input ) I wanted the program to ask the user to enter what I think is a valid value and keep doing that till the user provides one or exits the program.

You might see that I used "==" in a context where I needed Perl to do variable interpolation and "eq" in a context where I did not want it to do so. Here comes the interesting thing, somehow the program was not getting into this if loop at all. When I got to the prompt I hit an enter without entering an input with a hope that I would see the print statement "That's not valid according to me! Please try a number" being executed and then the subroutine call. I just didn't see it and was terribly disappointed. Instead of rejecting that input it took the value "0".

So, what's the problem and how would you report it?

A part of my investigation involved suspecting usage of chomp which eliminates the "enter key input" from the user input. So, experimented with chop and found other kind of problems that happens with chop for the program I was writing. Should I store the input in a temporary variable and regex it for any new line, tab or blank input or continue doing it the way I was?

Learning to code (and coding) is an exploratory process. While the book might say use "==" when you want variable interpolation and "eq" when you want to match text and no to avoid interpolation, it takes a couple of experiments and situations like these to really get the wiring in your brain to learn to use it wisely and identify if they are causing any problems.

What is not visible to you here is that the problem was occurring because of a mistake I did much above this piece of code :)

So, a problem that appears to a tester in one way might appear to a programmer in a different way.

Curious to see if this problem got fixed? Hold on, don't hurry up.

Tester maintaining the code for reuse

I didn't write couple of hundreds of lines of code but just a dozen tens. Chee, if someone is really good in Perl they'd have written it with much smaller footprint. Well, I will get there, don't hurry up :)

Although there were just a dozen tens of lines of code, I chose to comment on some of the important parts. Have you been in a situation of being unable to read your own handwriting? I have, just shy of a hundred times, maybe. I know I could forget why I wrote the code the way I did after an year of learning to write better code. I cant go back and be as stupid as I was to know why I did that. So, it is better to speak_aloud_your_stupidity so that the better you understands why you did so. The better you might have better ideas when you know the reason of why you wrote the code that way and what you intended to do with it. Forget yourself, think about others. I once was given a job to edit some scripts. I spent an hour to just edit a few lines and then proposed a cost saving technique to my manager - to re write the code. My dear colleague had obfuscated the code without knowing that he was doing it. Code that works today does not necessarily mean maintainable code for the future. Now, I think he is obfuscating elsewhere.

I am increasing my sense of appreciation towards good programmers as and when I see what my code does does versus what I thought it would do. That's a long mile of learning to me. It will help me work with developers much better in future.


Isn't it high time I allow you to play with the program?
  • The program you are about to download is a puzzle.
  • Your mission is to reverse engineer the logic of the program.
  • When you find "Nemo" dont be excited that is just a small part of the whole puzzle.
  • Finding Nemo is very easy but reverse engineering the logic is where the challenge for you is.
  • Please send your reverse engineered logic and or experience report to pradeep [/dot] srajan [/at] Jee mail [/dot] com. Got it?
  • Its not necessary that you crack the puzzle but it is necessary how your journey to crack the puzzle is.
Download this guy (which is an exe file). I know your organization might have blocked all exe downloads so I have a xex version ( which you have to change to exe after download ). You got a Mac? Ah! I got a exe :(

So, go find Nemo, send me your experience report / reverse engineered logic or just silently pass it on. Blub Blub!

7 comments:

Swapnil said...

Pradeep,

Thanks for Finding Nimo game... thats fantastic, I found Nimo in these move
12,9,5,7,2
Loic is good :)

When I press Cntl Z it goes to infinite loop saying wrong input data.

~Swapnil.

Pradeep Soundararajan said...

@Swapnil,

Firstly, I am glad you helped me learn about the Ctrl Z causing the program to go to infinite loop. It shall be great fun trying to fix it.

Next: You still haven't cracked the puzzle. You are a long way to get there if all you found is what you published above :)

sarmila said...

I think disassembling fnemo._1.7.6.exe would get me started in crackng the puzzle. So I gave a bit of read about disassembling exe file, legal issues and found a tool to do the job.

Itz a quiet useful tool,but I'm facing difficulty as I didn't understand intel assembly.

Before I put more time into it, I would like to know your thoughts on my approach.

thanks
Sarmila

Pradeep Soundararajan said...

@Sarmila,

Your approach is interesting. However the actual puzzle is in playing the game and reverse engineering the logic of the program.

However, here is a heuristic that might be of good help to you: In such instances it is absolutely fine to go wrong because you end up learning something vital that might not solve today's problem but of tomorrow.

Those who are getting it right today is the result of going wrong long back.

sellib said...

As a developer moving into the testing world, I feel that having development experience as a tester is valuable. And not so much that I am able to do 'white box' testing vs. black box, but in being able to communicate very clearly with the customer.

I feel that I am able to contribute to clearer bug reporting and faster turnaround time in helping the customer resolve bugs when I can talk in the language of the developer. Using the iPhone for example, I am able to use technical terms like 'pushing a view onto the nav controller' or 'in the tableview' or 'a modal view is displayed'.

As you allude to, being a developer does impact my testing/coverage thinking. I do think in a bit more of 'white box' testing than black box (let's call it grey box testing) because I have experience certain difficulties with application development. For example (again using the iPhone platform), there can sometimes be issues with populating a tableview, or that I know that the iPhone has limited memory so I look for ways to force the app into a low memory situation.

To conclude, I feel it is beneficial for a tester to have a basic knowledge of coding for the applications/platforms they work with regularly.

- Brad Sellick
Twitter: @sellib

Pradeep Soundararajan said...

@Brad,

Thanks for commenting.

I feel that I am able to contribute to clearer bug reporting and faster turnaround time in helping the customer resolve bugs when I can talk in the language of the developer. Using the iPhone for example, I am able to use technical terms like 'pushing a view onto the nav controller' or 'in the tableview' or 'a modal view is displayed'.

Nice you brought in the angle of the language of developer. I have been talking about it to a couple of testers here. Some testers who do not speak that language sometimes get intimidated by it and avoid trying to learn it.

You may have an advantage there + you may also be in a position to understand why the developer thought the way he did when he rejected the bug you reported. I am sure you are working on your bug advocacy skills.

As you allude to, being a developer does impact my testing/coverage thinking. I do think in a bit more of 'white box' testing than black box (let's call it grey box testing) because I have experience certain difficulties with application development.

Anything that others have learned is learn-able :) So you train your mind to have a balance of blackboxish and whiteboxish tester. I am sure that is going to happen over the coming years.


To conclude, I feel it is beneficial for a tester to have a basic knowledge of coding for the applications/platforms they work with regularly.


Right. The good thing is every tester knows it but the bad thing is not everyone works towards it.

We as a community must help those who are not putting a step forward learn it. For instance, if you like teaching, you may over the weekend or over online could teach programming for testers who do not yet currently program.

From my side, I should continue to learn to write better programs :)

Santosh Shukla said...

Hi Pradeep,

I spent some time over this puzzle last night.

After some exploration, Found Nemo in all 10(default) attempts:

Nemo is found at these locations:
1,3,5,3,2,7,3,4,7,4

Could not find much time to find the logic, but some observations for now:

This puzzle actually seems like a program written with some requirements like:
1) It is a circular array (may be)
2) Gill and Bubbles are always together.
etc.


Note: Found that puzzle has fixed answers for fixed number of attempts. The Nemo and group are always in the same places if puzzle is tried for same number of attempts.

Trying to reverse engineer under time constraints... But that is life :)

Santosh Shukla, http://proudtester.blogspot.com