Hi Reader,
On November 13th 2006 9:30 PM IST, I noticed Prashant Kumar online in Yahoo! Messenger. Prashant is a passionate tester working in Bangalore and I've known him for the past 6 years, since we both are from the same batch of engineering and we lived nearby.
I engaged Prashant to take up a testing exercise to help me practice conducting an exercise. We were having a voice chat through Yahoo Messenger conference and I asked him to test the window through which we were chatting.
He was smart enough to start with questions like:
Prashant: Do you mean the voice conference window between us?
Pradeep: Yes!
Prashant: Okay, what do you want me to test in this?
Pradeep: "Send File" feature.
Prashant: Okay, let me start off with something like this... ( he listed out some types of files he would send.. *.exe, *.doc, *.zip, *.mp3, *.xls... corrupt files, sizes of file 0kb .. 1kb, 100 kb, 1MB, 1.1MB and started doing it )
Pradeep: ( Meanwhile, I too started exploring the feature. I noticed that I was able to invoke more than one session of "send file" for the same user)
Pradeep: Prashant, try doing this ( I explained to him what I did).
Pradeep: Prashant, don't you think it is a bug if windows are over lapping over each other, which blocks the end user to be unaware of the multiple sessions that has opened on his own multiple clicks of "send file"?
Prashant: Not sure, but wait let me investigate.( He goes offline suddenly)
Pradeep: (Meanwhile, I tried clicking on "send file" option multiple times., My Yahoo Messenger disappeared too) We both logged into Yahoo and got into a conference again.
Prashant: Hey, lets say it together... ( we both knew, the observation was obvious )
Chorus: Yahoo! Messenger crashed. Pradeep: Are you sure, you know the steps to reproduce?
Prashant: Yes, I do know.
Pradeep: Go ahead!
Prashant: ( narrates what he did to observe Yahoo! Messenger crash )
Pradeep: Oh! I thought it is something else, wait let me try the steps you mentioned. ( I followed and went offline)
Pradeep: (I did try the version of "steps to reproduce", I had in mind, the issue was not reproducible. So pair testing worked well)
--
Deep investigation starts... My learning from the exercise I took on investigating a bug as a part of James Bach's coaching, came handy...
1. We both made an attempt to arrive at a conclusion on "steps to reproduce" the issue and verified the same with each other.
2. The bug was easy to reproduce on our configurations. We saw it happen about thirty times.
3. We checked with ourselves, our PC and Laptop configurations to ensure we both did not have any other application/program causing this crash other than Yahoo Messenger. Hopefully no virus/spyware that does not like Yahoo! (I used MSINFO32 to find out my laptop configuration and Prashant used DX , I compared both to see, they were different configurations. It gives evidence that it can be reproduced with more than one configurations or could be all. I applied an update to my AVG Free version anti virus and performed a quick scan for virus/spyware, I am not sure whether I asked Prashant to do this.)
4. We added another user to the chat session and discovered that the problem still occurs with three. We extended the experiment upto 5 users in the conference. The crash was seen each time by Prashant and me.
Voice was enabled in our original session, so we disabled voice and repeated the steps to reproduce the issue. Yahoo! crashed again.
5. Now we both started asking ourselves and each other; is there something that we could have missed? ( I am sure, we could have )
6. It is then an idea struck both of us; let's Yahoo and Google to check if this issue is already under "known limitations"? We could not find any such for the version we were using. Yahoo Messenger: 8.1.0.195 and MyYahoo Module 8.0.0.1. ( Its 1:20 AM IST, both of us remember our office work and the pending work we have tomorrow and set off to bed, Good night!)
7. (Nov 14th 2006, 10 30 PM IST) We craved for more evidence that could help to prove this bug and improve Yahoo Messenger. We then hit upon C:\Program Files\Yahoo!\Messenger\logs\ . Ah, at least I could not understand those logs, so we decided to store that as a supporting evidence since as testers we claim to know, why someone (dev_guys) would have enabled logging for.
8.All done. Not yet... we wanted to have screen shots but the learning from - Tools For Recording Exploratory Testing was applied. We used Wink to record our test and provide a visual of our steps to reproduce and make you all witness Yahoo disappearing :)
9. The report... which I am doing now...
Title: Yahoo Messenger crashes when multiple send file option is initiated for the same user from a conference window.
Steps to reproduce: ( pretty simple )
1. Download Yahoo Messenger latest version in your PC.
2. Log in/Log on with your user name and password.
3. Find a buddy online and invite him to a conference ( with or without voice enabled ).
4. Once your buddy joins the conference, click and highlight your buddy's id on the conference window.
5. Initiate a "send file" from the conference window.
6. Despite a "Select the file to send" window being opened, initiate "Send file" from the conference window for 7 times without selecting a file to send.
Observed result :
Yahoo! Messenger crashes.
Video link: Coming soon... ( Upload failure - My 14 upload attempts failed, am not giving up)
Yahoo Messenger logs collected from PC (C:\Program Files\Yahoo!\Messenger\logs\ ) : Linked here
_ _ _
I am sure, we could have done a lot more but for me, my approach was good enough, just that I need more practice to become "very good" on investiagating a bug.
Regards,
Pradeep Soundararajan
http://www.testertested.blogspot.com
Pradeep's first language is testing and not English - Michael Bolton
14 comments:
Hi very impressing one...i too tried the same..plz let me know wht is your feedback to fix the this bug to yahoo? i mean in what way should be fixed..
After reading your post I got fruit to think, as a tester mostly I don't believe on things easily unless I do myself, so I
decided to do it at my end with the steps given by you.
My observation after doing the same things are :-
I tried both of the issues with the help of steps given by you. I was able to duplicate the same issue.
I tried on four machines (4 laptops with different configurations)
The version on mine machine's YM was 7.5.0.819 and the version of YM on other machines was 7.0.0.249
I noticed it crashed between 3 to 7 tries.
Conclusion : This issue is not introduced in the latest versions 8, I had observed it in previous version(7) also.
Possible fix :
If we open and close the "send file" window, the issue does not persists the issue occurs only if we try to open multiple
windows at a time.
So the possible solution may be that the YM should not provide the multiple windows to open for the same user.
@ Mukesh,
Why do you want to suggest a fix?
Now, this calls for a lot of questions...
1. If only one file can be sent at a time from one user to another, that becomes a limitation, isn't it?
2. Do you know whether the flaw is in the design or code?
3. Do you know Yahoo might have fixed this by now? (probably not)
However, I would not say "Dont allow a user to open multiple windows" instead I would say "Dear Yahoo developers, the investigation report is ready, it must be easy for you to fix the crash"
If we open a send file window to send the files, we can send mutliple files at a time, so I think there is no use to give a separate window to do the same thing.
I am studying Cem Kaner's book "Testing Computer Software" which mentioned a template for Reporting a Defect. From that template I came to know that we can also suggest a diffect for an issue. In this case I was thinking this may be the possible solution. So I suggested it.
@Mukesh,
I am happy that my questions made you reveal "I am following bible".
Learning Software Testing - Cem Kaner, Jack Falck and Hung Ngyuen is Bible - Part 1
Lessons learnt in Software Testing - Cem Kaner, James Bach and Bret Pettichord is Bible - Part 2
However, did you try questioning "why should I follow this template?" , "Why not something else?" , "Why should I suggest a fix"?
That is how you can become a better student of the bible!
Hi Pradeep,
A good investigation of a bug.
A tester may/may not suggest a fix but a tester suggesting a probable solution for the bug, indicates his understanding of the product.
A developer may get a clue for providing a fix if the suggestion is applicable/qualifying.
More over the ultimate goal of a testing + dev team is to deliver quality product. Suggestions are to meet the ultimate goal.
@GB (Girija?)
What if the testers suggestion confuses the developer?
How many times have you seen a developer coming to a tester and asking "any idea how to fix this?"
If a tester does not understand his role, he/she would suggest a fix unless explicitly asked for.
Okay, lets have a new post on this and not get deviated from the content of the above post!
mukesh says "If we open a send file window to send the files, we can send mutliple files at a time, so I think there is no use to give a separate window to do the same thing".
Mukesh, sending multiple files at a time... Hmm gud... Y dont u answer this question ? Is there an option to send multiple files at a time? I dont think so.. Dont we need to wait till then first file is sent? Ive nt checke d this scenario... nd Im sure we would nt be having a option to do either !
Fix:
All yahoo needs to do is, change the "send file " method so that it either can queue the files and send one by one in a single window or design a proper loop(which is not done now) for the "send file" method !
About suggesting fixes:
It can work or it can piss the devs off :).
I would like to have a talk about suggesting fixes or not, but not here; like Pradeep said, let's keep on topic :).
Now, on topic: nice work on documenting the bug :).
Hmm.. nice bug but not much user affecting. I mean how many users will do
5. Initiate a "send file" from the conference window.
6. Despite a "Select the file to send" window being opened, initiate "Send file" from the conference window for 7 times without selecting a file to send.
Unless until he is testing the product?
I feel yahoo might have this bug in there bug list already, yet released the product ignoring this sev4.
@Saiprasad,
Now that this bug is on my blog, many other people who have looked into this were able to reproduce. They need not necesarrily try it out on their own but when someone informs "how to reproduce a specific bug", many users would.
What is a bug?
Anything that threatens the quality of a product, isn't it?
Why would Yahoo! put a crash in Severity 4?
I mean how many users will do
So can a bug exist that not many users would not try the feature out?
I am not trying to prove you wrong but I am happy that you came out with a good point of "Should a company bother about such bugs?"
Yes, it should. Crashes are usually the ones which threatens the reputation of a company.
If you were to be given a MOUSE with a limitation that "Don't click it 1000 times, the MOUSE blows up to pieces killing the person using it"... would you be interested to use that product?
As testers should you not be bothered about the very few users who might see a bug that has not been fixed?
I feel yahoo might have this bug in there bug list already
I guess Yahoo is not such a company to have a crash in their list of bugs and release the product to the market.
Moreover, Yahoo! is used by people world wide and even "few users" might mean - 10000 users.
I would be happy as a tester, if you carry out further investigation on this and provide evidence to challenge our research on this bug. It would be a good exercise for you or any other tester.
I tried reproducing this issue and it crashes all times i tried. Simply superb finding and report. great work Pradeep and Prashant
Hi Pradeep,
In one of your posts, I read like “think out of box” that post saying about camera software crashes and given a good/skilled tester to find the root cause for this. He started thinking from exactly where the customer fails to take the photo. Here my question(s) is what exactly mean by think out box, i heard this word from my peers and manager too.
Is this like Thinking against specification (negative testing)?
Could you please elaborate on this?
About my self I am presenly working as Software Tester
@Anonymous,
You came out with a good question.
What exactly mean by *think out of the box*?
Well, I can't say that but I can answer that if you can ask this question -
What do YOU mean by out of the box?
Simple, anything that is outside the box. Now, that isn't a foolish answer provided I say a BOX to me is the boundary we set in our mind.
Once, I asked my student to list all possible scenarios that could have contributed to of a specific bug example, I gave him and he listed 13 scenarios and said "I'm done".
I made him think and he could list out further 10 and then I asked him "How come you said you are done?". He laughed at himself after hearing that.
I would call the above as an example of out of the box (boundary of your mind).
Post a Comment