"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Tuesday, June 01, 2010

Usability of usability & performance of performance

Hiya there. My name is Pusher V9.3.4 and I am a web application software living in thousands of computers worldwide. I am twenty four years old. I was born to help humans transfer data from one secured location to another and to provide them with a report of all transfer they have done in the past. I have traveled around the world and for every economic meltdown, I was moved from one country to another. I don't do anything different from what I am programmed to do. However, in several situations, humans using me make conjectures that I do things I am not supposed to.

For twenty four years, I have been silent, mostly because software applications like me need to listen to Windows XP and Internet Explorer for our survival. Today, I was transferring some data for one of the human and just peeped in to see what is he browsing. I hit upon this blog and read the words "Some birds aren't meant to be caged" and instantly decided that I belong to that category. I feel happy to have broken my silence.

This would also be the first time I am going to be doing something that I am not programmed to do. I started writing this post when Chandok from Turkey hit the "Generate Report" button. I am aware that I am running a risk of being kicked out of CoWAP (Community of Web Applications) for violating the rules. I care a damn, I want to be of value to you.

I don't plan to take a lot of your time. I want to tell you a story that I witnessed and was forcibly a part of. So, here it goes...

As the number of computers grew so did the number of users. To those who were developing and testing me, it meant they had to help me handle thousand times more data than I ever did an year back. Someone came up with a plan to test my limits and I could hear a term, "performance testing" being used very often. I don't have any physical feelings like what humans have. To me, if you do a load or functional or stress test, its all the same. I won't empathize with a user when he is stressed. I am not programmed to do that, mind it.

So, one fine day, the product manager came up with a list of response times that I should possess for a set of actions that humans are likely to perform on me. I was wanting to know how good I am programmed to handle complex things quickly. The tests were planned and executed. 5 seconds was the target for a report generation. At that moment, I could generate a report in a little more than 2 minutes. So, there had to be a lot of tuning done on me. After five months of effort, they got me to perform the report generation in 10 seconds. Considering my earlier performance, they thought they had achieved something phenomenal. After they made necessary changes to the hardware on which I was hosted, they could see a further drop and I was generating reports in 3 seconds. To test if I would scale for future, they added about 10,000 more concurrent users performing complex operations while reports were being generated. I was varying somewhere between 8-12 seconds. That night they had a party. Only I knew why they shouldn't have had the party.

About 7.2.1 versions later or in other words, 11 years later, how long do you think the user takes to generate a report? 12 minutes.

While you might start suspecting that the reason behind 12 minutes is due to a highly expanded user base, I still take 5 seconds to generate a report but the user takes 12 minutes.

"How can that be?", is that what you are asking, too?  The math that you have learned is of actually little help in such situations. Allow me to explain that to you.


Due to changing requirements, the report generation which was a one click action was turned to a multiple entry and then do one click action in V6.3.1. So, a human has to input several things before he can actually do that one click on me, to see his data transfer report. For instance, he needs to input the date range, the kind of report he wants, the format in which he wants the report, the email id to mail the report, a couple of check boxes, huh, half a page of mandatory fields and rest half of optional fields.

Here is the scenario of 12 minutes : Most users enter incorrect values. I don't have a problem. Its not my problem if I crash. I want to be of help but if I am programmed not to be of help, I just do what I have to. When a user enters incorrect values in some fields, he is shown an error only for the first field and then he goes makes a correction, hits the "Generate Report" button. I then show him that another field has an incorrect value. He corrects that and then hits the button again. I then show him that the third field has gone wrong and force him to correct it. Like this I do for one full page of fields and check boxes. When everything filled in there appears to be right, I am forced to show a warning that a huge file will be emailed to the mail id given and ask them if they still want to go ahead with it. Finally, the human gets to hit the "Generate Report" button and I get into the branch of code that takes just 5 seconds to generate the report.

"Is that a problem?", you may ask. Trust me, those who developed and tested me believes it is not because they are not aware that the users are taking so much of time to generate a report. "How come?", again, you may ask. Simple, I have a page whose title is "Report Problems / Feedback", many thousands of users click on it and see a couple of fields and check boxes. I just giggle when they close me after seeing that page. That's what you might want to call as "Checkmate" if you are chess player.

I just wish I could have told those humans developing or testing me that, not paying attention to usability and focusing on performance would make them poor performing software professionals. I also wish I could tell them that the metrics they were collecting and the way they were drawing inferences was the best way to fool themselves and the people around them.

Even if I said that, would anyone bother to listen to me? No way. Am I bothered that they wont listen to me? No way.


Yikes, time to go back as it looks like the Chandok has finally passed through all the fields and check boxes.  Bye!

26 comments:

sumit said...

Again an excellent post. Infact it is a story of majority of organizations. we too experienced the same.

Performance and usability checks are must.

Most of the times it happens - Testers and developers are not caring about these two factors because these are not explictly men mentioned in tspcs.

Parthi said...


have a page whose title is "Report Problems / Feedback", many thousands of users click on it and see a couple of fields and check boxes. I just giggle when they close me after seeing that page. That's what you might want to call as "Checkmate" if you are chess player



I laughed to death on reading this.

Much to the users irony, there are applications which

• Have more than one server validations and on each such validation, the page gets re-loaded, extending the users' wait time

• Was there any mistake/field missed, those reloads will re-set all the fields in the page and the user will have to re-start filling everything all over again

I don’t know if Pusher had the opportunity to see some users scratching their heads in the above situations too!

While I agree with you that *Usability* of an application is very important, isn’t this a very subjective idea. For some users what seems to be a straight forward common sense can be an irritation for others(not talking about “one prompt per validation”). It also depends on the type of validations too, isn’t it? If the validations are of relative in nature between the fields in the same page, then the prompts will have to be for those fields that are left empty and upon supplying data few more required ones could spur.

My point is, usability design may not be as straightforward as it shows up on the screen and it entirely depends on the nature of the application, design and more importantly the users.

Performance of an application can well be objective (with number of connection; users, response time; configurations) of course it’s not linear but certainly quantifiable (with an array of parameters). May be this easiness (in defining) be the reason why Pushers’ developers and testers were talking more about performance!!

Would love to hear you !!

Cheers
Parthi

Selim Mia said...

Another nice post!

Some usability features on an application can effect on performance badly and vice-versa. So, testers or developers need to consider Usability and Performance issues of an application at the same instant.

- Selim

Parimala Hariprasad said...

Great Post Pradeep. Nothing more to say!

Regards,
Parimala Shankaraiah

MaikNog said...

Great read! I *very* much like the spirit and creativity to wrap what could have been another "ususal" explanation about (in this case) performance and why that factor as a standalone factor is not necessarily a good idea, into an engaging story.. easy to read, also for non-techies, and with enough inside details to also please the techies (aka testers, devs, and-what-not).

+10 and Thumbs up !

Cheers,
MaikNog

Prakasan K said...

You are amazing, Pradeep.. What a nice story telling... :)

Priya said...

Nice post written in a very interesting manner :)

Dhanasekar S said...

Hi Pusher V9.3.4,
Also usability will help to reduce lot of customer support issues,thus reducing the maintenance cost.But many miss to focus on those aspects.
Wonderful narration Pushy,Expecting more such posts,you are more interesting than Pradeep ;)

-Dhanasekar S
http://testingideas.wordpress.com

vvk said...

Good read. As most of the organisations are working in the agile mode, everyone has their focus on delivery of the product and hence the usability/performance of product/project takes the back seat. The company and the product faces the burnt over a period of time.
Great and a innovative way of delivering the message pradeep. :)

-Vijay V Kalkundri

Rahul Verma said...

Hi Pradeep,

Nice post!

I have always considered usability as a part of performance engineering, especially the aspects which impact the user perceived performance. That formed the basis of my paper - User Behavior and Performance Perception Analysis. It was based on insights from various industry experts like Scott Barber (User Experience Not Metrics), Alberto, Swing usability guidelines and so on...

Having said that, IMHO, providing for additional features to have customized report is neither a usability nor a performance issue, as long as the same default format report is still provided as an option for a quick report. Handling errors in the fashion you described in the post is definitely a usability issue turning out into user's perception about performance of website.

Regards,
Rahul Verma

H!M@N$HU said...

A very important message told in "The Best" way. Thanks Pradeep.

Mahathi Ramya said...

Nice story telling with a message :)Day by day your blog posts are becoming more interesting

Pradeep Soundararajan said...

@Sumit,

Performance and usability checks are must.

Yes and the world knows that but does ignore it very well.

@Mahathi,

Nice story telling with a message :)Day by day your blog posts are becoming more interesting

As I improve, it shall too.

@Priya, @Prakasan,

Many thanks.

Pradeep Soundararajan said...

@Parthi,

While I agree with you that *Usability* of an application is very important, isn’t this a very subjective idea. For some users what seems to be a straight forward common sense can be an irritation for others(not talking about “one prompt per validation”). It also depends on the type of validations too, isn’t it? If the validations are of relative in nature between the fields in the same page, then the prompts will have to be for those fields that are left empty and upon supplying data few more required ones could spur.

There are some common usability guidelines. For instance, I did a research on usability last year to fish out lots of information. One of the themes of Jakob Neilsen, the usability guru is, "Don't make me think". That's a great theme to have for usability.

I'd suggest you look into Corel's Usability lab videos in Youtube, interact with people who live on usability and then form an opinion. I am not aware if you have already done it but as I dont know, I am suggesting them.

My point is, usability design may not be as straightforward as it shows up on the screen and it entirely depends on the nature of the application, design and more importantly the users.

I wish there was a usability design for the products we test. The point of whether it is straight forward or not comes later.

Pradeep Soundararajan said...

@Parimala,

I loved to get your opinion. You are an upcoming writing from India whose opinions I want to hear. Thanks.

@Selim,

So, testers or developers need to consider Usability and Performance issues of an application at the same instant.

You got it right. I wish to read an experience report from you on how you took that forward to your teams and how it helped gather more information.

Pradeep Soundararajan said...

@Dhanashekar,

Also usability will help to reduce lot of customer support issues,thus reducing the maintenance cost.But many miss to focus on those aspects.

Yo! Yo! Yo! We do the kind of work that generates more work for support people!

@Kazzar (Maik Nog),

I am very glad you liked it. I am really glad. I know you would know why I am glad about it ;-)

Pradeep Soundararajan said...

@Vijay V,

As most of the organisations are working in the agile mode, everyone has their focus on delivery of the product and hence the usability/performance of product/project takes the back seat.

What's the value in delivering a product that would solve one problem to the customer and create many other? I just hope people asked themselves that question. You are right, the experiences I hear from people doing Agile in India is not so good. However, there are people like Naresh Jain in India who are helping people shape things with Agile.

Pradeep Soundararajan said...

@Rahul Verma,

Getting your opinion was important. Thanks.

Having said that, IMHO, providing for additional features to have customized report is neither a usability nor a performance issue, as long as the same default format report is still provided as an option for a quick report.

Precisely the point. I just wish people understood that change requirements shouldn't change the easiness with which people were using the software or have an option to make things quicker.

Glad we spoke over phone about this after your comment.

Flights of Fantasy said...

Nice blog....
well written no flaws......
very true in the industry this is exactly what happens
complicating things which can be easily done just for the sake of "user friendly UI"

Jassi said...

Kudos Pradeep!! an amazing Post ,Awesome :)

Cheers,

Jassi

Sathya said...

A Good interesting post. Well written.

Gokul said...

Hi Pratheep,

Good Narrating style!I've seen this in various applications..like ticket booking application.

Tarik Sheth said...

Great Post Pradeep,

IMHO,

Any product requirement specification should have two types of Benchmark requirement i.e.

Design requirement:design requirements that reflect the expert judgment of usability

performance requirements:To determine if system meets these requirements under test laboratoriesalong with test participants in a controlled environment

I think Usability system performance requirement have two significant advantages,

1. Performance requirement directly addresses the bottomline of about how a system should be built and with what usability (limitations as well)features.

2. Performance requirement is technology controlled and Usability requirement is mainly a functional control (again it varies from Product to Product)

Tony Rhode said...

nice post

Santhosh Tuppad said...

@Pradeep,
Nowadays, applications cry because they are not smart enough. Applications say,

End-user: I can't access my profile from this module- This is irritating.

Application: Calm down - You can do it from here which is workaround. Not my mistake, some testers did not test me for usability and end-users blame me :(

Thanks,
Santhosh Shivanand Tuppad

Anonymous said...

this is what i make a keynote.To be implemented by many testing teams.Because i come across this many times ex-specially business related sites.