"Some birds aren't meant to be caged, their feathers are just too bright"- Morgan Freeman, Shawshank Redemption. This blog is from one such bird who couldn't be caged by organizations who mandate scripted software testing. Pradeep Soundararajan welcomes you to this blog and wishes you a good time here and even otherwise.

Saturday, April 18, 2009

Checkmate heuristic :: A security testing attack

It has been five times over the last six months that someone considered hiring me before making a release decision or after getting skeptical about scripted tests.

Amidst recessionary times, someone considered outsourcing some testing work to me from United States. An advanced version of the product was slated for a release in the next couple of days. The United States company had outsourced development and testing work to a Structured Fancy Name Process Following Disciples company who had ran through thousands of tests over it and had achieved >98% test case pass, a week before today.

Someone in the United States company thought they'd like some Exploratory Testing and I got the opportunity to lay my hands on it to perform a Rapid Testing on it. The charter for me was to report any security related threats and usability problems.

I found about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems:
  • I could reset the password of any account by tweaking the variables that the client was using to interact with the server.
  • I could stop auto e-mailers reaching any registered mail account in a similar manner as above.
I then tried to reset the password of the dummy account I was using. No e-mail reached me. I then thought, "How about reset of admin account?" and then did the same.

The password was reset and no e-mail reached the admin, as auto e-mailers were stopped. So, I asked the admin of the UnitedStates company to login from his credentials and the response was a pleasent, "What did you do and How did you do that?"

Subsequently, all other users were blocked. Only the admin could release the lock but the admin could not log in to the system. 2 hours of outage till someone got into the database to recover the admin account.

You write a lengthy email, hit the submit button and the application prompts for your Username and Password. You enter them and it says, "Incorrect Username or Password". You attempt to reset your own password but the email does not reach you.

Checkmate!

11 comments:

Anonymous said...

Great work Pradeep! what was going on in your head when you found "about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems" that ~50 people that tested before you missed out. What was your thought process or the approach that led you to these bugs?

Pradeep Soundararajan said...

what was going on in your head when you found "about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems" that ~50 people that tested before you missed out. What was your thought process or the approach that led you to these bugs?A number of heuristics and oracles constantly run in my mind with a focus on the coverage I am able to achieve. I probably think that was one of the things going on.

Another thing is, I having been a scripted tester and having burned my hands on it, I make guesses of what kind of test cases will testers avoid writing and those form a set of heuristics for me.

So, those are some of the things that I can remember of.

Anna said...

Wow. That's an interesting idea - tests that testers avoid scripting. As a scripted tester, that's a blind spot that I hadn't really considered before.

I wrote a post over in the Software Testing Club about it.

Ashwin Palaparthi said...

A good use-case of testers tested by someone.

Investigation followed by "abuse" cases is the real way to find (and exploit) the vulnerabilities, and there is only a very tiny percentage of scripted test suites that cover such abuse cases.

Ashwin Palaparthi,
apalaparthi.blogspot.com

Pradeep Soundararajan said...

@Anna,

Wow. That's an interesting idea - tests that testers avoid scripting. As a scripted tester, that's a blind spot that I hadn't really considered before.Also, there are way too many blind spots for a scripted tester.

For instance, if you are using Gmail, and I ask you, "What is there on the right bottom of the screen when you logon to Gmail?", What's your answer?

I see scripted testers are finding stars through a telescope and that's a bad idea of finding many stars.

An investigation might need a telescope.

Philk said...

Would love to read a follow-up post where you go into more detail on your heuristics on tests that scripted testers avoid.
Very interesting post

Anonymous said...

I could reset the password of any account by tweaking the variables that the client was using to interact with the server.

- Can you elaborate this in more detail - coz I have a serious doubt on the developers skill - and the architecture of the application if what you said is correct.

Pradeep Soundararajan said...

@Philk,

Would love to read a follow-up post where you go into more detail on your heuristics on tests that scripted testers avoid.
Very interesting post
Yeah, the comment from Anna made me more conscious of what was running on my mind.

I am hopeful to follow up.

Pradeep Soundararajan said...

@Anonymous,

Can you elaborate this in more detail - coz I have a serious doubt on the developers skill - and the architecture of the application if what you said is correct.Some systems have a feature provided to turn off all auto e-mail notifications. Its similar to a situation where you register on a new website and there is an option for you to switch off all update e-mails to you.

Nothing about developers skill though, I think. However being able to allow me to switch off is probably a different kind of problem.

santosh shinde said...

Hi
Nice blog with good content. Can you write about your real time experience regarding automated testing.So that it will be beneficial to candidates who know about automated testing but never tried in real time.

Pradeep Soundararajan said...

@Santosh,

Nice blog with good content. Can you write about your real time experience regarding automated testing.So that it will be beneficial to candidates who know about automated testing but never tried in real time.Oh, I have and will.