Amidst recessionary times, someone considered outsourcing some testing work to me from United States. An advanced version of the product was slated for a release in the next couple of days. The United States company had outsourced development and testing work to a Structured Fancy Name Process Following Disciples company who had ran through thousands of tests over it and had achieved >98% test case pass, a week before today.
Someone in the United States company thought they'd like some Exploratory Testing and I got the opportunity to lay my hands on it to perform a Rapid Testing on it. The charter for me was to report any security related threats and usability problems.
I found about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems:
- I could reset the password of any account by tweaking the variables that the client was using to interact with the server.
- I could stop auto e-mailers reaching any registered mail account in a similar manner as above.
The password was reset and no e-mail reached the admin, as auto e-mailers were stopped. So, I asked the admin of the UnitedStates company to login from his credentials and the response was a pleasent, "What did you do and How did you do that?"
Subsequently, all other users were blocked. Only the admin could release the lock but the admin could not log in to the system. 2 hours of outage till someone got into the database to recover the admin account.
You write a lengthy email, hit the submit button and the application prompts for your Username and Password. You enter them and it says, "Incorrect Username or Password". You attempt to reset your own password but the email does not reach you.
Checkmate!
11 comments:
Great work Pradeep! what was going on in your head when you found "about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems" that ~50 people that tested before you missed out. What was your thought process or the approach that led you to these bugs?
what was going on in your head when you found "about 14 potential problems in about 5 hours. On the 6th hour I found 2 more security problems" that ~50 people that tested before you missed out. What was your thought process or the approach that led you to these bugs?A number of heuristics and oracles constantly run in my mind with a focus on the coverage I am able to achieve. I probably think that was one of the things going on.
Another thing is, I having been a scripted tester and having burned my hands on it, I make guesses of what kind of test cases will testers avoid writing and those form a set of heuristics for me.
So, those are some of the things that I can remember of.
Wow. That's an interesting idea - tests that testers avoid scripting. As a scripted tester, that's a blind spot that I hadn't really considered before.
I wrote a post over in the Software Testing Club about it.
A good use-case of testers tested by someone.
Investigation followed by "abuse" cases is the real way to find (and exploit) the vulnerabilities, and there is only a very tiny percentage of scripted test suites that cover such abuse cases.
Ashwin Palaparthi,
apalaparthi.blogspot.com
@Anna,
Wow. That's an interesting idea - tests that testers avoid scripting. As a scripted tester, that's a blind spot that I hadn't really considered before.Also, there are way too many blind spots for a scripted tester.
For instance, if you are using Gmail, and I ask you, "What is there on the right bottom of the screen when you logon to Gmail?", What's your answer?
I see scripted testers are finding stars through a telescope and that's a bad idea of finding many stars.
An investigation might need a telescope.
Would love to read a follow-up post where you go into more detail on your heuristics on tests that scripted testers avoid.
Very interesting post
I could reset the password of any account by tweaking the variables that the client was using to interact with the server.
- Can you elaborate this in more detail - coz I have a serious doubt on the developers skill - and the architecture of the application if what you said is correct.
@Philk,
Would love to read a follow-up post where you go into more detail on your heuristics on tests that scripted testers avoid.
Very interesting postYeah, the comment from Anna made me more conscious of what was running on my mind.
I am hopeful to follow up.
@Anonymous,
Can you elaborate this in more detail - coz I have a serious doubt on the developers skill - and the architecture of the application if what you said is correct.Some systems have a feature provided to turn off all auto e-mail notifications. Its similar to a situation where you register on a new website and there is an option for you to switch off all update e-mails to you.
Nothing about developers skill though, I think. However being able to allow me to switch off is probably a different kind of problem.
Hi
Nice blog with good content. Can you write about your real time experience regarding automated testing.So that it will be beneficial to candidates who know about automated testing but never tried in real time.
@Santosh,
Nice blog with good content. Can you write about your real time experience regarding automated testing.So that it will be beneficial to candidates who know about automated testing but never tried in real time.Oh, I have and will.
Post a Comment