~10 years ago
In 2003, I worked for Impulsesoft, a start-up company that claimed to be first in demonstrating CD Quality (which is 44.1 KHz sampling, at that time) stereo music over Bluetooth. I was extremely proud to have been a tester for the world's first Bluetooth Headphones. At that time, you may have called it a bleeding edge technology.
On a yet another beautiful Sunday morning I got a call from my manager to help someone in the marketing team who was flying to Germany for a conference. He was flying to a conference which had delegates who were potential buyers of our product. At that time, I don't know if the business model was B2B or B2C. However, there were a few known crashes of the Bluetooth Headphones I was testing. I knew those crashes, after all, I found them.
My idea was to help him learn to not do anything that makes the headphones crash in front of the delegates. I gave him a list of do's and don'ts. I also demoed it to him on what would be happy path and what would be a dangerous path. I remember telling him to be careful of making the claims to potential buyers. On inquiring him I found he was seeking my help few hours before he was about to board a flight to Frankfurt. I did my best and wished him good luck for the demo.
2 days later, I get an email forwarded by my manager. With no surprises, the email had a content like, "Pradeep didn't test well, there are plenty of crashes and our potential buyers were not happy to experience it" written by the marketing fella whom I had coached on do's and dont's. I was called for a meeting to explain the same. I went with a print out of how we had already reported those bugs and with an explanation of how last minute prep for a demo like this is a bad idea. Thankfully, my manager and the CTO were sane to understand. So, they said, "this is not a meeting to catch you responsible for it but to check how did you help the marketing guy".
I was excited about the whole experience. I started to think I could have done a bunch of stuff if I had time and I could really help customers. Later, I improved upon my approach to help customers and my employers if they were to ask me to "Test for Demo"!
~ 8 months ago
One of Moolya's customer is Cogknit. They are making products in the e-learning space to take the e-learning to i-learning and have been liking what Moolya has been doing to their product. They are boot strapped start up, young fellas, big dream and lot of sacrifice to get to a position where they are right now. About 8 months ago, they informed me that they were to be giving a demo and sought for my help.
I was so excited to bring in all my experience and expertise on this matter to Cogknit and gave this a high priority. I donned upon the role of Demo Test Architect without having to seek anyone's permission for it.
I had a meeting fixed with the founders of Cogknit and asked them a bunch of questions on
a) the conference - World Education Congress
b) the delegates (the potential buyers)
b) the objective
c) the demo plan they had (no matter how primitive it was)
d) their assumptions on what would not block them from delivering the demo
I set up a meeting with the developers from Multunus who were helping with the GUI and or front end, developers from Cogknit, testers from Moolya, the marketing fella - Deepak (co-founder of Cogknit), and Anuroop (the founder) with an objective to bring all to common ground and brief them about my plan and interest.
As a next step, I asked Anuroop and Deepak to give me a demo of what they intend to demo and I kept recording the steps they followed.
I did my home work on what could go wrong. For instance when Anuroop was demoing, he had already logged into Nimit (the web based i-learning product). I asked him to logout and then login again, start from first. Surprise! When he tried entering the password quickly, he made a mistake and bang, an error message, blood red in color appeared. At a demo, customers don't intend to see if errors are handled well but would want to see more of how the product works. There were too many blood red error info displayed for other actions that went wrong. If Anuroop were to accidentally type something in the search whose results are zero then the blood red appears which is good maybe for users but here there are no users except Anuroop and Deepak who are going to be demoing.
My further homework and thinking suggested that the demo ready version of the product may not need to have these error messages at all. I held another meeting and requested the developers to give me a version that has no error messages displayed. "Remove all blood red" was my instruction. Probably, all of that code commented. I asked Anuroop to give me a demo again after the change happened. We also changed the password to something so simple that going wrong typing that is a sin :)
We went through multiple iterations of demo practice and kept tweaking the product to make it more suitable for demo.
I am guessing that Anuroop might have thought - why is this guy trying to teach me how to use the product I am building :) All for a good reason, dear Anuroop! The testers on the team supported me with a Don't Do Anything Like This list. As an example, to copy paste from their recommendation:
Do not drag & drop any book from the "Library" row to the Mashup tray The books other than Mashup shown in the "Library" row are either rented or bought contents. So, mashup of rented or bought contents are not allowed.
Even if Anuroop accidentally did this, the product wouldn't shout, "Hey, here is an error message for you"!
My goal was to ensure Anuroop didn't do any of this. So every time he gives me a demo of what he thinks his demo is, I secretly run through the don't do this list to figure out if Anuroop is crossing the boundary of product areas marked as danger.
There was also a plan to make a feature work in the last minute and get it for the demo version. I humbly rejected the idea. No last minute changes to the product was my directive.
Once I had a version that looked to me as what can be presented during demo, I asked my fellow Moolya testers to run it through automated checks and exploratory tests and give me information that could make my decision change. I personally also toured the demo version of the product to find if my oracles help me be warned of something I should be. Things were starting to look OK barring a few minor changes needed.
The conference being on a week day and hosted on the same server where test instance and dev instance was running, I gave a directive to ask people to stop working on the dev / test / demo instance 2 hours before the time of the demo and wait for confirmation of demo over to re-start their work. This is to ensure nobody ends up doing anything to the server, even by accident. So, there was a stoppage of work for several hours during the demo.
I made a list of infrastructure requirements and checked the laptop they were getting for demo and made recommendations to turn off all alerts of the OS, Anti Virus, Apps and other nuisance that could disturb the demo from going smooth. I can't exactly remember if all of that was done but I guess Anuroop did mention that he would take care of it. Probably he did.
Then about internet connection for the laptop. I made recommendations to carry backup of internet dongles and to check at the venue a day before on the connectivity. Deepak and Anuroop did do a on the stage previous day test of internet connection and deemed it to be good enough.
During these days, the only question I kept asking myself, "What could I be missing?". A new idea kept emerging and I kept refining things better.
Last 2 days before the conference was more awesome adrenaline for me. Continuous stream of ideas and thought process. All through this period, I kept updating my notes and updating them to keep them reminded of what I am capturing and if it needs correction.
So the real D-Day came and I was nervously calling Ullas every one hour to check if Anuroop or Deepak were done with the demo and if they have any issues that they would need my help. I didn't do any other work that day and was totally thinking what would be going on for the demo. This was not nervousness, it was excitement to see the demo go well and feel happy about all the effort I took to get it going well.
The demo did happen, thankfully but the internet was slow. Thankfully nothing did time out or I heard that way. However, the next thing I include in my list is changing time out duration to ensure slow internet connections don't scare too much. The demo did generate good business interest for Cogknit and they got inquiries from Singapore and other countries that I don't exactly remember.
To add a feather to the crown, Nimit was awarded The Most Innovative Learning Software Product. Here is a picture of Anuroop (right) and Deepak (left) receiving the award from his excellency Nurul Islam Nahid, Minister of Education - Bangladesh & David Namwandi, Minister of Education, Namibia
Many thanks to Anuroop, Ullas and Deepak for being patient with me during the journey and allowing me to play the Demo Test Architect role. Equal good thanks to the developers at Multunus and Cogknit (especially to Himanshu) for supporting me on this. A special thanks to my own test team who have been helping Cogknit for more than a year now. Cogknit got their first big paying customer as well recently and they are sailing to get more.
I just hope, my colleague, Dhanasekar would be as proud of me as he is about Apple on how they prepare themselves for a product demo. More experience reports, coming soon.
In 2003, I worked for Impulsesoft, a start-up company that claimed to be first in demonstrating CD Quality (which is 44.1 KHz sampling, at that time) stereo music over Bluetooth. I was extremely proud to have been a tester for the world's first Bluetooth Headphones. At that time, you may have called it a bleeding edge technology.
On a yet another beautiful Sunday morning I got a call from my manager to help someone in the marketing team who was flying to Germany for a conference. He was flying to a conference which had delegates who were potential buyers of our product. At that time, I don't know if the business model was B2B or B2C. However, there were a few known crashes of the Bluetooth Headphones I was testing. I knew those crashes, after all, I found them.
My idea was to help him learn to not do anything that makes the headphones crash in front of the delegates. I gave him a list of do's and don'ts. I also demoed it to him on what would be happy path and what would be a dangerous path. I remember telling him to be careful of making the claims to potential buyers. On inquiring him I found he was seeking my help few hours before he was about to board a flight to Frankfurt. I did my best and wished him good luck for the demo.
2 days later, I get an email forwarded by my manager. With no surprises, the email had a content like, "Pradeep didn't test well, there are plenty of crashes and our potential buyers were not happy to experience it" written by the marketing fella whom I had coached on do's and dont's. I was called for a meeting to explain the same. I went with a print out of how we had already reported those bugs and with an explanation of how last minute prep for a demo like this is a bad idea. Thankfully, my manager and the CTO were sane to understand. So, they said, "this is not a meeting to catch you responsible for it but to check how did you help the marketing guy".
I was excited about the whole experience. I started to think I could have done a bunch of stuff if I had time and I could really help customers. Later, I improved upon my approach to help customers and my employers if they were to ask me to "Test for Demo"!
~ 8 months ago
One of Moolya's customer is Cogknit. They are making products in the e-learning space to take the e-learning to i-learning and have been liking what Moolya has been doing to their product. They are boot strapped start up, young fellas, big dream and lot of sacrifice to get to a position where they are right now. About 8 months ago, they informed me that they were to be giving a demo and sought for my help.
I was so excited to bring in all my experience and expertise on this matter to Cogknit and gave this a high priority. I donned upon the role of Demo Test Architect without having to seek anyone's permission for it.
I had a meeting fixed with the founders of Cogknit and asked them a bunch of questions on
a) the conference - World Education Congress
b) the delegates (the potential buyers)
b) the objective
c) the demo plan they had (no matter how primitive it was)
d) their assumptions on what would not block them from delivering the demo
I set up a meeting with the developers from Multunus who were helping with the GUI and or front end, developers from Cogknit, testers from Moolya, the marketing fella - Deepak (co-founder of Cogknit), and Anuroop (the founder) with an objective to bring all to common ground and brief them about my plan and interest.
As a next step, I asked Anuroop and Deepak to give me a demo of what they intend to demo and I kept recording the steps they followed.
I did my home work on what could go wrong. For instance when Anuroop was demoing, he had already logged into Nimit (the web based i-learning product). I asked him to logout and then login again, start from first. Surprise! When he tried entering the password quickly, he made a mistake and bang, an error message, blood red in color appeared. At a demo, customers don't intend to see if errors are handled well but would want to see more of how the product works. There were too many blood red error info displayed for other actions that went wrong. If Anuroop were to accidentally type something in the search whose results are zero then the blood red appears which is good maybe for users but here there are no users except Anuroop and Deepak who are going to be demoing.
My further homework and thinking suggested that the demo ready version of the product may not need to have these error messages at all. I held another meeting and requested the developers to give me a version that has no error messages displayed. "Remove all blood red" was my instruction. Probably, all of that code commented. I asked Anuroop to give me a demo again after the change happened. We also changed the password to something so simple that going wrong typing that is a sin :)
We went through multiple iterations of demo practice and kept tweaking the product to make it more suitable for demo.
I am guessing that Anuroop might have thought - why is this guy trying to teach me how to use the product I am building :) All for a good reason, dear Anuroop! The testers on the team supported me with a Don't Do Anything Like This list. As an example, to copy paste from their recommendation:
Do not drag & drop any book from the "Library" row to the Mashup tray The books other than Mashup shown in the "Library" row are either rented or bought contents. So, mashup of rented or bought contents are not allowed.
Even if Anuroop accidentally did this, the product wouldn't shout, "Hey, here is an error message for you"!
My goal was to ensure Anuroop didn't do any of this. So every time he gives me a demo of what he thinks his demo is, I secretly run through the don't do this list to figure out if Anuroop is crossing the boundary of product areas marked as danger.
There was also a plan to make a feature work in the last minute and get it for the demo version. I humbly rejected the idea. No last minute changes to the product was my directive.
Once I had a version that looked to me as what can be presented during demo, I asked my fellow Moolya testers to run it through automated checks and exploratory tests and give me information that could make my decision change. I personally also toured the demo version of the product to find if my oracles help me be warned of something I should be. Things were starting to look OK barring a few minor changes needed.
The conference being on a week day and hosted on the same server where test instance and dev instance was running, I gave a directive to ask people to stop working on the dev / test / demo instance 2 hours before the time of the demo and wait for confirmation of demo over to re-start their work. This is to ensure nobody ends up doing anything to the server, even by accident. So, there was a stoppage of work for several hours during the demo.
I made a list of infrastructure requirements and checked the laptop they were getting for demo and made recommendations to turn off all alerts of the OS, Anti Virus, Apps and other nuisance that could disturb the demo from going smooth. I can't exactly remember if all of that was done but I guess Anuroop did mention that he would take care of it. Probably he did.
Then about internet connection for the laptop. I made recommendations to carry backup of internet dongles and to check at the venue a day before on the connectivity. Deepak and Anuroop did do a on the stage previous day test of internet connection and deemed it to be good enough.
During these days, the only question I kept asking myself, "What could I be missing?". A new idea kept emerging and I kept refining things better.
Last 2 days before the conference was more awesome adrenaline for me. Continuous stream of ideas and thought process. All through this period, I kept updating my notes and updating them to keep them reminded of what I am capturing and if it needs correction.
So the real D-Day came and I was nervously calling Ullas every one hour to check if Anuroop or Deepak were done with the demo and if they have any issues that they would need my help. I didn't do any other work that day and was totally thinking what would be going on for the demo. This was not nervousness, it was excitement to see the demo go well and feel happy about all the effort I took to get it going well.
The demo did happen, thankfully but the internet was slow. Thankfully nothing did time out or I heard that way. However, the next thing I include in my list is changing time out duration to ensure slow internet connections don't scare too much. The demo did generate good business interest for Cogknit and they got inquiries from Singapore and other countries that I don't exactly remember.
To add a feather to the crown, Nimit was awarded The Most Innovative Learning Software Product. Here is a picture of Anuroop (right) and Deepak (left) receiving the award from his excellency Nurul Islam Nahid, Minister of Education - Bangladesh & David Namwandi, Minister of Education, Namibia
 |
| Anuroop and Deepak receiving The Most Innovative Learning Software Product from his excellency Nurul Islam Nahid, Minister of Education - Bangladesh & David Namwandi, Minister of Education, Namibia in World Education Congress 2012 |
Many thanks to Anuroop, Ullas and Deepak for being patient with me during the journey and allowing me to play the Demo Test Architect role. Equal good thanks to the developers at Multunus and Cogknit (especially to Himanshu) for supporting me on this. A special thanks to my own test team who have been helping Cogknit for more than a year now. Cogknit got their first big paying customer as well recently and they are sailing to get more.
I just hope, my colleague, Dhanasekar would be as proud of me as he is about Apple on how they prepare themselves for a product demo. More experience reports, coming soon.
9 comments:
Awesome post, lot of learnings....
great post! learnt while reading....
I am curious to know the duration of this test to demo. The reason being, my deeper curiosity on what automated checks did you run (as you stated you asked your team to do automated checks) and how quick was your automation scripts generated.
@Andy Rocks,
As we have been working with them for the past year, we didn't build automated checks specific for testing the demo version and hence we harnessed that.
For a different project context, we needed quick dirty automation checks and did a record playback. I thought it was a great context driven approach of record playback.
Great post & unbelieveable timely and relevant for me & my team at Xibis! Thanks Pradeep :-)
@oadby_alex
Informative! Great experience of reading from your blog. Thanks
Very informative post. I have had this experience of doing a lot of client demos. My product being a client server app, I set it up entirely in my laptop (both the client and server) and I do 2 or 3 dry runs before the demos. Only I knew exactly what flows would work in the demo version.
And I documented those for others to do the demo in case that requirement arised. With this approach, we ended up saving our lives in a lot of such client demos where I could not give the demo.
Regards
Rajaraman Raghuraman
Test Data Management Blog
Nice post Pradeep. It's interesting how many roles a tester have to put on in different contexts. I once worked on a project where our objective was to have the product showcased at a massive electronic show. It was a fantastic experience for the team because the challenges as you have explained are very different.
I like the way you have attacked (may be not the right word) the product(happy/demo scenarios, error msgs), product environment(connectivity, pop ups/alerts on the system), project environment(people, last min changes) and the demo delivery flow. Brilliant. Just one other thing how much time did you all spend on setting up the data?
To impress Dhanasekar I guess you will have to add demo environment to your list(lighting, how r the seats arranged, do they all get the view, do background music helps/add in the surprise?, is the sound audible to everyone, stage background, how much of stage do you use, etc.) May be you did have a look into it. I picked some of these from Steves book :)
Thank you Pradeep. Your post illustrated on not just what all things that are not be taken care of for a demo but also for any important event. I will never forget the lesson learnt for this.
Always a fan of you.
Post a Comment