On Sunday July 23rd ‘09, Ajay and I had planned to do a Pair Testing, the concept which Ajay and Parimala started the week before. So we were supposed start at around 9pm in the evening but as I had some unavoidable tasks that came up and it was 11:15pm by the time I and Ajay got together. And we found a sleepless wanderer in Sharath, who more than made up for our disappointment of starting late. So now we three decided to start the testing threesome. The first challenge was to find an application for testing. We didn’t expect this to be a huge challenge and had decided to pick an application at run-time, and we made a strategic blunder here. It took us more than 45 mins. to decide upon which application to Test. We tried many Open Source applications to begin with but we couldn’t find one for which we three had all software installed. So we left the Open Source applications and Ajay came up with the idea to test http://www.tinyurl.com/ . We finally had an application which could serve our thirst for the night.
Our mission was set as “Find as many bugs as possible”. We started our testing and the three testing brains were together attacking this application and the bugs started to surface. All three were connected over g-talk and it was fun to learn the ideas coming out and putting my ideas upfront. There was focus on usability, functionality and security (we couldn’t do much on this front like bypassing the firewall with proxy url [sharat’s idea] but some very good ideas were discussed). There were good ideas put up by Ajay too on the functionality of tinyurl, I found some interesting user experiences (some very embarrassing) over net and tried some similar ones. You can read the postmortem of the session done by “Ajay” in the form of a very nice report here. It was a great learning, in terms of testers testing the same application sharing their ideas and getting to know their approach, which rarely happen at workplaces.
Thanks to Ajay and Sharath for a fun filled, sharing and learning weekend session.
If you are a Tester and want to share similar fun and learning with us, you can put a mail to weekendtesting@gmail.com and we will come back to you with details of how and when sessions can be arranged.
We will Infect you too,
-Bangalore Weekend Testers
This post is created by spying on a Testing exercise that Pradeep was doing with Santhosh. I didn’t carry my moleskine diary at that time so I will to put down all I could remember. These are the benefits which one can get when you are with Pradeep, you will find unexpected ways of learning things. So I am sharing my experience for all others who were not there.
So the context was, we were near the chatting zone in Edista office. Pradeep asked Santhosh while pointing at a small coconut which was lying on the floor to come up with ideas of how that coconut might have got there. Santhosh began putting forward his ideas by looking at the coconut tree which was behind us, and said it might have fallen from the tree there. To this Pradeep asked, the place where the coconut is lying may suggest that the coconut may not have fallen directly to this place. To this Santhosh modified his idea by suggesting that it may have fallen here by hitting the parapet on its way. Then he came up with ideas like it may not actually be a coconut although it seems to be. Then it might have been blown here by the wind. Next idea was, may be some birds might have brought it here. Then Santhosh gave up.
Now Pradeep took up the challenge and showed how he would have approached this problem. He started by saying that it might would had fallen somewhere else and the guy who might have come for a walk may have actually kicked the coconut here. Then he said may be this coconut may not have fallen from the tree we are looking at and may be from other tree which was bit far and some squirrel or bird might have brought it here. Next idea was he went and picked up that coconut which was lying there and upon looking at it, he found some marks which indicated that it was eaten from one side. This gave more validity to the idea that it would have been brought there by squirrel or a bird. Then the idea that it may not be a coconut was also validated. Then other ideas were like, may be some coconut vendor might had come there and would have thrown it over there, then may be ideas like ones from Tamil movies of this coconut falling on the adjoining building and would had traversed different paths by hitting various parapets and may have landed to the place where we are seeing it etc.
This exercise gives us a learning that if we are conscious of our surroundings and environment then lots of things can be learnt. Such exercises would improve our observation skills, logical skills, interpretation of various facts and lots of other stuff which can help a Tester in his Testing activities. If you come across such exercises make sure to collect those and discuss and play it with your friends, it can be a lot of fun as well as learning.
Happy Learning,
- Manoj
One of the things which most of the Testers (me inclusive, many a times I have also not realized the importance of doing follow-up testing) don’t do when they get an error is - to do follow up testing. This mostly reduces the value of the Bug that you are reporting. The reason is you still have not identified the Bug in its worst form or not investigated the potential of that Bug. This would sometimes make the developers to defer the bug you have reported and the Bug may take its worst form in the field.
If a Tester wants the Bug reported by him to be fixed by the developer , then ensuring that he has investigated the Bug and reporting that Bug in its worst form, goes a long way in getting this done.
I was testing a text field in a web application which was not supposed to accept more than 1000 characters. I generated 1010 chars using PerlClip and pasted on this field. It just accepted 1000 characters and rejected the remaining 10 characters. I again tried appending more characters to the existing 1000 characters by trying to paste more characters, but it was also rejected. So I tried appending by giving keyboard input beyond 1000 characters and to my surprise the characters were getting appended. At this point, I could had logged a Bug saying that I am able to input more than 1000 characters to the text field which is an issue as per requirements. But I decided to do some follow up tests and tried to input as many characters as possible. I found that the field was allowing only 1000 characters to be pasted at a time, but if you give some keyboard input then again you can paste 1000 characters. So using this method I gave inputs of around 3 lakh characters. Then when I did further operation with the application where this data in the field had to be retained across couple of pages, the application initially slowed down when passing this data to the first page and then by the second page the application hung for around for 10mins before recovering.
Here if I had reported the Bug for the first error that I encountered i.e. “the text field accepting more than 1000 characters as input”, the severity of this Bug could not have been captured. But my follow-up test results, where I was able to input around 3 lakh characters to the text field and upon doing further operations the web application getting hanged showed the Bug in a more severe form. So the Bug report for the second case is more likely to catch the attention of the developer and has a better probability of getting fixed. The follow-up tests are sometimes simple and helps Testers in coming up with more credible and persuasive Bug Reports.
Bug Reports are one of the most important work products, produced by a Tester. Many a times the client, management, developers and test team members judge a Tester’s skill from the Bug Reports he creates. So Bug Report is a very important document which affects the credibility of a Tester.
Here I would like to list down some of the vocabulary given/described by Cem Kaner, which would help Testers in writing better Bug Reports.
Error: An error (or fault) is a design flaw or deviation from a desired or intended state. For eg. -: This could be a Coding mistake or a missing code.
Symptom: This might be a characteristic of a failure which helps a Tester in recognizing that the program has failed. For example -: Your application slows down when a user performs certain operations, is a symptom.
Failure: The failure is the program’s actual incorrect or missing behavior under the error triggering conditions.
Critical Condition: The condition which would trigger a failure.
It would be useful for a Tester to distinguish between an Error, critical condition and failure while writing their bug reports. I would like to take an example from the Testing activity that I was doing to drive in these points. I was testing a Username and Password field of a Sign-in page of a website. It worked fine when I provided a correct username and password and I was able to login. Then I thought of checking the maximum allowed characters in each of these fields. I found that the maximum field length of both the Username and Password fields were not set. I was able to provide more than 3 lakh characters in each field and when I clicked on 'Go', it was trying to send these huge strings to the server and the Sign-in page hanged and did not respond for more than 25 mins and eventually I had to forcefully close the browser.
In the above scenario, the “missing code for setting the maximum field length of the Username and Password field’s” is an Error. The Critical Condition (or failure triggering condition) is the user giving more than 3 lakh characters as input in both Username and Password field’s and clicking on 'Go' button. And the Failure was the Sign-in page hanged for the Critical condition.One of the ideas I totally disagree with people who advocate Test Automation is their claims of Automation tools being able to run tests much faster as compared to Human Testers. But my question is why do you want to compare an Automation Tool to a Human Tester? How did this comparison even arise?
According to me, a Tool can be some device which may be used to facilitate or aid in performing manual tasks. So, why should speed of Human is compared to speed of an aid he uses?
I remember using Scientific Calculators in my Engineering Degree. The calculator was used as a tool aiding in calculations. Few formulae could also be fed or stored into it. But, I have never heard anybody saying that the calculators should be replacing the engineers as it is able to execute the calculations much faster than humans. But what good is a calculator without humans? It cannot perform any task on its own, it cannot have logic, it cannot think. But it is very useful if an Engineer who has a calculator knows what benefits he can derive from it and knows when, how and where to use it and get maximum benefits from it.
We should understand that the purpose of a Calculator or a Test Automation tool is to aid humans in doing their tasks in a better way. This would allow humans to spend their time more judiciously on more important things in what he is good at and allowing the tools to do tasks for which they were specifically made for. Even the Test Automation is a heavily human intensive task. So a Test Automation tool can execute tests much faster than humans, but it cannot do anything by itself. The Test Design has to be done by humans, identifying tests for automation, creating automation scripts, maintaining automation scripts, identifying the comparison criteria (pass/fail criteria) and investigating the issues if any mis-match is found in the Test execution results, all these activities has to be done by human testers.
So, next time when you hear someone speaking about Test Automation tools being able to execute tests faster than Human Testers, make him understand why he should not be doing and talking about such comparisons.
