On Sunday July 23rd ‘09, Ajay and I had planned to do a Pair Testing, the concept which Ajay and Parimala started the week before. So we were supposed start at around 9pm in the evening but as I had some unavoidable tasks that came up and it was 11:15pm by the time I and Ajay got together. And we found a sleepless wanderer in Sharath, who more than made up for our disappointment of starting late. So now we three decided to start the testing threesome. The first challenge was to find an application for testing. We didn’t expect this to be a huge challenge and had decided to pick an application at run-time, and we made a strategic blunder here. It took us more than 45 mins. to decide upon which application to Test. We tried many Open Source applications to begin with but we couldn’t find one for which we three had all software installed. So we left the Open Source applications and Ajay came up with the idea to test http://www.tinyurl.com/ . We finally had an application which could serve our thirst for the night.
Our mission was set as “Find as many bugs as possible”. We started our testing and the three testing brains were together attacking this application and the bugs started to surface. All three were connected over g-talk and it was fun to learn the ideas coming out and putting my ideas upfront. There was focus on usability, functionality and security (we couldn’t do much on this front like bypassing the firewall with proxy url [sharat’s idea] but some very good ideas were discussed). There were good ideas put up by Ajay too on the functionality of tinyurl, I found some interesting user experiences (some very embarrassing) over net and tried some similar ones. You can read the postmortem of the session done by “Ajay” in the form of a very nice report here. It was a great learning, in terms of testers testing the same application sharing their ideas and getting to know their approach, which rarely happen at workplaces.
Thanks to Ajay and Sharath for a fun filled, sharing and learning weekend session.
If you are a Tester and want to share similar fun and learning with us, you can put a mail to weekendtesting@gmail.com and we will come back to you with details of how and when sessions can be arranged.
We will Infect you too,
-Bangalore Weekend Testers
This post is created by spying on a Testing exercise that Pradeep was doing with Santhosh. I didn’t carry my moleskine diary at that time so I will to put down all I could remember. These are the benefits which one can get when you are with Pradeep, you will find unexpected ways of learning things. So I am sharing my experience for all others who were not there.
So the context was, we were near the chatting zone in Edista office. Pradeep asked Santhosh while pointing at a small coconut which was lying on the floor to come up with ideas of how that coconut might have got there. Santhosh began putting forward his ideas by looking at the coconut tree which was behind us, and said it might have fallen from the tree there. To this Pradeep asked, the place where the coconut is lying may suggest that the coconut may not have fallen directly to this place. To this Santhosh modified his idea by suggesting that it may have fallen here by hitting the parapet on its way. Then he came up with ideas like it may not actually be a coconut although it seems to be. Then it might have been blown here by the wind. Next idea was, may be some birds might have brought it here. Then Santhosh gave up.
Now Pradeep took up the challenge and showed how he would have approached this problem. He started by saying that it might would had fallen somewhere else and the guy who might have come for a walk may have actually kicked the coconut here. Then he said may be this coconut may not have fallen from the tree we are looking at and may be from other tree which was bit far and some squirrel or bird might have brought it here. Next idea was he went and picked up that coconut which was lying there and upon looking at it, he found some marks which indicated that it was eaten from one side. This gave more validity to the idea that it would have been brought there by squirrel or a bird. Then the idea that it may not be a coconut was also validated. Then other ideas were like, may be some coconut vendor might had come there and would have thrown it over there, then may be ideas like ones from Tamil movies of this coconut falling on the adjoining building and would had traversed different paths by hitting various parapets and may have landed to the place where we are seeing it etc.
This exercise gives us a learning that if we are conscious of our surroundings and environment then lots of things can be learnt. Such exercises would improve our observation skills, logical skills, interpretation of various facts and lots of other stuff which can help a Tester in his Testing activities. If you come across such exercises make sure to collect those and discuss and play it with your friends, it can be a lot of fun as well as learning.
Happy Learning,
- Manoj
One of the things which most of the Testers (me inclusive, many a times I have also not realized the importance of doing follow-up testing) don’t do when they get an error is - to do follow up testing. This mostly reduces the value of the Bug that you are reporting. The reason is you still have not identified the Bug in its worst form or not investigated the potential of that Bug. This would sometimes make the developers to defer the bug you have reported and the Bug may take its worst form in the field.
If a Tester wants the Bug reported by him to be fixed by the developer , then ensuring that he has investigated the Bug and reporting that Bug in its worst form, goes a long way in getting this done.
I was testing a text field in a web application which was not supposed to accept more than 1000 characters. I generated 1010 chars using PerlClip and pasted on this field. It just accepted 1000 characters and rejected the remaining 10 characters. I again tried appending more characters to the existing 1000 characters by trying to paste more characters, but it was also rejected. So I tried appending by giving keyboard input beyond 1000 characters and to my surprise the characters were getting appended. At this point, I could had logged a Bug saying that I am able to input more than 1000 characters to the text field which is an issue as per requirements. But I decided to do some follow up tests and tried to input as many characters as possible. I found that the field was allowing only 1000 characters to be pasted at a time, but if you give some keyboard input then again you can paste 1000 characters. So using this method I gave inputs of around 3 lakh characters. Then when I did further operation with the application where this data in the field had to be retained across couple of pages, the application initially slowed down when passing this data to the first page and then by the second page the application hung for around for 10mins before recovering.
Here if I had reported the Bug for the first error that I encountered i.e. “the text field accepting more than 1000 characters as input”, the severity of this Bug could not have been captured. But my follow-up test results, where I was able to input around 3 lakh characters to the text field and upon doing further operations the web application getting hanged showed the Bug in a more severe form. So the Bug report for the second case is more likely to catch the attention of the developer and has a better probability of getting fixed. The follow-up tests are sometimes simple and helps Testers in coming up with more credible and persuasive Bug Reports.
Bug Reports are one of the most important work products, produced by a Tester. Many a times the client, management, developers and test team members judge a Tester’s skill from the Bug Reports he creates. So Bug Report is a very important document which affects the credibility of a Tester.
Here I would like to list down some of the vocabulary given/described by Cem Kaner, which would help Testers in writing better Bug Reports.
Error: An error (or fault) is a design flaw or deviation from a desired or intended state. For eg. -: This could be a Coding mistake or a missing code.
Symptom: This might be a characteristic of a failure which helps a Tester in recognizing that the program has failed. For example -: Your application slows down when a user performs certain operations, is a symptom.
Failure: The failure is the program’s actual incorrect or missing behavior under the error triggering conditions.
Critical Condition: The condition which would trigger a failure.
It would be useful for a Tester to distinguish between an Error, critical condition and failure while writing their bug reports. I would like to take an example from the Testing activity that I was doing to drive in these points. I was testing a Username and Password field of a Sign-in page of a website. It worked fine when I provided a correct username and password and I was able to login. Then I thought of checking the maximum allowed characters in each of these fields. I found that the maximum field length of both the Username and Password fields were not set. I was able to provide more than 3 lakh characters in each field and when I clicked on 'Go', it was trying to send these huge strings to the server and the Sign-in page hanged and did not respond for more than 25 mins and eventually I had to forcefully close the browser.
In the above scenario, the “missing code for setting the maximum field length of the Username and Password field’s” is an Error. The Critical Condition (or failure triggering condition) is the user giving more than 3 lakh characters as input in both Username and Password field’s and clicking on 'Go' button. And the Failure was the Sign-in page hanged for the Critical condition.One of the ideas I totally disagree with people who advocate Test Automation is their claims of Automation tools being able to run tests much faster as compared to Human Testers. But my question is why do you want to compare an Automation Tool to a Human Tester? How did this comparison even arise?
According to me, a Tool can be some device which may be used to facilitate or aid in performing manual tasks. So, why should speed of Human is compared to speed of an aid he uses?
I remember using Scientific Calculators in my Engineering Degree. The calculator was used as a tool aiding in calculations. Few formulae could also be fed or stored into it. But, I have never heard anybody saying that the calculators should be replacing the engineers as it is able to execute the calculations much faster than humans. But what good is a calculator without humans? It cannot perform any task on its own, it cannot have logic, it cannot think. But it is very useful if an Engineer who has a calculator knows what benefits he can derive from it and knows when, how and where to use it and get maximum benefits from it.
We should understand that the purpose of a Calculator or a Test Automation tool is to aid humans in doing their tasks in a better way. This would allow humans to spend their time more judiciously on more important things in what he is good at and allowing the tools to do tasks for which they were specifically made for. Even the Test Automation is a heavily human intensive task. So a Test Automation tool can execute tests much faster than humans, but it cannot do anything by itself. The Test Design has to be done by humans, identifying tests for automation, creating automation scripts, maintaining automation scripts, identifying the comparison criteria (pass/fail criteria) and investigating the issues if any mis-match is found in the Test execution results, all these activities has to be done by human testers.
So, next time when you hear someone speaking about Test Automation tools being able to execute tests faster than Human Testers, make him understand why he should not be doing and talking about such comparisons.
I am blogging after some time gap. One of the things that I did during this break was to do BBST Foundation course. Here I would like to share my experience of this 1 month long course and why I think it would be useful for Indian Software Testing community.
BBST foundation course is a basic introduction to ‘Black Box’ Testing. You can read the details about this course here. This course includes video lectures, quizzes, homework of various kinds, and a final exam. The course is provided Free of Cost for the AST members. So this is a course which is non-commercial in nature and is dedicated for the betterment of the Testing community.
Ideally one should have some Industry experience for taking up this course although it is not mandated. But I would say, for reaping better benefits from the course a Software Tester should have atleast 2-3 years of experience in IT industry.
There were 11 participants in my batch mostly from US, although there were participants from other countries as well. Apart from me, there was one more Indian (Rajkumar Pandian from Chennai) in my batch. The batch began with ‘Meet and Greet’ session where all the Testers had to introduce oneself to the batch. It was really cool and gave all Testers a platform to share their interests and make friends with like-minded testers and other testers in the group. There were Testers with wide range of experience, right from beginner (1 tester) to Testers with 5-7 years experience and also people who have chosen Testing after having experience in different fields. So this a playground where Testers can put forward their point of view, debate, learn from other Testers, learn how Testers work in different geographies, how the challenges differ and their experiences. This is place where your work would be challenged and put under scrutiny by Testers with different set of ideas and expectations, and you will have to defend your work. Your work would also be evaluated by some of the Experts in the Software Testing field who have got decades of experience.
It is really great Fun!! and lots of learning too. I am not sure whether Testers who had appeared for CSTE or ISTQB or other similar exams would have experienced anything close to this. This course had got many ingredients where a Tester’s skills are really put to test and clearing this exam would build a lot of “Credibility” for the Tester within the Testing community.
Having gone through the grind and fun for 1 month of BBST course, I believe this course deserves more recognition from the Indian IT Industry. This course would give the IT industry a better insight about their Testers ability and skills than any CSTE or ISTQB exam can actually give. But for this to happen, I believe more Testers from India will have to take-up this course and share their experience with the Testing community.
I was logging into my yahoo account, when I realized when I had logged in an hour back the home page was different. I thought, the homepage might have been updated? Or was there a difference the way I accessed the Yahoo! Mail home page the last time?
Then upon little investigation, I realized that the last time I had traversed to the home page was in the following way– I went to www.yahoo.com and clicked on the ‘Mail’ option. The second time I had reached homepage through – www.mail.yahoo.com
This got me interested and I wanted to have a quick comparison of the two homepages which had opened. So for discussion sake, I want to use the following conventions -:
Option 1 -: Go to www.yahoo.com and click on ‘Mail’ option
Option 2 -: Go to www.mail.yahoo.com
[NOTE -: Option1 took me to ‘Yahoo! India’ homepage, while Option2 took me to ‘Yahoo! Mail’ default homepage.]
I did a quick round of comparison and following are my observations -:
(i) Option2 (The default ‘Yahoo! Mail’ home page) had a link to go to ‘Yahoo!’ homepage But in Option1 (‘Yahoo! India’ homepage) I couldn’t find any link to traverse to ‘Yahoo!’ homepage.
(ii) I noticed that the Window Title in Option1 showed “Sign in to Yahoo! India” while the Window Title in Option2 showed “Yahoo! Mail: the best web-based mail”. I wondered why Option1 didn’t have ‘Mail’ in their Window title.
So I opened the Yahoo! UK and Canada sites. Oh, there the Window title was “Sign in to Yahoo!”, Country names were not mentioned (although it is mentioned within the respective homepages). So Yahoo! doesn’t seem to have uniformity in naming the Window titles.
(iii) In Option1 (‘Yahoo! India’ homepage) there is an amazing link “Forget your ID or Password?” . Wow!!, it seems Yahoo might want to help its users to Forget their UserID or Password. What an innovative idea?
I had first found similar message a couple of weeks back when I was testing a website during the Finishing School session that Pradeep was conducting. That site was still to go Live, but it could be that they may have picked up the idea from Yahoo!.
Again I got curious, is this message present only in Option1 or is it also available in Option2 ? Luckily, Option2 had “I can’t access my account” link. But here too, Yahoo! doesn’t seem to have uniformity in naming the links. For same functionality it has got different names.
(iv) I decided to move on and have a peek into the ‘Sign Up’ page for both Option1 and Option2. Hey surprise!! , there is another “Forgot your password or Yahoo! Id?” link when entering Sign Up page through Option1 AND you know what, there is a “Forget your password or Yahoo! Id?” when you navigate to Sign Up page through Option2.
First, why do you need “Forgot you password or Yahoo! Id” within the Registration page? May be because if the user cannot see this link on the homepage, he could dig into the Registration page and find this option. Highly probable!!.
Second, even for people entering Yahoo! site by Option2 have a chance to ‘Forget their password or Yahoo Id’ by going to the Registration page. Good, why should the people using Option1 have all the fun alone?
(v) Finally l had a look at the ‘Security Question’ for both Option1 and Option2 and interestingly there were different set of Security questions asked. Why is it so? Once again I got curious and looked into Yahoo! UK and Canada sites and found that those sites have got exactly the same set of Security questions which is available from Option2 -> ‘Registration Page’. So probably Yahoo! Might have come up with special set of questions for Indian users.
------------------------------------------------------------------------------------------------------
The intention of this post is to demonstrate that if anyone wants to practice testing, he can begin with almost any application he can get. I have nothing personal against Yahoo!, it is just my observations which I have listed here. And maybe, Yahoo! has got its own reasons for implementing things the way they are.
The title of this post is inspired from Ben Simo’s ‘Is There A Problem Here?’ blog.
Hereon I plan to post the Test Notes of the “Session Based Testing” of various applications that I Test - which helps me to learn and develop Test Ideas which I can use as heuristics while testing other applications. Here is the first of the series.
===================
Product -: Wordsearch Creator V 0.9
Product Link -: http://sourceforge.net/projects/wsc/
Environment -: Windows Vista Home Basic, 32 bit OS
Tester -: M.V. Manoj
Start Date/Time -: 17/03/09 , 00.00 am
Charter/Mission -: To find out as many problems as possible that could influence the stakeholders to consider fixing all the bugs I report in order to improve the quality of the product.
#1. I am testing this product for the first time and I found that upon clicking the Help Contents, I am directed to http://openforschools.co.uk/wordsearchcreator/doc/0.9 But since at present I don’t have Internet connection, I would have to do without Help and learn/test by exploring the application. But then why a desktop application would require/mandate a user to have internet connection?
#2. Because of lack of Internet connectivity, I am unable to search the Internet to see if I can get any info or reviews about Wordsearch Creator OR similar applications.
#3. I launched 150 instances of the application. In task manager I can see that upon opening multiple instances there is a large variation in the memory usage of Wordsearch creator and it varied from 804K to 7424K. This large variations of memory usage could be a potential problem. I can’t see a reason when the user might have to open so many instances simaltaneously, so why not restrict the number of instances to one at a time?
#4. The multiple instances did not have any numbering on them, so upon cascading I could not identify in which order the application instances were launched.
#5. Close all the 150 instances in one go -> Close Group. Application closed gracefully.
#6. The ‘Control Box’ -> Height and Width fields have a lower boundary of ‘1’ each and an upper boundary of ‘99’ when I used the scroll bars. When I provided the value ‘99’ to both height & width fields it made the squares to go out of bounds of the page. The squares should not go beyond bounds of the page, whatever a user does.
#7. I was able to save the Out of Bounds page Results as ‘.PDF’ and as ‘.wordsearch’
#8. Upon providing 276496 in the Word List and clicking on create , a message ‘Only 0 words placed’ message is shown (scrnshot aval).
#9. When I copied entire A-Z in one line - multiple times and clicked on create. A message box with “Creating Wordsearch” blocks the screen and doesn’t return the result until I stopped the search.
#10. Once the grid is created even if you modify your Word List, the New answers are not displayed. i.e. After create, modify word list does not work and you have to recreate the grid to include the new wordlist.
#11. (Menu) Wordsearch à ‘Set allowed directions’ seems to be working fine in all directions. I tried a few tests and it passed.
#12. Cntrl++ does not work for Zoom In.
#13. If the grid goes out of bound of the page, Wordsearch à ‘Maskdrawer’ has problems, it also goes out of bounds and ‘Done’ cannot be obtained.
#14. Menu Options, ‘File’ and ‘Format’ has been given the same Shortcut Key ‘F’. “Alt + F” opens ‘File’. So ‘Format’ doesn’t have shortcut key.
#15. Wordsearch -> ‘Copy as Image’ works fine and passed few tests I ran.
#16. WordSearch à ‘Word List Creator’ generates the same list as many times as we click on ‘Generate List’ button. When I clicked on OK, I found that this list gets appended to ‘Control Box’ à ‘Word List’ rather than overwriting the existing list.
#17. The dialogs that comes up upon clicking the Menu options doesn’t have proper naming conventions. Wordsearch à ‘Set Allowed Directions’ –Opens- ‘Set Word Directions’ dialog. Similarly, Wordsearch -> ‘Mask Drawer’ –Opens- ‘Word Search Creator’ dialog.
#18. ** *Test Ideas Used ***
- Try to open multiple instances of the application and look if multiple instances makes any sense?
- Open lots of instances and find the memory usage pattern of the application.
- Learn by the application behaviour, the inputs and output patterns
- Try to identify the boundaries of the grid/word list
- Menu Traversal to learn the functionalities provided. Open and look at the functionalities.
- Check if short cut keys are functioning properly.
- Check the word search application with different font and colors.
- Check the title of the dialog boxes in the application.
- Check the completed grid with only answers, only word list and with both.
- Check the various directions in which search can be done.
- Check the export and import functionalities.
---End Session -: 2:22 am-----
