Testing Tips 2 -: When you find an Error in your application, do some follow up Testing

One of the things which most of the Testers (me inclusive, many a times I have also not realized the importance of doing follow-up testing) don’t do when they get an error is - to do follow up testing. This mostly reduces the value of the Bug that you are reporting. The reason is you still have not identified the Bug in its worst form or not investigated the potential of that Bug. This would sometimes make the developers to defer the bug you have reported and the Bug may take its worst form in the field.

If a Tester wants the Bug reported by him to be fixed by the developer , then ensuring that he has investigated the Bug and reporting that Bug in its worst form, goes a long way in getting this done.

I was testing a text field in a web application which was not supposed to accept more than 1000 characters. I generated 1010 chars using PerlClip and pasted on this field. It just accepted 1000 characters and rejected the remaining 10 characters. I again tried appending more characters to the existing 1000 characters by trying to paste more characters, but it was also rejected. So I tried appending by giving keyboard input beyond 1000 characters and to my surprise the characters were getting appended. At this point, I could had logged a Bug saying that I am able to input more than 1000 characters to the text field which is an issue as per requirements. But I decided to do some follow up tests and tried to input as many characters as possible. I found that the field was allowing only 1000 characters to be pasted at a time, but if you give some keyboard input then again you can paste 1000 characters. So using this method I gave inputs of around 3 lakh characters. Then when I did further operation with the application where this data in the field had to be retained across couple of pages, the application initially slowed down when passing this data to the first page and then by the second page the application hung for around for 10mins before recovering.

Here if I had reported the Bug for the first error that I encountered i.e. “the text field accepting more than 1000 characters as input”, the severity of this Bug could not have been captured. But my follow-up test results, where I was able to input around 3 lakh characters to the text field and upon doing further operations the web application getting hanged showed the Bug in a more severe form. So the Bug report for the second case is more likely to catch the attention of the developer and has a better probability of getting fixed. The follow-up tests are sometimes simple and helps Testers in coming up with more credible and persuasive Bug Reports.

Yahoo Mail, Is there a problem here?

I was logging into my yahoo account, when I realized when I had logged in an hour back the home page was different. I thought, the homepage might have been updated? Or was there a difference the way I accessed the Yahoo! Mail home page the last time?
Then upon little investigation, I realized that the last time I had traversed to the home page was in the following way– I went to www.yahoo.com and clicked on the ‘Mail’ option. The second time I had reached homepage through – www.mail.yahoo.com
This got me interested and I wanted to have a quick comparison of the two homepages which had opened. So for discussion sake, I want to use the following conventions -:
Option 1 -: Go to www.yahoo.com and click on ‘Mail’ option
Option 2 -: Go to www.mail.yahoo.com
[NOTE -: Option1 took me to ‘Yahoo! India’ homepage, while Option2 took me to ‘Yahoo! Mail’ default homepage.]

I did a quick round of comparison and following are my observations -:
(i) Option2 (The default ‘Yahoo! Mail’ home page) had a link to go to ‘Yahoo!’ homepage But in Option1 (‘Yahoo! India’ homepage) I couldn’t find any link to traverse to ‘Yahoo!’ homepage.

(ii) I noticed that the Window Title in Option1 showed “Sign in to Yahoo! India” while the Window Title in Option2 showed “Yahoo! Mail: the best web-based mail”. I wondered why Option1 didn’t have ‘Mail’ in their Window title.

So I opened the Yahoo! UK and Canada sites. Oh, there the Window title was “Sign in to Yahoo!”, Country names were not mentioned (although it is mentioned within the respective homepages). So Yahoo! doesn’t seem to have uniformity in naming the Window titles.

(iii) In Option1 (‘Yahoo! India’ homepage) there is an amazing link “Forget your ID or Password?” . Wow!!, it seems Yahoo might want to help its users to Forget their UserID or Password. What an innovative idea?
I had first found similar message a couple of weeks back when I was testing a website during the Finishing School session that Pradeep was conducting. That site was still to go Live, but it could be that they may have picked up the idea from Yahoo!.

Again I got curious, is this message present only in Option1 or is it also available in Option2 ? Luckily, Option2 had “I can’t access my account” link. But here too, Yahoo! doesn’t seem to have uniformity in naming the links. For same functionality it has got different names.

(iv) I decided to move on and have a peek into the ‘Sign Up’ page for both Option1 and Option2. Hey surprise!! , there is another “Forgot your password or Yahoo! Id?” link when entering Sign Up page through Option1 AND you know what, there is a “Forget your password or Yahoo! Id?” when you navigate to Sign Up page through Option2.

First, why do you need “Forgot you password or Yahoo! Id” within the Registration page? May be because if the user cannot see this link on the homepage, he could dig into the Registration page and find this option. Highly probable!!.

Second, even for people entering Yahoo! site by Option2 have a chance to ‘Forget their password or Yahoo Id’ by going to the Registration page. Good, why should the people using Option1 have all the fun alone?

(v) Finally l had a look at the ‘Security Question’ for both Option1 and Option2 and interestingly there were different set of Security questions asked. Why is it so? Once again I got curious and looked into Yahoo! UK and Canada sites and found that those sites have got exactly the same set of Security questions which is available from Option2 -> ‘Registration Page’. So probably Yahoo! Might have come up with special set of questions for Indian users.
------------------------------------------------------------------------------------------------------

The intention of this post is to demonstrate that if anyone wants to practice testing, he can begin with almost any application he can get. I have nothing personal against Yahoo!, it is just my observations which I have listed here. And maybe, Yahoo! has got its own reasons for implementing things the way they are.

The title of this post is inspired from Ben Simo’s ‘Is There A Problem Here?’ blog.

Testing as a Team Sport

If an entire Product Development is considered to be a Team Sport (say Football), what would be Testing team’s role in it? What are the things that a Testing team can adopt from Football?

Let’s consider the case when a Football Team is playing for a win. There is an entire Team who has the common objective (mission) to win the game. The only way to win the Game is by scoring goals and conceding lesser goals than what you have scored. This doesn't mean that every player in the team is trying to score goals. The Coach draws out a Game-plan to achieve the mission. Players in the team are given certain specific tasks like goal-keeping, defending the goal, controlling mid-field and there would be usually 2 strikers whose responsibility is to score goals. The combined effort of each of these roles is required if a team has to win the game. And there would be times when the coach has to redraw the game-plan according to the situation and find the best tactics to achieve the mission. The success of the Team depends on whether the players in the team know what to do when they get the ball. The task of goal-keeper when he receives the ball would be different from what the defender has to do when he gets the ball or even different from what the striker have to do when he gets the ball.

Similarly in a Product Development the Project Management draws out the game-plan. Then tasks are assigned to the respective Development and Testing teams. The combined mission of Development and Testing team is Successful Release of the Product. Both these teams will have their individual missions too which would ultimately ensure the combined mission is achieved.

The mission of the Testing team is to provide quality related information to the Stake Holders (who would be part of Management) and to facilitate them to take decisions. The Testing effort may still be broken further into sub-groups like UI Testing group, Function testing group, Installation Testing group, Database testing group etc. The Test Plan provides the game-plan as to what all things the Testing team would be responsible for (what all Testing would be done and what are the deliverables), what are the resources available and dates for completion of these tasks are provided. The Test Lead would be the captain who will marshal the resources and the coach would be the Test Manager who would ensure that the mission is on track and take decisions like altering the game-plan etc. The Testers would fill in for the various roles that the Manager provides him based on the Testers skill-sets. The success of Testing efforts depends on whether the Tester knows when they have received the ball, what should they do once they receive the ball. It also depends on how well each Tester has played his role and what information they have provided to the Management. Was the information provided by Testers help the Management to take decisions like - whether the product has matured enough to be released OR would some changes be made to the game-plan and take out the alternate game-plan?

Disclaimer : All contents in this article are my personal views.

Gambler And an Expert Tester

Of late I have been reading a lot of stuff on Expert Testers, articles written by them and have also personally met an Expert Tester. Interacting and reading articles written by them, I feel they have certain traits which best fits a Gambler. In this article I am trying to compare few qualities of a Gambler and Expert Testers, which I personally think these Expert Testers possess and should benefit the Testers who aspire to become an Expert Tester.

#1. The Gambler would always track every single bet he makes by noting it down in his diary. It would usually include date/bets/odds/what's at risk/result and why he made the bet. By tracking his bets he would be able to remain organized, won't fall into traps he had fallen earlier into, would help him to develop a nose for identifying newer traps, would help him to think clearly in uncertain situations and would enable him to identify some sort of patterns admist of chaos.

An Expert Tester would always take down notes of the activities he is performing, the approach he used for identifying a bug in a particular context and the learnings he got while testing a particular product. He would keep on appending his list of learnings all through his testing career. He would be searching for similarities and dis-similarities, different contexts, different heuristics, more oracles and almost anything (things which normal testers are likely to miss) which could help him to achieve his mission.

#2. The Gambler doesn't fear going to the unknown territory and believes he can conquer it with his skill, courage and spirit. He has to train his mind to look beyond the obvious and to look into things which would elude normal people. He has so much at stake (money, reputation, existence ...) that he cannot miss even on minute details.

An Expert Tester challenges the application by stretching its boundaries and testing scenarios which normal Tester would usually miss or can't imagine. He would train his mind at all times to look beyond things which eludes normal testers. He challenges himself to better the benchmarks he sets for himself.

#3. The Gambler has to find his way through the traps, identifying and clearing them to reach his target. He has to make moves under condition of uncertainity. Uncertainity should not be mistaken to randomness. It requires dedicated practice and planning to play moves under uncertain situtation. Even the game of chess is uncertain until one of the Kings is check-mated, but the moves made are not random (infact it depends on the moves you make as per the given context).

Given an application to Test, an Expert Tester first tries to learn the application and establish a pattern for the way application works. He tries to identify the Traps and ways to clear them. Powered with his oracles and heuristics, he tries to find ways to break the application in a carefully planned and systematic manner (and not by Ad-Hoc appraoch).

Disclaimer : All contents in this article are my personal views.