Testing Lessons from - The Coconut Exercise

This post is created by spying on a Testing exercise that Pradeep was doing with Santhosh. I didn’t carry my moleskine diary at that time so I will to put down all I could remember. These are the benefits which one can get when you are with Pradeep, you will find unexpected ways of learning things. So I am sharing my experience for all others who were not there.

So the context was, we were near the chatting zone in Edista office. Pradeep asked Santhosh while pointing at a small coconut which was lying on the floor to come up with ideas of how that coconut might have got there. Santhosh began putting forward his ideas by looking at the coconut tree which was behind us, and said it might have fallen from the tree there. To this Pradeep asked, the place where the coconut is lying may suggest that the coconut may not have fallen directly to this place. To this Santhosh modified his idea by suggesting that it may have fallen here by hitting the parapet on its way. Then he came up with ideas like it may not actually be a coconut although it seems to be. Then it might have been blown here by the wind. Next idea was, may be some birds might have brought it here. Then Santhosh gave up.

Now Pradeep took up the challenge and showed how he would have approached this problem. He started by saying that it might would had fallen somewhere else and the guy who might have come for a walk may have actually kicked the coconut here. Then he said may be this coconut may not have fallen from the tree we are looking at and may be from other tree which was bit far and some squirrel or bird might have brought it here. Next idea was he went and picked up that coconut which was lying there and upon looking at it, he found some marks which indicated that it was eaten from one side. This gave more validity to the idea that it would have been brought there by squirrel or a bird. Then the idea that it may not be a coconut was also validated. Then other ideas were like, may be some coconut vendor might had come there and would have thrown it over there, then may be ideas like ones from Tamil movies of this coconut falling on the adjoining building and would had traversed different paths by hitting various parapets and may have landed to the place where we are seeing it etc.

This exercise gives us a learning that if we are conscious of our surroundings and environment then lots of things can be learnt. Such exercises would improve our observation skills, logical skills, interpretation of various facts and lots of other stuff which can help a Tester in his Testing activities. If you come across such exercises make sure to collect those and discuss and play it with your friends, it can be a lot of fun as well as learning.

Happy Learning,

- Manoj

Testing Tips 2 -: When you find an Error in your application, do some follow up Testing

One of the things which most of the Testers (me inclusive, many a times I have also not realized the importance of doing follow-up testing) don’t do when they get an error is - to do follow up testing. This mostly reduces the value of the Bug that you are reporting. The reason is you still have not identified the Bug in its worst form or not investigated the potential of that Bug. This would sometimes make the developers to defer the bug you have reported and the Bug may take its worst form in the field.

If a Tester wants the Bug reported by him to be fixed by the developer , then ensuring that he has investigated the Bug and reporting that Bug in its worst form, goes a long way in getting this done.

I was testing a text field in a web application which was not supposed to accept more than 1000 characters. I generated 1010 chars using PerlClip and pasted on this field. It just accepted 1000 characters and rejected the remaining 10 characters. I again tried appending more characters to the existing 1000 characters by trying to paste more characters, but it was also rejected. So I tried appending by giving keyboard input beyond 1000 characters and to my surprise the characters were getting appended. At this point, I could had logged a Bug saying that I am able to input more than 1000 characters to the text field which is an issue as per requirements. But I decided to do some follow up tests and tried to input as many characters as possible. I found that the field was allowing only 1000 characters to be pasted at a time, but if you give some keyboard input then again you can paste 1000 characters. So using this method I gave inputs of around 3 lakh characters. Then when I did further operation with the application where this data in the field had to be retained across couple of pages, the application initially slowed down when passing this data to the first page and then by the second page the application hung for around for 10mins before recovering.

Here if I had reported the Bug for the first error that I encountered i.e. “the text field accepting more than 1000 characters as input”, the severity of this Bug could not have been captured. But my follow-up test results, where I was able to input around 3 lakh characters to the text field and upon doing further operations the web application getting hanged showed the Bug in a more severe form. So the Bug report for the second case is more likely to catch the attention of the developer and has a better probability of getting fixed. The follow-up tests are sometimes simple and helps Testers in coming up with more credible and persuasive Bug Reports.

Testing Tips 1 – Bug Reporting

Bug Reports are one of the most important work products, produced by a Tester. Many a times the client, management, developers and test team members judge a Tester’s skill from the Bug Reports he creates. So Bug Report is a very important document which affects the credibility of a Tester.

Here I would like to list down some of the vocabulary given/described by Cem Kaner, which would help Testers in writing better Bug Reports.

Error: An error (or fault) is a design flaw or deviation from a desired or intended state. For eg. -: This could be a Coding mistake or a missing code.

Symptom: This might be a characteristic of a failure which helps a Tester in recognizing that the program has failed. For example -: Your application slows down when a user performs certain operations, is a symptom.

Failure: The failure is the program’s actual incorrect or missing behavior under the error triggering conditions.

Critical Condition: The condition which would trigger a failure.

It would be useful for a Tester to distinguish between an Error, critical condition and failure while writing their bug reports. I would like to take an example from the Testing activity that I was doing to drive in these points. I was testing a Username and Password field of a Sign-in page of a website. It worked fine when I provided a correct username and password and I was able to login. Then I thought of checking the maximum allowed characters in each of these fields. I found that the maximum field length of both the Username and Password fields were not set. I was able to provide more than 3 lakh characters in each field and when I clicked on 'Go', it was trying to send these huge strings to the server and the Sign-in page hanged and did not respond for more than 25 mins and eventually I had to forcefully close the browser.

In the above scenario, the “missing code for setting the maximum field length of the Username and Password field’s” is an Error. The Critical Condition (or failure triggering condition) is the user giving more than 3 lakh characters as input in both Username and Password field’s and clicking on 'Go' button. And the Failure was the Sign-in page hanged for the Critical condition.

Test Automation Tools Vs Human Testers

One of the ideas I totally disagree with people who advocate Test Automation is their claims of Automation tools being able to run tests much faster as compared to Human Testers. But my question is why do you want to compare an Automation Tool to a Human Tester? How did this comparison even arise?

According to me, a Tool can be some device which may be used to facilitate or aid in performing manual tasks. So, why should speed of Human is compared to speed of an aid he uses?

I remember using Scientific Calculators in my Engineering Degree. The calculator was used as a tool aiding in calculations. Few formulae could also be fed or stored into it. But, I have never heard anybody saying that the calculators should be replacing the engineers as it is able to execute the calculations much faster than humans. But what good is a calculator without humans? It cannot perform any task on its own, it cannot have logic, it cannot think. But it is very useful if an Engineer who has a calculator knows what benefits he can derive from it and knows when, how and where to use it and get maximum benefits from it.

We should understand that the purpose of a Calculator or a Test Automation tool is to aid humans in doing their tasks in a better way. This would allow humans to spend their time more judiciously on more important things in what he is good at and allowing the tools to do tasks for which they were specifically made for. Even the Test Automation is a heavily human intensive task. So a Test Automation tool can execute tests much faster than humans, but it cannot do anything by itself. The Test Design has to be done by humans, identifying tests for automation, creating automation scripts, maintaining automation scripts, identifying the comparison criteria (pass/fail criteria) and investigating the issues if any mis-match is found in the Test execution results, all these activities has to be done by human testers.

So, next time when you hear someone speaking about Test Automation tools being able to execute tests faster than Human Testers, make him understand why he should not be doing and talking about such comparisons.

Why Indian Testers need to Wake Up, Come Out and Interact?

I mentioned in my previous post about some of the learnings and experiences I had during my BBST course. I was interacting with two of my friends (and fellow Testers) from my BBST batch, ‘Juha Johansson’ and ‘Charles Penn’. I learnt from ‘Juha’ that he is in touch with some of the Testing professionals with mutual interests in Houston and they are starting Software Testers Group there. ‘Juha’ had also registered for ‘CAST 2009’ which began on 13th July in Colorado. Meanwhile, ‘Charles’ is one of the organizers of IWST (Indianapolis Workshop on Software Testing) which conducts regular Testers meets and discussions etc.

I was wondering, How many Software Testing Groups that we have heard of, are in India? Even though, India is supposed to be a Software Testing Hub. How many Indian Software Testing Groups are here, which are active or doing some significant contribution towards the Software Testing community. Hardly any…

But you might find many Indian Testers being part of Testing Communities outside India. I am not trying to say it is bad, don’t misunderstand me. The point I want to make here is, when there are so many Indian Testers who are part of many good Testing Communities hosted abroad and they have interaction with many Good Testers across the world, then why are we unable to come up with Testing Groups of our own. Why can’t we have the Groups where we can share the learnings, project experiences, hold workshops, come up with new ideas, experiment with new and existing ideas, talk about innovations, debate, argue etc. Why can’t we come up with Testing Conferences like ‘CAST’ of our own? When will we have the Software Testing Experts whom the entire world recognizes? When will we have our own Jerry Weinberg, Cem Kaner, James Bach and Michael Bolton?

I have seen and learnt about some Software Testing Groups within companies, but they were hardly active or doing anything noteworthy. To begin with, we should atleast try to make use of such groups which are already in place and make them functional. If we (Indian Testers) are only concerned with what we are doing in our company and think only about ourselves and our survival, And are not actually coming out and interacting with other Testers, then we are seriously missing out on an opportunity of mutual learning and improvement.

There are some initiatives which are being taken in this direction though, and one such initiative was Bangalore Workshop for Software Testers – 1 which was held in May this year. But what we really need is - to have more and more Software Testers getting together and interacting frequently, and more such events/workshops happening in India.

It is high time that we (Indian Testers) wake up to challenges like this, this, this, this and this, which is staring in our face and begging us to come out of the comfort zone and show to the world that When India is getting Software Testing projects – it is not just because of the cost/currency advantage.

A Testing Course with a Difference – “BBST Foundations”

I am blogging after some time gap. One of the things that I did during this break was to do BBST Foundation course. Here I would like to share my experience of this 1 month long course and why I think it would be useful for Indian Software Testing community.

BBST foundation course is a basic introduction to ‘Black Box’ Testing. You can read the details about this course here. This course includes video lectures, quizzes, homework of various kinds, and a final exam. The course is provided Free of Cost for the AST members. So this is a course which is non-commercial in nature and is dedicated for the betterment of the Testing community.

Ideally one should have some Industry experience for taking up this course although it is not mandated. But I would say, for reaping better benefits from the course a Software Tester should have atleast 2-3 years of experience in IT industry.

There were 11 participants in my batch mostly from US, although there were participants from other countries as well. Apart from me, there was one more Indian (Rajkumar Pandian from Chennai) in my batch. The batch began with ‘Meet and Greet’ session where all the Testers had to introduce oneself to the batch. It was really cool and gave all Testers a platform to share their interests and make friends with like-minded testers and other testers in the group. There were Testers with wide range of experience, right from beginner (1 tester) to Testers with 5-7 years experience and also people who have chosen Testing after having experience in different fields. So this a playground where Testers can put forward their point of view, debate, learn from other Testers, learn how Testers work in different geographies, how the challenges differ and their experiences. This is place where your work would be challenged and put under scrutiny by Testers with different set of ideas and expectations, and you will have to defend your work. Your work would also be evaluated by some of the Experts in the Software Testing field who have got decades of experience.

It is really great Fun!! and lots of learning too. I am not sure whether Testers who had appeared for CSTE or ISTQB or other similar exams would have experienced anything close to this. This course had got many ingredients where a Tester’s skills are really put to test and clearing this exam would build a lot of “Credibility” for the Tester within the Testing community.

Having gone through the grind and fun for 1 month of BBST course, I believe this course deserves more recognition from the Indian IT Industry. This course would give the IT industry a better insight about their Testers ability and skills than any CSTE or ISTQB exam can actually give. But for this to happen, I believe more Testers from India will have to take-up this course and share their experience with the Testing community.