How to be a badass ninja QA tester

Quality Assurance is more than just finding bugs in an application. The QA team focuses on delivering a quality product that is developed by a developer,within a deadline. The QA team’s primary task is to eliminate the issues that may affect the way the product works and thereby hampering the user experience.

When we approach testing in a project, irrespective of how small or big a project is, we always strive to achieve the testing pyramid. If you have never come across this model then go and look it up, it’s on the list of being a badass!

“Quality is not an act, it is a habit.”— Aristotle

Here is the pyramid,

Why Quality Assurance (QA) is a must in software development?

The longer a bug goes undetected, more expensive it is to fix. A simple cost vs. benefits analysis overwhelmingly shows that the benefits of employing a QA test engineer to validate the code far outweighs the costs.Most importantly, it also influences your product’s reputation!

Here is why the QA process is important in software development:

An extensive QA process is performed to eliminate avoidable defects or bugs before a site is made live.
QA is done to make the website credible and easy to operate.
What if the search button of a search engine like Google, which is used by millions of people every single day, doesn’t work? It might require a simple fix, which can be done by a developer in a jiffy. However,this defect will encourage users to use a different search engine,which leads to a loss in users
A proper QA process will help you find defects that can be fixed before the website goes live.
When a product is launched, it is a must for the product to have undergone the complete QA process. What if the product is launched and the users find that something is not working as expected? The company will lose its credibility, reputation, and resolving the issue will become expensive and time-consuming
The QA process is not just for delivering stable products there is a higher purpose. The purpose is to make users and customers believe that the company is credible, retain the number of users, provide the users a great experience.

If you are still skeptical, look at these stats!

In April 26 1994, China Airlines Airbus A300 crashed due to a software bug killing 264 passengers.
In April 1999, a software bug caused the failure of a $1.2 billion military satellite launch, one of the most expensive accidents in history.
In May 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank to be credited with 920 million US dollars.

The QA process can be a lifesaver sometimes, can’t it?

How is the QA process performed?

Method

Courtesy:Jack Sheppard

The QA should be aware of the stack, the frameworks, the business purpose of the feature, and most importantly understand the customer’s and user’s pain! This is where the QA process starts and this is where you should begin!

We believe that the QA then enters into the Design Phase and starts its transformation from then on. Having a pre-design QA assessment is even better. Once the company decides to build a product, the PM schedules a meeting with the developers, QA engineers, and designers.

During this meeting, the PM explains the purpose, need, and user requirements that will be used when the product is built.This clarifies information about design, engineering, stack, and other engineering requirements. During this time the QA engineer will understand and clarify information about the origination of the requirement.From there on, QA becomes an integral part of every process till and after the delivery of the product

The following processes must be followed for an effective QA process:

Design QA

Once the design is completed, there should obviously be the obvious design QA. The QA engineer has a discussion with all the members of the software development team.

In this informal discussion table,the features in the product would be explained to the entire team,which they will soon be starting to develop.Then we start the design QA, in which the QA team will go through the entire design to check about functionality, feature, enhancements, user requirements, business purpose validation, potential issues, foresee complexities that may exist and are briefly made aware on the drawing board to the devs during the design QA process.

Create the test scenario document

After the design QA is done, the QA team starts designing a test scenario document (sample template), which is a hybrid of use case and test case documents.

This document will be used throughout the testing phase. It contains a list of all the possible scenarios that are identified for testing in the product based on the design. Once the testing begins, the scenarios will be iteratively added during the Executing phase.

Documentation review

Documentation review is a critical step in the QA process.This review decides the direction of the testing process and direction is very important.

This review is done by developers or the PM before the execution phase starts. In this step, either the developer or the PM goes through the test scenario document that was created by the QA team and checks whether all the scenarios have been covered.

If a scenario is missing, it is the shared responsibility of the PM, developers, and QA engineer to ensure that it is added. It is recommended that you do not start testing until all the scenarios have been added to the document.

Execution

Execution is the phase where the real testing happens. The testing process is started when 75% of the product has been developed thus avoiding rework by the time the development reaches 95%.

Rework is mitigated by the test scenario document which was created during the test scenario phase. If you take up QA earlier, then you will not have the appropriate QA and test coverage. If you do QA after the product or feature has been fully developed, then you will have to deal with a lot of demotivation among the developers due to rework.

You must test all the scenarios which are covered in the test scenario document along with the scenarios which you come across while testing the actual product. Add the new scenarios to the document while testing.

The execution phase takes its own time.There will not be any compromise on the time for QA. The results of the execution will be noted in the test scenario document and the same will be shared with the developers who have developed the product.

During the execution phase, not only the scenarios are just covered but the user experience would be tested too.The way the product behaves,all the functionalities,features,UI would also be tested.If a scenario fails.We would add the explanation of it along with the screenshot,URL,steps to reproduce.

Defect reporting

As Joel says in his blog,

“A great tester gives programmers immediate feedback on what they did right and what they did wrong. Believe it or not, one of the most valuable features of a tester is providing positive reinforcement. There is no better way to improve a programmer’s morale, happiness, and subjective sense of well-being than a La Marzocco Linea espresso machine to have dedicated testers who get frequent releases from the developers, try them out, and give negative and positive feedback. Otherwise it’s depressing to be a programmer.”

During a sprint ,the defects that are found should be noted in the a defect summary sheet (sample defect summary) of a test scenario document (sample template).This helps the developer to view the defect along with the explanation, screenshot, URL, and steps to reproduce the defect.

The developer then fixes the defects. If a defect is not valid, then it will be classified as one of the following:

Feature
Bug that cannot be fixed for various reasons or requires a design change.

Otherwise, the defect is marked as fixed in the relevant column of the test scenario document and also updates the Maniphest log. The defects can be related to the functionality, features, UI, or anything that affects the way the product works.

Maniphest

Maniphest is one of the defect-management tools that is used during the QA process. It helps to manage the entire bug life-cycle. As tests are executed, you may find bugs in the existing flow or may feel there is scope for a few enhancements. You should immediately, create a task in Maniphest and assign it to the relevant developer.

Based on the priority, bugs will be fixed. Once the bug is fixed, the author of the bug i.e. the relevant QA engineer will receive an email notification.This helps the QA engineer to retest the bug and change the status accordingly.

Retesting

Once the issue is marked as fixed by the developer in the test scenario document, the retesting of the specific defect is the responsibility of the QA engineer who reported the issue. The results of the retesting should also be recorded in the test scenario document.

The Testing phase ends with this step. Retesting will be done at the end of the Execution phase because execution is usually done on a fully developed product—the most stable version of the product.

Test closure

This is the step where the QA team prepares the release notes of the testing process.

Release notes contain the following information:

Description of the product that was tested
Time taken
Approach followed
Reference links
Test results
Type of testing that was done

After the release notes are sent to the whole team, the QA process ends.

Types of testing at HackerEarth

Automation testing

While there is plenty of room for improving the QA process at HackerEarth, we are now trying to put an emphasis on building automated tests so that we can let people do what people are good at and have computers do what computers are good at. That doesn’t mean that we never do manual testing or drop out of the pyramid. Instead we do the “right” amount of manual testing with more human-oriented focus (e.g. exploratory testing) and try to ensure that we never do repetitive manual testing.

Performance testing

Performance testing is an important type of testing which is a must before deploying any new change. Performance testing is performed to determine the behavior of a system under both normal and expected peak-load conditions. It helps to identify the maximum operating capacity of an application.We use New Relic for the performance testing.

“An application can work fine for a single user but may break when multiple users use it simultaneously.”

We use JMeter to perform load testing.

JMeter creates realistic & accurate scenarios, sends requests to appropriate servers which show the performance of an appropriate server/application via tables, graphs etc.

Environments used for testing

We have an environment that is a replica of the production environment. It is called the ‘staging environment’.We use this staging environment for the testing process.

The developers develop an application in their local environment and push it to staging where the QA engineer will test the application. All the testing takes place in the staging environment.

We would never have been able to get this far and achieve an effective QA process without a dedicated grassroots effort from everyone in the team. This effort would have failed if it hadn’t been combined with huge improvements in our testing tool, processes, and mind shift along with the business and developers.

“QA is hard!! If it was easy, anyone could have done it. The ‘hard’ part is what makes QA important!”

We have taken a serious and pragmatic approach to establish a definite QA process. The important thing about this process is that QA at HackerEarth has evolved into a multi-dimensional process. We have formulated this QA process collaboratively and improved it over time.