Table of Contents
Introduction
Sanity testing is the testing which verifies that the code changes in the build are working as per the expectations. Therefore, it consists of a non-exhaustive set of tests which validates that the build or the patch does what it is supposed to do. For instance, the build fixes the bug or the new functions are present in the build. Hence, the testing team uses Sanity testing as a checkpoint to decide whether or not to go ahead with exhaustive testings. For this reason, sanity testing consists of the test cases which will verify that the new code changes in the build are working as per the expectations.
Sanity testing example
Let us consider one example of Sanity testing. For instance, the typical Sanity test cases of an email application in which a new feature search email is added in the build will look like this:
- Login with valid credentials and load the email. Search email icon should be visible on the Home page.
- Enter the text in the search box. User should be able to edit, copy and paste in the search box.
- Enter the valid search text in the search box and click on the button. It should load the emails which consist of the search text.
Sanity testing consists of test cases to verify the new code changes. But it is not an is important testing which helps in the decision-making for accepting or rejecting the build.
Why Sanity testing
When there are any bugs to be fixed or enhancements are added to the product, builds are released to QA team for testing. Consequently, the QA team does exhaustive testing like regression, performance, and acceptance before it is released to the customers. This entire testing takes a lot of time and effort to complete. But, if the build does not do what it is expected to do, it is not wise for the QA team to invest so much time in testing it exhaustively.
In other words, if the new changes in the product are not working properly, it should not be considered for detailed testing. Thus, the QA team uses Sanity testing as a parameter to decide if they can accept the build for further testing. In case, the test cases of Sanity test fail, there is no benefit in going ahead with detailed testing phases. Thus, Sanity testing saves the important time of testing team by returning the faulty build which does not serve its purpose.
Advantages of Sanity testing
- Sanity testing helps in saving the time of QA team by rejecting the build with faulty code changes in the initial stage itself.
- It verifies only the new changes in the product. So they are similar to unit tests written for that change. Thus unit tests of the functionality can be used as a base to write Sanity tests at the system level.
- Sanity tests can be integrated with regression tests for further releases.
- In case of urgent builds, it gives a boost to the confidence of the user by ensuring that the changes are working fine in the initial evaluation.
- It is quick to perform but serves as a milestone in the build lifecycle.
Sanity Testing Steps
Let us look at the typical life cycle of Sanity testing for any project.
Planning
The QA team writes Sanity test cases. At times, it can also work in collaboration with other stakeholders like project manager, BAs, and product users as well to develop the test plan and scenarios. Every build has its own new scenarios using which it can be tested. The typical planning of Sanity tests looks like:
- New functionality analysis for validation during Sanity test.
- Sanity testing test plan development
- Test case development
- Sanity testing environment setup for the testing
- Test data setup
- Decide exit criteria
Pre-requisite
- Sanity test plan having test cases
- Test environment and Test data
- Build with the changes
Test Environment
- QA or Dev environment
Test Execution
- Execute the test cases
- Register the test results and decide on pass and failure
Test closure
- Evaluate the test results
- Decide to return the build or to go ahead with the further testing
When to do Sanity testing
The simple purpose of Sanity testing is to decide whether or not to go ahead with exhaustive testing for the build or the patch. Sanity testing is a good fit for the following scenarios :
- When the development team releases a new patch or the build with the bug fixes or new functionality.
- At times, when there are many build release frequently, the QA team does only Sanity Testing for each. Then the builds are combined and regression testing is done for all the builds together.
- When the project is following the Agile model by enhancing the product with every sprint.
Challenges of Sanity testing
- Sanity testing consists of very limited test scenarios. Thus, it can miss the critical bugs depending on its test cases coverage. Hence, post Sanity testing the detailed regression or functionality testing is required.
- QA team should have good knowledge of the new functionality or bug fix to Sanity test it.
Confusion between Sanity and Smoke testing
Many people confuse Sanity testing with Smoke testing but both are a different type of testing serving different objectives. Sanity testing validates if the new changes in the product are working fine by running minimum required test cases covering the new functionality. The purpose of Sanity testing is to ensure that the changes expected from the build are in place and are working fine. Thus, the QA team rejects the build or the changes, if the expected modification from the build into the product is not proper.
Whereas, Smoke testing validates if the crucial functionalities of the product work fine with the build. It also verifies if the build installs properly and look and feel of the product are as per the product requirements.
Difference between Sanity and Smoke Testing
| Sanity Testing | Smoke Testing |
| Verifies the new functionality or the bugs in the build. | Verifies the existing functionality is working fine with the build. |
| Sanity tests can become part of regression tests for this and upcoming builds. | Smoke tests are taken from regression or functionality or acceptance tests of previous builds and releases. |
| It only tests one area of the product | It covers the product end to end |
| Automation usually happens when tests become part of the regression test suite. | It is a good candidate for automation because of repetitive nature. |
| Different tests for every build. | Same tests for every build. |
Please read about smoke testing in detail here.
However, the QA team performs both Sanity and Sanity only to decide whether or not to go ahead with detailed testing.
Conclusion
QA team uses Sanity testing as a decision criterion for accepting or rejecting the build. Although, it should be quick enough to complete yet should consist of test scenarios to validate the new functionality or bug fix of the build. To summarise, Sanity testing should be capable enough to ensure that the QA team can continue with the further exhaustive testings like regression or performance with this build. Therefore, it not only works to save the time of the team but also if it goes well, increases the confidence of the stakeholders in the changes.