Table of Contents
Introduction
Smoke testing is the set of non-exhaustive tests which ascertain that the critical functionality of the product works fine with the new build. In initial stages of the product lifecycle, builds are unstable. Thus, the testing team uses smoke testing as a checkpoint to decide whether or not to go ahead with exhaustive testings like regression or functional with the new build. Smoke testing consists of test cases which will just verify the main functionalities of the product.
Smoke testing example
For instance, the typical Smoke test cases of an email application will look like this:
- Verify the login page loads when URL is given in the browser.
- User is able to successfully login with valid credentials.
- Email home page is loaded.
- Look and feel of the Home page is as expected.
- User is able to log out of the email application successfully.
Smoke testing is also known as build verification testing or confidence testing.
Why Smoke testing
In the initial phase of the project, many cycles of builds are released. As the team and the project is new, the builds can be unstable. In other words, the new changes in the product can break the important functionalities of the product. Thus, the QA team uses Smoke testing as a parameter to decide if they can accept the build for further testing. In case, the test cases of Smoke test fail, there is no benefit in going ahead with detailed testing phases. Thus, Smoke testing saves the important time of testing team by returning the badly broken build then itself.
Advantages of Smoke testing
- Smoke testing helps in saving the time of QA team by rejecting badly broken build in the initial stage itself.
- Smoke testing is repetitive testing repeating the same tests for every build. Hence, it is a very good candidate for automation.
- With automated Smoke testing, it takes very little time and effort to validate the build.
- In case of urgent builds, it gives a boost to the confidence to the user ascertaining that at least the important functions are working fine.
Smoke testing steps
Let us look at the typical life cycle of Smoke testing for any project.
Planning
The QA team in collaboration with other stakeholders like project manager, BAs and product users do planning for the Smoke testing once with the start of the project. Then, only minor changes like the addition of new tests or removal of the tests are done depending on the new learnings of the project. The typical planning of Smoke tests looks like:
- Important functionality analysis for validation during Smoke test.
- Smoke testing test plan development
- Test case development
- Smoke testing environment setup for the testing
- Test data setup
- Decide exit criteria
Pre-requisite
- Smoke test plan having test cases
- Test environment and Test data
Test Environment
- QA or Dev environment
Test Execution
- Execute the test cases
- Register the test results and decide on pass and failure
Test closure
- Evaluate the test results
- Decide to return the build or to go ahead with further testing
When to do Smoke testing
The simple purpose of Smoke testing is to decide whether or not to go ahead with exhaustive testing for the build or the patch. Sanity testing is a good fit for the following scenarios :
- When the development team releases frequent patches or builds usually in the initial stage of product development.
- A software product goes through changes in the form of new enhancements or bug fixes.
- When the project is following the Agile model by enhancing the product with every sprint.
Challenges of Smoke testing
- Smoke testing consists of very limited test scenarios. Thus, it can miss the critical bugs depending on its test cases coverage. Hence, Smoke testing can still qualify the bad builds for further testing.
- Every build undergoes the same set of test scenarios in Sanity testing. Thus, Sanity testing is repetitive testing for every build. But, the repetitive scenarios may stop finding any new bugs with time.
Confusion between Sanity and Smoke testing
Many people confuse Smoke testing with Sanity testing but both are a different type of testing serving different objectives. Sanity testing validates if the new changes in the product are working fine by running minimum required test cases covering the new functionality. The purpose of Sanity testing is to ensure that the changes expected from the build are in place and are working fine. Thus, the QA team rejects the build or the changes, if the expected modification from the build into the product is not proper.
Whereas, Smoke testing validates if the crucial functionalities of the product work fine with the build. It also verifies if the build installs properly and look and feel of the product are as per the requirements.
Please read the difference between Smoke and Sanity testing here.
However, the QA team performs both Sanity and Smoke only to decide whether or not to go ahead with detailed testing.
Conclusion
QA team uses Smoke testing as a decision criteria for accepting or rejecting the build. Although, it should be quick enough to complete yet should consist of test scenarios crucial to validate the main functionality of the product. To summarise, Smoke testing should be capable of ascertaining that whether or not the QA team can continue with the further exhaustive testings like regression or performance with this build. Therefore, it not only works to save the time of the team but also if it goes well, increases the confidence of the stakeholders in the changes.