Table of Contents
Introduction
End to end testing refers to the testing of the functional and data flow of an application consisting of several sub-systems working together from its start to end. As we know, the software applications of modern days are sophisticated systems. Usually, they consist of several sub-systems working together each performing a definite task. At times, these systems are developed in different technologies by different teams or organizations. Finally, they come together to form a functional business application. Hence, testing a single system would not suffice. Therefore, end to end testing verifies the application from start to end putting all its components together.
Definition of End to End testing
End to end testing is testing a software application with all its interfacing systems together from its starting point to the endpoint for functional and data integrity. Undoubtedly, the objective of end to end testing is to simulate the production like environment for the system with all interfacing components working together. Clearly, end to end testing happens after the completion of unit, integration and system testing.
Why End to End testing
- The software applications are complicated consisting of several sub-systems. Hence, if any one of the components breaks, the entire application will fail. Therefore, testing them together in real user scenario is important.
- The components of the application can consist of varied technologies. Also, they can be developed by different teams or organizations.
- System and regression testing verify only the system under test. But, they do not evaluate the integration of the system with outside systems. Subsequently, in the real environment, the system has to work together with other systems. Hence, end to end testing validates the flow from the starting point to the ending point.
Difference between End to End and System testing
Often, people get confused between End to end testing and system testing. Now that we already know what end to end testing is, let us consider the system testing. System testing is verifying the system under test as a whole giving relevant input to it and observing the output. Some of the key differences between both the testing are:
| End to End Testing | System Testing |
| This testing tests the application together with all its subsystems, network dependencies, servers, and databases. | This testing tests a particular system as a whole after the integration of all its code modules. |
| It uses Production like real environment for the testing | It uses the QA test environment for the testing |
| Test data is a copy of real production data | The data is any testing data setup on the QA environment for that particular system. |
| Test cases are the scenarios from the perspective of the user | The test scenarios verify the system’s features and functionalities from developers and QA testers perspective |
| End to end testing happens after system testing | System testing happens before the end to end testing and after unit and integration testing |
| The test scenarios cover the application from its starting point to endpoint involving the subsystems. | The tests only cover the system under test ignoring the dependencies on other interfacing systems. |
| It is a manual testing | It can be manual or automation testing. |
Process of End to End Testing
Like any other testing, End to end testing also goes through formal planning, test execution, and closure phases.
End to end testing is done with the following steps:
Planning
- Business and Functional Requirement analysis
- Test plan development
- Test case development
- Production like Environment setup for the testing
- Test data setup
- Decide exit criteria
Pre-requisite
- System Testing should be complete for all the participating systems.
- All subsystems should be combined together to work as a complete application.
- Production like test environment should be ready.
Test Execution
- Execute the test cases
- Register the test results and decide on pass and failure
- Report the Bugs in the bug reporting tool
- Reverify the bug fixes
Test closure
- Test report preparation
- Evaluation of exit criteria
- Test phase closure
Test metrics
The tracing the quality metrics gives insight about the current status of testing. Some of the common metrics of end to end testing are:
- Test case preparation status: Number of test cases ready versus the total number of test cases.
- Weekly Test progress: Number of test cases executed in the week versus a weekly target number of the test cases. Thus, this shows week wise test cases execution progress through the percentage of test cases pass, fail, executed and the remaining number of test cases.
- Defects Status: This metric represents the status of the defects found during testing. Defects should be logged into defect tracking tool and resolved as per their severity and priority. Therefore, the percentage of open and closed defects as per their severity and priority should be calculated to track this metric.
- Test environment availability: This metric tracks the duration of the test environment used for end to end testing versus its scheduled allocation duration.
End to End Testing Framework
While designing an end to end testing framework, we need to consider three important aspects. These are
- User functions
- Conditions
- Test cases
User functions
User functions are the functions or activities a user can perform using the user interface of the application. As the end to end testing consists of real user scenarios, the first step is a list out what all user can do.
- list out all the functions a user can perform
- establish whether or not the functions are reusable
- understand the input and output of the functions
- the flow of data through the related subsystems
- work out the relation between different functions
Conditions
Conditions are the different set of input data and other dependencies like environment settings which can cause the functions to provide different outputs. Thus, the behavior of the function depends on its inputs and other parameters.
- list out the conditions for each user function
- environment settings like type of system, time, date, network, input data, etc all together form different conditions which can affect functions behavior
Test cases
Test cases are the steps of doing the testing. Therefore, here we will make use of the above two steps, user functions, and conditions and come up with several different test cases.
- write test case for every input scenario
- all the behavior of the function should form different test cases
- Test cases should cover all user functions
Example of End to End Testing
Let us take a simple example of an e-commerce web application. This application has:
- a front end for the user interaction
- central server
- a database for inventory
- users database
- seller web application using which sellers can register and upload the items for sale.
- payment gateway with several banks and online payment wallets
Let us come up with an end to end testing framework details for this e-commerce system
User functions
Some of the examples of user functions can be:
- Access the e-commerce web application
- Seller Login
- User login
- Edit profile
- Browse through the items
- Checkout
Conditions
For simple login user function, some of the conditions can be
- Valid user id and password
- Invalid user id and password
- A correct user id and wrong password
- Valid user id and password and wrong captcha
- Forgot password
Similarly, for other user functions like edit profile or checkout, there will be other corresponding conditions.
Test cases
So a few of End to end testing test cases can be:
| Sn No. | Test case details |
| 1 | Seller will create a profile and register a product. Next, User will create a profile and login with correct details. Then, he will search for the product. After this, the user will add the item to the cart and do the checkout with payment. |
| 2 | Seller will create a profile and register a product. User will log in with wrong details and will not be able to log in. |
| 3 | Seller will update the product details. Next, the user will log in and go through the updated product. User will buy an item and make the payment. Then, the user will cancel the order and refund should be processed. |
| 4 | Seller will add the item. User will buy the item. He would initiate a return request. The seller should receive the item back and refund the balance. The refund should get processed. |
Conclusion
With new age software applications becoming more and more complex, thorough testing including all interconnected systems is necessary. Although the system under test can be working perfectly fine, other dependent systems may breakdown in a certain scenario to cause a failure. Hence, end to end testing comes to rescue verifying the entire application with all its interfacing components like other subsystems, network and firewall, servers, and databases.