What is API Testing?
An API (Application Programming Interface) is a collection of software functions and procedures, called API calls, which can be executed by other software applications. API testing is mostly used for the system which has collection of API that needs to be tested. The system could be system software, application software or libraries.
API testing is different from Unit, white box and UI testing, UI is rarely involved in API Testing. Tester need to setup initial environment, invoke API with required set of parameters and then finally analyze the result.
Initial environment means test environment setup and application setup, database creation, server configuration, config and properties file setup and deployment of application or any coding (if it is required).
Usually people or company synonym as Unit or white box testing but there is huge difference between API, Unit and while box testing. Tester need to or may need to do coding during API testing.
Difference between Api Testing and Unit Testing
Unit testing is an activity that is owned by the development team; developers are expected to build unit tests for each of their code modules (these are typically code may or may not contains classes, functions, stored procedures, or some other ‘atomic’ unit of code), and to ensure that each module passes its unit tests before the code is included in a build.
Unit tests are typically designed by the developers to verify the functionality of each unit. The scope of unit testing often does not consider the system-level interactions of the various units; the developers simply verify that each unit in isolation performs as it should.
API testing is typically an activity owned by the QA team, API tests are often run after the build has been created, and it is common that the authors of the tests do not have access to the source code; they are essentially creating black box tests against an API rather than the traditional GUI.
In API testing, QA team must consider the ‘full’ functionality of the system, as it will be used by the end user. This means that API tests must be far more extensive than unit tests, and take into consideration the sorts of ’scenarios’ that the API will be used for, which typically involve interactions between several different modules within the application.
API is mostly black box testing where as unit testing is essentially kind of white box testing. Unit test cases are typically designed by the developers and there scope is limited to the unit under test. In API testing, test cases are designed by the QE team and there scope is not limited to any specific unit, but it normally covers complete system.
So a tester should able to identify before start API testing…
*Find the way to approach the task?
*Do boundary analysis.
*Create or focus on the most likely usage scenarios (Functional Scenario).
* Check Return Values
*Focus also on negative testing to get exception and error handling.
*Check Event triggers (Optional and depends on API Type)
*Modify resources (Optional and depends on API Type)
Type of API and how to approach them:
API can be called directly or it can be called because of some event or in response of some exception. Output of API could be some data or status or it can just wait for some other call to complete in a-synchronized environment.
A. IF API return value based on input condition:
- In this condition, test cases will be based on the input and relative output.
- This is relatively simple to test as input can be defined and results can be validated against expected return value.
- User can pass different combinations of values or parameters and validate these against known results.
B. If API does not return anything:
- In this situation tester should need to identify some mechanism to check behavior of API on the system.
- For example, if you need to write test cases for delete (List Element) function you will probably validate size of the list, absence of list element in the list.
C. If API Trigger some other API/event/interrupt:
- If API is triggering some event or raising some interrupt, then you need to listen for those events and interrupt listener.
- Test suite should call appropriate API and asserts should be on the interrupts and listener.
D. If API is used to Update data structure:
- Updating data structure will have some effect on the system and that should be validated.
- If you have other means of accessing the data structure, it should be used to validate that data structure is updated.
E. If API is Modifying certain resources:
- If API call is modifying some resources, for example updating some database, changing registry, killing some process etc, then it should be validated by accessing those resources.
Challenges of API Testing:
a. Parameter Selection:
Ensuring that the test harness varies parameters of the API calls in ways that verify functionality and expose failures. This includes assigning common parameter values as well as exploring boundary conditions.
b. Parameter combination:
Generating interesting parameter value combinations for calls with two or more parameters.
c. Setting environment:
Determining the content under which an API call is made. This might include setting external environment conditions (files, peripheral devices, and so forth) and also internal stored data that affect the API.
d. Call sequencing:
Sequencing API calls to vary the order in which the functionality is exercised and to make the API produce useful results from successive calls.
Most common seniors in API Testing:
a. Test Response: Each API method in isolation using only the mandatory elements and typical content so if system calls any API then it send any response so tester need to check valid response from API.
b. Test Limit: These tests exercise each API method using all optional elements and maximum allowable content lengths and/or instances of repeated elements.
c. Test Business Logic: This is where the business application logic is simulated in the test code. Each API method will have a defined set of test cases that explore its interaction and influence on other API methods and any underlying database.
d. Test Negative or Illegal: These tests contain a sampling of typical error scenarios, such as missing required elements, empty content, and content exceeding maximum limits, across a representative sampling of the API methods, so API should enough intelligent to handle exception, errors and missing parameters issues.
e. Test Load: Most of applications are web 2.0 and running on internet and access by number of users so API should able to take heavy load and should not break during peak time when number of people will use application, tester also measure response time, through put , latency, memory leak and any other factor according to requirement
If you were to ask testers how to test API, you would get several different perspectives. Every people have different thinking and approaches to solve problems and every approach have option for enhancements.