Skip to main content

Testing guidance for consuming and publishing APIs

 

API testing is a crucial part of the development and integration process, ensuring that APIs function correctly, securely, and efficiently.

This guide is for both API providers and API consumers, outlining the expected use cases, testing requirements, and documentation standards needed for successful API onboarding. 

For API publishers the focus is on ensuring that the APIs they develop are robust, secure, and performant across all possible use cases. This includes testing all endpoints, verifying data integrity, handling errors gracefully, and adhering to industry-standard security protocols. 

For API consumers the testing process is more targeted, focusing on the specific API endpoints and functionalities their applications will use. Consumers must ensure that their applications can interact with the API as expected, handle expected/unexpected API responses correctly, and manage any errors or security concerns that may arise. 

 

 

 

Functional use cases 

Purpose: Verify that the API performs its intended functions correctly. 

For API publishers: 

  • Basic CRUD operations: Ensure that Create, Read, Update, and Delete operations (where applicable) across all endpoints are functional, return the expected results, and have sufficient boundaries in place to protect the underlying integrity of the data. 
  • Endpoint/specification verification: Verify that each API endpoint behaves according to its specification, returning correct responses for a variety of valid and invalid inputs. 
  • Data validation: Validate that all API responses contain accurate and valid data, consistent with the API's documented design and specification.

For API consumers: 

  • Basic operations: Confirm that the API endpoints being accessed perform the necessary Create, Read, Update, and Delete operations required by the application. 
  • Endpoint interaction: Test the interaction of the consuming application with the specific API endpoints it will utilize. Ensure these endpoints return the expected results when called by the consuming application. 
  • Data usage: Verify that the data returned by the API is correctly processed and displayed within the consuming application as per the business requirements. 

Note: If any data is being sent to the API then you will need to work with the provider to ensure that this is formatted correctly, with all the expected fields, as consistent with all the use cases you are making available within your application. 

 

Performance use cases 

Purpose: Assess the API’s performance under various conditions. 

For API publishers: 

  • Response time: Measure and optimize the API's response times under normal and peak loads to ensure they meet predefined performance standards. 
  • Load handling: Conduct extensive load testing to validate that the API maintains performance and stability under high traffic conditions, such as multiple concurrent requests. 
  • Stress testing: Execute stress tests to determine how the API performs under extreme conditions, ensuring it can handle unexpected spikes in usage and recover gracefully. 

For API consumers:

  • Application responsiveness: Ensure that the API's performance (response time) meets the application's requirements for delivering a seamless user experience, especially under typical usage scenarios. 
  • Load impact: Assess how the API's performance under load conditions impacts the consuming application, especially if the application depends on real-time or near-real-time data. 
  • Error handling: Validate that the consuming application handles any performance-related issues/errors from the API gracefully (such as response delays/timeouts, token expiry, quota/spike arrest warnings). 

 

Security use cases 

Purpose: Ensure the API is secure and adheres to security best practices. 

For API publishers: 

  • Authentication and authorisation: Verify that authentication mechanisms (e.g. OAuth, API keys) work as intended, ensuring that only authorized users can access the API. 
  • Data encryption: Ensure that data in transit and at rest is encrypted to protect against unauthorized access or breaches. 
  • Security compliance: Validate the API against industry-standard security protocols and compliance requirements, e.g. using 3rd party assessment (e.g. penetration test) as appropriate. 

For API consumers: ​​​​​

  • Token management: Test the proper handling and secure storage of authentication tokens (e.g. API keys, access tokens) within the consuming application. 
  • Access control: Ensure that the consuming application requests only the necessary permissions and adheres to the principle of least privilege when interacting with the API. 
  • Data security: Verify that sensitive data received from the API is handled securely within the application, including proper encryption and data masking as required. 
  • Security compliance: The use of a 3rd party assessment (e.g. penetration test) is also expected as a pre-requisite to access our APIs. 

 

Error handling use cases 

Purpose: Confirm that the API handles errors gracefully and provides useful feedback. 

For API providers:

  • Error codes: Ensure the API returns appropriate HTTP status codes for various error scenarios, such as 400 for bad requests and 500 for server errors. 
  • Error logging: Implement robust error logging mechanisms to capture and analyse errors for future improvements. 
  • User-friendly messages: Provide clear, descriptive error messages that help API consumers understand the issue and how to resolve it.

For API consumers: 

  • Error handling in application: Test the consuming application’s ability to correctly handle and display API errors, ensuring users receive appropriate feedback without exposing sensitive information. 
  • Retry logic: Implement and test retry logic in the application for handling transient API errors or network issues. 
  • User notifications: Ensure the application provides users with meaningful error messages or prompts when API errors occur, helping them take corrective action if necessary. 

 

 

 

 

Conformance and witness testing

In some cases, conformance testing or witness testing may be required to provide additional assurance that an API meets the necessary standards and performs as expected, particularly when requested by API product owners. These tests are especially important for APIs that play a critical role in business operations or need to comply with industry-specific guidelines.

  • Conformance testing: This involves verifying that the API adheres to defined standards, protocols, or guidelines. It’s essential when interoperability across different systems is a priority, or when the API must meet specific regulatory requirements. Ensuring that an API conforms to these standards helps prevent issues during integration and supports smoother collaboration across platforms.
  • Witness testing: Sometimes, testing needs to be observed by a designated representative or product owner to validate that the API behaves as expected in a real-world setting. Witness testing allows key stakeholders to review and confirm that the API meets their expectations, offering a level of assurance before the API is approved for production. This real-time feedback is crucial for catching potential issues early and making adjustments as necessary.

Depending on the API's nature and use case, these tests might be mandatory before full onboarding is approved. The results of conformance or witness testing will be documented and sent back to the API product owners for review. Ideally, there should be no major issues at this stage since the API should have already passed functional, security, and performance tests. These tests are more about ensuring alignment with standards, protocols, or specific design expectations, rather than uncovering new issues.

API Consumers should work closely with the API platform team to determine whether conformance or witness testing is necessary before moving forward with the final onboarding assurance process.

 

Test coverage 

For API publishers: 

  • Comprehensive coverage: The Testing Report should include tests for all API endpoints, including edge cases, error scenarios, and security vulnerabilities. 
  • Detailed test cases: Each test case should describe the input, expected output, and testing methodology. 

For API consumers: 

  • Targeted coverage: The Testing Report should focus on the specific API endpoints and functionalities that the application will use. Not all endpoints need to be tested if they are irrelevant to the application. 
  • Relevant test cases: Include test cases that reflect the actual use cases of the consuming application, with a focus on integration, performance, and error handling. 

 

Documentation 

For API publishers: 

  • Test report: The report should provide a detailed account of all test cases executed, results obtained, and any issues encountered, including steps to reproduce bugs and their severity. 
  • Bug tracking: Document and track all defects found during testing, along with their resolution status. 

For API consumers: 

  • Test summary: Provide a summary of the tests conducted on the API endpoints used by the consuming application, highlighting any issues encountered and how they were addressed. 
  • Integration notes: Include any specific notes on how the API was integrated into the application and any custom logic implemented to handle API responses or errors. 

 

User acceptance testing (UAT) 

For API publishers: 

  • Stakeholder feedback: Gather and include feedback from key stakeholders and end-users to ensure the API meets the business requirements. 
  • Usability testing: Verify that the API is easy to use, well-documented, and integrates smoothly with other systems. 

For API consumers: 

  • Application UAT: Conduct user acceptance testing with the actual application that consumes the API, ensuring it meets the business requirements and provides a seamless user experience. 
  • End-user feedback: Include feedback from end-users who interact with the application to validate that the API integration is effective and user-friendly. 

 

Conclusion 

API testing is a shared responsibility between providers and consumers, each with distinct but complementary roles. For API publishers, the goal is to deliver a fully functional, secure, and high-performance API that meets the needs of a wide range of users. For API consumers, the focus is on ensuring that their applications can effectively and securely integrate with the API, utilising only the necessary endpoints and handling responses and errors appropriately. 

By adhering to our guidelines and expectations, both providers and consumers can ensure that their APIs are reliable, secure, and optimised for their intended purposes. This collaboration fosters a seamless integration process, leading to robust, user-friendly applications that meet business objectives.