Set up API testing infrastructure

[OhmSpectator]

Description

To ensure the reliability and robustness of our API, we need to set up a comprehensive API testing infrastructure. This infrastructure will facilitate generating, maintaining, and executing tests based on our Swagger (OpenAPI) documentation.

Requirements

• Evaluate and select appropriate testing frameworks and tools for API testing (e.g., Mocha, Jest for test running; Dredd or Swagger Test Templates for contract testing).
• Install and configure the selected testing framework in our development environment.
• Integrate the Swagger (OpenAPI) specification with the chosen tool to generate initial contract tests.
• Configure environment handling for test execution, including variables and setup for different stages (development, staging, production).
• Establish guidelines and standards for writing and maintaining API tests, including directory structure, naming conventions, and documentation practices.
• Set up a continuous integration (CI) pipeline to automatically run API tests against every commit or pull request.
• Ensure the ability to run tests locally for development and debugging purposes.
• Document the setup process, usage instructions, and any relevant information to enable team members to run and write new tests.

Additional Information

• Authentication and authorization mechanisms need to be handled or mocked during testing.
• Consider how to report and visualize test results and coverage effectively.
• Keep scalability in mind to accommodate future API growth and complexity.

Related Issues

• Initial Setup of Test Suite #182

[OhmSpectator]

Feature/Aspect	Dredd	Swagger Test Templates
Primary Function	API testing tool to validate API implementation against documentation	Utility to generate API test code from Swagger documentation
Approach	Sends requests to API and validates the responses against documentation	Generates customizable test code for manual integration
Language Support	JavaScript for hooks; language-agnostic for API testing	Generates JavaScript code; best used with JS testing frameworks
Ease of Use	Relatively easy to start with; can become complex with hooks	Requires initial setup and integration, but highly customizable
Customization	Through hooks for pre/post testing actions	High - generated tests are customizable templates
Framework Compatibility	Primarily standalone but can work with any language/framework through hooks	Designed for JavaScript environments; works well with Mocha, Chai, Jest, etc.
Setup	Install and point to API specification and endpoint	Install, generate tests, and integrate into test suite
Use Case	Good for out-of-the-box API contract testing	Better for projects requiring detailed test customization
Community and Support	Large user base, actively maintained	Part of the Swagger ecosystem, good community support

If you need a quick, out-of-the-box solution to start validating your API against its documentation, Dredd might be the way to go. On the other hand, if you require more control over your testing process, detailed customization, or integration with specific JavaScript testing frameworks, Swagger Test Templates might be more suitable. In any case, both tools bring powerful capabilities for ensuring your API behaves as documented, which is crucial for API reliability and contract adherence.

[OhmSpectator]

Ok, I'm thinking about using Swagger Test Template + Jest.

[OhmSpectator]

Setting Up Testing for API with Swagger Test Template and Jest

Follow these detailed instructions to set up testing for your API using Swagger Test Template and Jest:

Step 1: Setting Up Jest

1. Navigate to Your Backend Directory: Ensure you are in the backend directory where your package.json resides.

   cd backend

2. Install Jest: Jest is a JavaScript Testing Framework focusing on simplicity. Install it as a dev dependency.

   npm install --save-dev jest

3. Configure Jest: In your package.json, add a script to run Jest and specify the test environment.

   "scripts": {
       "test": "jest"
   },
   "jest": {
       "testEnvironment": "node"
   }

Step 2: Setting Up Swagger Test Template

1. Install Swagger Test Template: This tool generates test cases based on your Swagger/OpenAPI documentation. Install as a dev dependency.

   npm install --save-dev swagger-test-templates

2. Generate Test Cases: Use Swagger Test Template to generate test cases. Specify the path to your API specification and the desired output for the test cases.

   npx swagger-test-templates -s ../api/api.yaml -o ./test -t templates/

   • -s: Path to your Swagger/OpenAPI file.
   • -o: Output directory for generated test files.
   • -t: Path to your test templates if you have custom ones.

Step 3: Writing Test Templates

Before generating test cases, consider defining or customizing test templates according to your requirements. Swagger Test Template uses Handlebars syntax for templates. Modify templates to accommodate specifics of your API testing, like adding authentication headers or setting up mock data.

Step 4: Customizing Generated Test Cases

After generating the test cases:

1. Review and Customize: Navigate to the generated test files and review them. You may need to add additional tests or modify existing ones to suit your needs, such as setting up prerequisites for the tests, mocking databases, or adding authentication.

2. Add Setup and Teardown: In your test files, add any necessary setup and teardown steps. This might involve connecting to a test database, initializing data, or cleaning up after tests are complete.

Step 5: Running Tests

1. Run Your Tests: Use the script you added to your package.json to run the tests.

   npm test

2. Review Test Output: Jest will execute the tests and provide a report of passed and failed tests. Review the output to understand what's working as expected and what needs adjustment.

Additional Considerations:

• Mocking External Services: If your API interacts with external services, consider using tools like nock to mock HTTP responses.
• Continuous Integration: Integrate testing into your CI/CD pipeline to automatically run tests with each build or push to specific branches.
• Test Coverage: Consider adding a Jest coverage report to track how much of your codebase is covered by tests.

By following these steps, you'll have set up a robust testing suite for your API using Swagger Test Template and Jest, ensuring that your application works as expected and is resilient to changes over time. Adjust and expand your tests as your API evolves to maintain reliability and quality.

[OhmSpectator]

Here is also an article about the contract testing
https://medium.com/geekculture/contract-testing-with-openapi-42267098ddc7

[OhmSpectator]

Dredd cannot work with OpenAPI 3.1...
Postman is a UI-based tool... I'm lost a little bit on how to continue here...

[OhmSpectator]

Also, see tools mentioned here
https://openapi.tools/

[OhmSpectator]

Here is a new tool to be checked: https://www.usebruno.com/