-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate API design tests to ensure CAMARA compliance #200
Comments
@Kevsy I'm supportive of automating tests as far as possible.
Actually, linting should run before every merge of changes to the OAS. That's the way it's today implemented in QualityOnDemand sub project (can be used as a blueprint).
Established in QualityOnDemand with mega linter as umbrella. You can fork the repo and test it.
That's what #15 is about.
I'm not sure, but from first view the tool is generating tests based on the OAS but then they supposed to be executed against an API implementation? But maybe I need to spend more time on it.
|
@hdamker To transfer this issue back to Commonalities as agreed within https://wiki.camaraproject.org/display/CAM/2024-05-14+Release+WG+Minutes, as this concerns linting & tests to be defined in Commonalities. |
I propose to to close this issue and continue in #188 |
This proposal is to create a pipeline of three tests to ensure that the API definition has been defined according to CAMARA API design principles and is 'secure by design'. These tests are against the API definition itself and not the implementation.
Part of this topic has been raised before - see e.g. Commonalties camaraproject/QualityOnDemand#6 - but this proposal extends the concept with a test pipeline. The tests should be run as part of every release and act as a gate for release publication.
The tests are:
1. Check compliance against OAS:
Using e.g. Spectral lint GitHub action.
2. Check compliance against CAMARA API design guidelines:
Spectral lint again, but with a custom ruleset based on the rules in the API design guidelines. This ruleset would need to be derived from the API design guidelines
3. Check Security by Design:
Fuzz testing using Schemathesis . Detects errors/vulnerabilities at design level. Produces a report at the end too! Available as a GitHub app.
4. Check API meets business requirements
BDD (Gherkin .feature) needs to be present, although I'm not sure if the .feature file itself can be tested to determine CAMARA compliance.
Tests 1 and 3 are 'out of the box' and require no further configuration. Test 2 would require a linting ruleset to be derived from the API Design Guidelines.
The text was updated successfully, but these errors were encountered: