Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests (libssh, httpd, bind) #358

Merged
merged 6 commits into from
Mar 13, 2024

Conversation

The-Mule
Copy link
Collaborator

This PR adds two new tests (for httpd and bind) and re-factors previously added test for libssh. Tests are intended to run in CI (github actions) but they can be executed on any Fedora (39 and newer) as follows (GIT_URL defaults to this repository and GIT_REF to the main branch):

# cd tests/integration
# GIT_URL=https://github.com/The-Mule/pkcs11-provider.git GIT_REF=integration-tests bash -e libssh.sh

Testing requires bash -e option. Non-zero exit code means failure. Provider debug logs are always printed out at the end (regardless of the result). All three tests uses softhsm token (bind test does so via p11-kit server proxy). Please notice that until Fedora 40 is released both httpd and bind test need to use a specific docker image.

@The-Mule The-Mule force-pushed the integration-tests branch from ce8f1f6 to c252279 Compare March 12, 2024 12:39
@The-Mule
Copy link
Collaborator Author

The-Mule commented Mar 12, 2024

New integration tests results can be found in the checks too (all passes).

@The-Mule The-Mule force-pushed the integration-tests branch from 4732395 to ef4bb48 Compare March 12, 2024 14:07
Copy link
Member

@simo5 simo5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, but I am afarid of the time it may take to run this is there are a lot of steps that pull lots of data from the network.

.github/workflows/integration.yml Outdated Show resolved Hide resolved
tests/integration/libssh.sh Show resolved Hide resolved
@simo5
Copy link
Member

simo5 commented Mar 12, 2024

So the integration step took 6 minutes.
4.42 minutes were spent in downloading/installing stuff.
We should probably use caches aggressively.

See this for inspiration:
https://github.com/latchset/kryoptic/blob/15bd875f6ce5acbaa93c909074f1da2812215a68/.github/workflows/build.yml#L27

@The-Mule
Copy link
Collaborator Author

I'll take a look how to make it faster, a lot of time is spent on installing F40 packages now. But maybe that can be cached too somehow.

@simo5
Copy link
Member

simo5 commented Mar 12, 2024

I'll take a look how to make it faster, a lot of time is spent on installing F40 packages now. But maybe that can be cached too somehow.

Se previous comment, you just need to copy that code :-)

@The-Mule
Copy link
Collaborator Author

With matrix strategy integration tests workflow only takes ~2 minutes (1:57) now. I tried to add actions/cache on top of that but it had no effect in this case (it looks like dnf step is now running at the same time in all three jobs).

@simo5
Copy link
Member

simo5 commented Mar 13, 2024

Awesome thanks,
I think we should still cache because some tests are still dominated by install time, and it is all in download time. But we can do that in a followup PR.

@simo5 simo5 merged commit ec4a4a1 into latchset:main Mar 13, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants