Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new test result for slow tests that cause failures statistically. #67

Open
Sqeaky opened this issue Aug 24, 2020 · 0 comments
Open

Comments

@Sqeaky
Copy link
Member

Sqeaky commented Aug 24, 2020

Currently we have some microbenchmarks in various test suites for Mezzanine Packages. This usually pass. But occasionally one will fail if the build system is particularly loaded. If an algorithm is faster 99% of the time and fails because of system load this shouldn't be test blocking failure. That said this information still needs to be captured. So decisions can be made and potential actions can be taken.

To make this possible a new test result level should be added for poorly performing but passing tests. This needs to be higher than warning and lower than success.

@Sqeaky Sqeaky added this to the PreJagatiSprint milestone Aug 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant