Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove inaccurate validation tests. #1428

Merged
merged 7 commits into from
Nov 18, 2022

Conversation

joaander
Copy link
Member

Description

Remove inaccurate validation tests.

Motivation and context

These tests provide a false sense of correctness. They are not run long enough to sample the chosen state points. We have implemented a new validation test suite, hoomd-validation that implements these tests (and more) in a signac-flow project that runs on HPC resources.

Resolves #210

How has this been tested?

The CI checks pass.

Change log

Internal change, no change log entry is needed.

Checklist:

@joaander joaander added the validate Execute long running validation tests on pull requests label Oct 31, 2022
@joaander
Copy link
Member Author

https://github.com/glotzerlab/hoomd-validation is now public and provides a superset of all tests removed in this pull request (some tests are still in open pull requests at the time this message is posted).

@joaander joaander marked this pull request as ready for review November 17, 2022 16:40
@joaander joaander requested review from a team, tommy-waltmann and Charlottez112 and removed request for a team November 17, 2022 16:40
Copy link
Contributor

@tommy-waltmann tommy-waltmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just have one point for discussion about the label for longer tests.

Comment on lines +22 to +23
**Long running tests** check for correct behavior, but require up to a minute to execute. Mark long
running tests with the ``validate`` label.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to keep a label for the longer running tests, but what if we changed the label to be --long-tests or something along those lines intead of --validate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change. We could do this in separate pull request for v4. The canonical name would be slow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

@joaander joaander merged commit 358b5f3 into trunk-patch Nov 18, 2022
@joaander joaander deleted the remove-inaccurate-validation-tests branch November 18, 2022 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
validate Execute long running validation tests on pull requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Additional validation tests
3 participants