Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance the Test Suite #3

Open
5 tasks
teryror opened this issue Mar 7, 2022 · 0 comments
Open
5 tasks

Enhance the Test Suite #3

teryror opened this issue Mar 7, 2022 · 0 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@teryror
Copy link
Owner

teryror commented Mar 7, 2022

coca's test suite currently consists mostly of documentation tests, mainly because I wanted to avoid duplicating code between unit tests and code examples. During the development of version 0.3, however, the drawbacks of this approach have become clear to me.

Starting with the most worrying, I introduced (and fixed) a soundness bug, which I only had Miri detect by chance, because the problem only manifested when using a particular storage type; otherwise identical client code wouldn't trigger it. The obvious answer here is to run each test for each storage type, but I didn't want to pollute the code examples with that, and, again, I didn't want to just duplicate the example code into the test suite either, potentially multiple times.

More generally, I often found myself writing examples I wasn't quite happy with, because I needed them to serve double duty as unit tests, and at the same time I had to trade off their overall effectiveness as a test suite because I also wanted to ensure they'd still be somewhat instructive as code samples. It just feels impossible to strike a good balance with this approach.

The course of action here seems clear, though far from glamorous:

  • Extract the existing doc tests into a new standard test suite. Take the time to review each test, and expand those that don't quite test enough. Use macro trickery to duplicate test functions so as to run each test for each storage type (where applicable), and make sure to support wasm-bindgen-test!
  • Clean up the actual example code. The data structures introduced before version 0.3 in particular often have multiple lines of setting up a backing array for slice storage, because at the time I wrote them, the quicker-to-construct InlineStorage still required a nightly compiler.
    • Add more examples that aren't glorified single method unit tests, to better illustrate higher-level usage patterns.
  • Set up additional static analysis in CI. While this isn't exactly related to testing, this is another means of quality assurance. Rudra in particular seems promising.
@teryror teryror added documentation Improvements or additions to documentation enhancement New feature or request labels Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant