You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
coca's test suite currently consists mostly of documentation tests, mainly because I wanted to avoid duplicating code between unit tests and code examples. During the development of version 0.3, however, the drawbacks of this approach have become clear to me.
Starting with the most worrying, I introduced (and fixed) a soundness bug, which I only had Miri detect by chance, because the problem only manifested when using a particular storage type; otherwise identical client code wouldn't trigger it. The obvious answer here is to run each test for each storage type, but I didn't want to pollute the code examples with that, and, again, I didn't want to just duplicate the example code into the test suite either, potentially multiple times.
More generally, I often found myself writing examples I wasn't quite happy with, because I needed them to serve double duty as unit tests, and at the same time I had to trade off their overall effectiveness as a test suite because I also wanted to ensure they'd still be somewhat instructive as code samples. It just feels impossible to strike a good balance with this approach.
The course of action here seems clear, though far from glamorous:
Extract the existing doc tests into a new standard test suite. Take the time to review each test, and expand those that don't quite test enough. Use macro trickery to duplicate test functions so as to run each test for each storage type (where applicable), and make sure to support wasm-bindgen-test!
Clean up the actual example code. The data structures introduced before version 0.3 in particular often have multiple lines of setting up a backing array for slice storage, because at the time I wrote them, the quicker-to-construct InlineStorage still required a nightly compiler.
Add more examples that aren't glorified single method unit tests, to better illustrate higher-level usage patterns.
Set up additional static analysis in CI. While this isn't exactly related to testing, this is another means of quality assurance. Rudra in particular seems promising.
The text was updated successfully, but these errors were encountered:
coca's test suite currently consists mostly of documentation tests, mainly because I wanted to avoid duplicating code between unit tests and code examples. During the development of version 0.3, however, the drawbacks of this approach have become clear to me.
Starting with the most worrying, I introduced (and fixed) a soundness bug, which I only had Miri detect by chance, because the problem only manifested when using a particular storage type; otherwise identical client code wouldn't trigger it. The obvious answer here is to run each test for each storage type, but I didn't want to pollute the code examples with that, and, again, I didn't want to just duplicate the example code into the test suite either, potentially multiple times.
More generally, I often found myself writing examples I wasn't quite happy with, because I needed them to serve double duty as unit tests, and at the same time I had to trade off their overall effectiveness as a test suite because I also wanted to ensure they'd still be somewhat instructive as code samples. It just feels impossible to strike a good balance with this approach.
The course of action here seems clear, though far from glamorous:
wasm-bindgen-test
!fixed-slice-vec
, as per the recommendation of Reddit user /u/ajrw.InlineStorage
still required a nightly compiler.The text was updated successfully, but these errors were encountered: