Testing and Review

For each bug fix or feature, state the tests done to verify that the code works properly. Compare results against known data where possible. Ideally, developers in a should repo work together (and with code owners and admin) to develop a suite of tests with a proven data set. Before writing code, developers should design desired outcomes, i.e. test results. They should outline, on a high level, how the code is supposed to work. They should also state any design requirements for their code. When testing code, a developer should test their code according to the following workflow to ensure the code will not cause problems when it is merged with existing code:

1. Unit test –Smallest in scope, a unit test is a test in isolation. For the sake of example, imagine the code in question is a subfeature. In a unit test, the developer would test that the subfeature by itself gets its desired results and does not have any bugs.
2. Integration test – Test the code’s functionality in context. In this example, the developer would test that the subfeature does what it’s intended to do when combined with its parent feature and does not create any new bugs in the parent feature.
3. End-to-end test – Test the code’s functionality from start to finish in context of the larger workflow. For example, say that the subfeature’s parent feature belongs to a suite of features intended to perform analysis on a piece of data. For an end-to-end test, the developer would use the suite of features to make sure, now that the subfeature has been added, that they obtain the desired results from the data and don’t encounter any new bugs.

Regression testing is in a similar vein as the testing described above. Regression testing ensures that new code works with code and tests that are already in place. When creating new fixes or features, developers should try to be thoughtful of how that code fits in with tests that are already in place.

Code should always be thoroughly reviewed when it is being merged with the master branch. Because the master branch should always be production-ready, an admin, along with at least one other developer, should review the code before it is merged. If a bad merge happens, it’s realistically not a big problem because the master branch can be rolled back to a previous commit, but every contributor should do their due diligence that their pull request won’t cause any major issues.

When a new feature has been successfully merged, the developer(s) should create a new wiki page—or make an addition to an existing page—describing their feature. New toolboxes will likely require their own wiki section.