Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize use of parameterization for testing #103

Open
leeren opened this issue Sep 22, 2023 · 0 comments
Open

Standardize use of parameterization for testing #103

leeren opened this issue Sep 22, 2023 · 0 comments
Assignees

Comments

@leeren
Copy link
Contributor

leeren commented Sep 22, 2023

For cases where we care about testing across spectrums of well-defined inputs (NOT fuzzing, which concerns itself with identifying malformed inputs), we should standardize use of a common test parameterization methodology.

Ideally, we can use something like table tests, which will likely eventually be supported more natively in foundry (see here).

Until then, the test parameterization strategy used by PRBMath provides a great baseline.

The overall idea here is to declare your test suite (for an expected set of test inputs and outcomes) as a modifiable storage array, paired with another single input-outcome set that will be used for individual validations.

struct SPTestSet {
  uint256 ipAssetInputA;
  uint256 ipAssetInputB;
  uint256 expectedIpAssetOutput;
}

SPTestSet[] spTestSets; // To iterate over
SPTestSet spTestSet;     // To run individual validations on

We can then create a modifier for parameterization using the following:

modifier parameterizeSPTestSet(SPTestSet[] memory spTestSets) {
        for (uint256 i = 0; i < testSuite.length; ++i) {
            spTestSet = spTestSets[i];
            _;
        }
    }

Then, we can create a function for creating different test set variations:

function generateSPTestSet() {
  delete spTestSets; // This is to ensure we can start fresh for each test.
  spTestSets.push(SPTestSet(1, 4, 5));
  spTestSets.push(SPTestSet(2, 6, 8));
  ...
 return spTestSets;
}

And finally, here is what an actual test might look like:

function test_SPValidation() parameterizeSPTestSet(generateSPTestSet()) {
  assertEq(spTestSet.ipAssetInputA + spTestSet.ipAssetInputB, spTestSet.ipAssetOutput);
  // Other assertions
}

Standardizing such a method of parameterization will allow us to more quickly iterate on tests that require large sets of defined inputs and outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants