Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support captured parameters being used after the iteration completes #10

Closed
BenWoodworth opened this issue Nov 10, 2023 · 0 comments
Closed
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@BenWoodworth
Copy link
Owner

BenWoodworth commented Nov 10, 2023

In v0.1, this use case isn't supported:

suite("My test suite") {
    parametrize {
        val letter by parameter('a'..'z')

        test("Some dummy test for $letter") {
            letter.uppercase() shouldNotBe letter
        }
    }
}

That is, testing frameworks that declare their tests with DSLs. In this case, declaring separate test cases for each parameter argument. The test function registers a lambda that will execute later. And, since the letter parameter is captured by the test lambda, the parameter will be accessed after that parameterize iteration completed.

There are two issues with the current implementation that prevent this:

1. The parameter delegate is shared between iterations

When letter is finally accessed, the delegate's state will be pointing to an argument for a later iteration. That, or the parameter will be reset to an uninitialized state as if it weren't declared yet, and throw when accessed. Either way, letter won't give the expected value when the test lambda is executed.

The solution here is simple enough. Instead of the delegate representing "the current iteration's parameter", have it be a fixed value. Roughly equivalent to this in v0.1:

suite("Dummy test suite") {
    parametrize {
        val letter by parameter('a'..'z')
        val fixedLetter = letter

        test("Some dummy test for $letter") {
            fixedLetter.uppercase() shouldNotBe fixedLetter
        }
    }
}

Though not this straightforward, since the parameter delegate should still be able to communicate whether the parameter's been used or not (for failure reporting, only reporting the parameters that were used, which could have contributed to the failure)

2. Unused parameters do not contribute to argument combinations

This is an additional optimization to help the programmer in case a parameter is determined to be redundant, based on if it's used in the iteration it's declared. There's an issue specific to this here: #11

But how that affects this issue, take this code for example (assuming the issue above is resolved):

suite("Dummy test suite") {
    parametrize {
        val a by parameter(0..9)
        val b by parameter(0..9)

        test("Property should hold for $a and another digit") {
            a.assertThatPropertyHoldsWith(b)
        }
    }
}

Here, b is never used in the iteration, since it's only accessed afterwards from the registered test. So, b is assumed to be redundant so is never iterated over. The solution here is to drop the assumption that parameters will never be used (since they may have been captured and relied on later), and evaluate and iterate over them unconditionally.

@BenWoodworth BenWoodworth added the enhancement New feature or request label Nov 11, 2023
@BenWoodworth BenWoodworth added this to the 0.2 milestone Nov 11, 2023
@BenWoodworth BenWoodworth self-assigned this Nov 16, 2023
BenWoodworth added a commit that referenced this issue Nov 17, 2023
Previously the same delegate instance was shared between all iterations, and its argument points to the value in the current iteration. This was fine if the parameter is only used in the current iteration, but if a parameter is captured in a lambda and used later, the argument will change for a future iteration's argument combination, and the value will not be the same as when it was captured.

To fix this, the delegate is now unique to its iteration instead of being shared, and maintains an unchanging argument so that it can be captured and accessed later.

See issue #10 for context.
BenWoodworth added a commit that referenced this issue Nov 17, 2023
Previously the same delegate instance was shared between all iterations, and its argument points to the value in the current iteration. This was fine if the parameter is only used in the current iteration, but if a parameter is captured in a lambda and used later, the argument will change for a future iteration's argument combination, and the value will not be the same as when it was captured.

To fix this, the delegate is now unique to its iteration instead of being shared, and maintains an unchanging argument so that it can be captured and accessed later.

See issue #10 for context.
BenWoodworth added a commit that referenced this issue Nov 20, 2023
Previously the same delegate instance was shared between all iterations, and its argument points to the value in the current iteration. This was fine if the parameter is only used in the current iteration, but if a parameter is captured in a lambda and used later, the argument will change for a future iteration's argument combination, and the value will not be the same as when it was captured.

To fix this, the delegate is now unique to its iteration instead of being shared, and maintains an unchanging argument so that it can be captured and accessed later.

See issue #10 for context.
BenWoodworth added a commit that referenced this issue Nov 20, 2023
Previously the same delegate instance was shared between all iterations, and its argument points to the value in the current iteration. This was fine if the parameter is only used in the current iteration, but if a parameter is captured in a lambda and used later, the argument will change for a future iteration's argument combination, and the value will not be the same as when it was captured.

To fix this, the delegate is now unique to its iteration instead of being shared, and maintains an unchanging argument so that it can be captured and accessed later.

See issue #10 for context.
@BenWoodworth BenWoodworth changed the title Allow parameters to be used after its iteration completes. Support captured parameters being used after the iteration completes Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant