Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flaky test assertions in org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages #2682

Conversation

hofi1
Copy link
Contributor

@hofi1 hofi1 commented Oct 27, 2023

Issues

  • My PR addresses the following Helix issues and references them in the PR description:

Fixes #2667

Description

This fix changes the flaky assertions of the tests in the class org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages.
Sets return the elements in a non-deterministic order, which means that this assertion is not correct, because it checks whether the collections contain the same elements in the same order (this was changed in the library). This leads to a flaky test.

Assert.assertEquals(instances, _instancesMap.get(CLUSTER_NAME), "Instances from response: "
+ instances + " vs instances actually: " + _instancesMap.get(CLUSTER_NAME));

To fix this problem, the assertion has been rewritten to check if the collections contain the same amount of elements as well as both collections contain all values of the other collection.

String errorMessage = "Instances from response: "+ instances + " vs instances actually: "
+ _instancesMap.get(CLUSTER_NAME);
Assert.assertEquals(instances.size(), _instancesMap.get(CLUSTER_NAME).size(), errorMessage);
Assert.assertTrue(instances.containsAll(_instancesMap.get(CLUSTER_NAME)), errorMessage);
Assert.assertTrue(_instancesMap.get(CLUSTER_NAME).containsAll(instances), errorMessage);

The flaky test has been found by using the NonDex tool – to reproduce run

mvn -pl helix-rest edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.helix.rest.server.TestPerInstanceAccessor -DnondexSeed=933178

Tests

No test have been written – one existing test has been updated.

  • The following is the result of the "mvn test" command on the appropriate module:
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 01:23 min
    [INFO] Finished at: 2023-10-17T22:47:43-05:00
    [INFO] ------------------------------------------------------------------------

fix flaky test assertion in org.apache.helix
.rest.server.TestPerInstanceAccessor.testGetAllMessages
Copy link
Contributor

@desaikomal desaikomal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this.

@junkaixue junkaixue merged commit 442be11 into apache:master Feb 27, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky test assertion in org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages
3 participants