Skip to content

fix: fix flaky test assertions in org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages#5

Open
hofi1 wants to merge 1 commit into
masterfrom
fix/fix-flaky-test-org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages
Open

fix: fix flaky test assertions in org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages#5
hofi1 wants to merge 1 commit into
masterfrom
fix/fix-flaky-test-org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages

Conversation

@hofi1

@hofi1 hofi1 commented Oct 18, 2023

Copy link
Copy Markdown
Owner

Issues

  • My PR addresses the following Helix issues and references them in the PR description:

Fixes apache#2667

Description

This fix changes the flaky assertions of the tests in the class org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages.
Sets return the elements in a non-deterministic order, which means that this assertion is not correct, because it checks whether the collections contain the same elements in the same order (this was changed in the library). This leads to a flaky test.

%TODO before

To fix this problem, the assertion has been rewritten to check if the collections contain the same amount of elements as well as both collections contain all values of the other collection.
%TODO after

The flaky test has been found by using the NonDex tool – to reproduce run

mvn -pl helix-rest edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.helix.rest.server.TestPerInstanceAccessor -DnondexSeed=933178

Tests

No test have been written – one existing test has been updated.

  • The following is the result of the "mvn test" command on the appropriate module:
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 01:23 min
    [INFO] Finished at: 2023-10-17T22:47:43-05:00
    [INFO] ------------------------------------------------------------------------

@harshith2000

Copy link
Copy Markdown

This fix looks good to me!

@zzjas

zzjas commented Oct 26, 2023

Copy link
Copy Markdown

One small thing to consider is that the developers might want to keep the error message, but other than that, the fix looks ok to me and you can proceed to open a real PR. Once you open a real PR, please mark this tentative PR as Opened in your tentative_pr.csv file and also raise a PR to IDoFT marking this as Opened. Thanks!

@same8891

Copy link
Copy Markdown

Hi, I think if you put code to the comment will make it easier to know how you fix it. Others LGTM.

fix flaky test assertion in org.apache.helix
.rest.server.TestPerInstanceAccessor.testGetAllMessages
@hofi1 hofi1 force-pushed the fix/fix-flaky-test-org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages branch from 7f2c73c to e0b1834 Compare October 27, 2023 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test assertion in org.apache.helix.rest.server.TestPerInstanceAccessor#testGetAllMessages

4 participants