Skip to content

Fixed 2 flaky tests#2

Open
yesh385 wants to merge 49 commits into
v5-masterfrom
flaky-fix
Open

Fixed 2 flaky tests#2
yesh385 wants to merge 49 commits into
v5-masterfrom
flaky-fix

Conversation

@yesh385

@yesh385 yesh385 commented Oct 30, 2023

Copy link
Copy Markdown
Owner

Created this PR to fix 2 flaky tests namely baseSynthesisAnnotationWorkTest which can be found here and getMethodsFromInterfaceTest which can be found here.

  1. How was this test identified as flaky?
    This test was identifies as flaky by using an open-source research tool named NonDex which is responsible for finding and diagnosing non-deterministic runtime exceptions in Java programs.

  2. What do the tests do?

  • baseSynthesisAnnotationWorkTest
    It focuses on verifying the functionality related to annotations and annotated elements. Specifically, it involves retrieving and manipulating annotations associated with various classes and elements. The test also assesses the behavior of certain extension methods, as well as attributes and processors related to synthesized annotations.

  • getMethodsFromInterfaceTest
    Confirms that when working with interfaces, calling getMethods directly retrieves all public methods. It obtains a total of four methods from TestInterface1, TestInterface2, and TestInterface3. The test then verifies that the results are identical when using getPublicMethods.

  1. Why do the tests fail?
  1. How I fixed these tests?

This PR fixes baseSynthesisAnnotationWorkTest by sorting the synthesizedAnnotations before comparing it with the expected order of childAnnotation, grandParentAnnotation, parentAnnotation.

This PR fixes getMethodsFromInterfaceTest by sorting both methods and publicMethods before checking that they are equal.

You can run the following commands to run the tests using NonDex tool:

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.GenericSynthesizedAggregateAnnotationTest#baseSynthesisAnnotationWorkTest
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.util.ReflectUtilTest#getMethodsFromInterfaceTest

(Optional) You can also run the following command to run the test:

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.GenericSynthesizedAggregateAnnotationTest#baseSynthesisAnnotationWorkTest
mvn -pl hutool-core test -Dtest=cn.hutool.core.util.ReflectUtilTest#getMethodsFromInterfaceTest

Test Environment:

java version "1.8.0_202"
Apache Maven 3.6.3

looly and others added 30 commits September 13, 2023 21:40
Merge pull request !1075 from handy/handy
Merge pull request !1076 from summertime-wu/v5-dev
使用 contentEquals(app) 直接进行 char 数组比对,避免 equals(app.toString()) 中将 StringBuilder 转为 String,会多一道 char 数组拷贝
Merge pull request !1084 from summertime-wu/v5-dev
Merge pull request !1092 from flavormark/v5-dev
@prathyushreddylpr

Copy link
Copy Markdown

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants