Fixed flaky tests

[yesh385]

Created this PR to fix 6 flaky tests namely AnnotationUtilTest.getAnnotationSyncAlias, AnnotationUtilTest.getAnnotationValueTest2, AnnotationUtilTest.getAnnotationValueTest, AnnotationUtilTest.getCombinationAnnotationsTest, AnnotationUtilTest.getCombinationAnnotationsWithClassTest and AnnotationUtilTest.scanMetaAnnotationTest

which can be found here.

1. How was this test identified as flaky?
   This test was identifies as flaky by using an open-source research tool named NonDex which is responsible for finding and diagnosing non-deterministic runtime exceptions in Java programs.

2. What do the tests do?

• getAnnotationSyncAlias
  This test verifies the functionality of retrieving and adapting annotations in Java, including handling aliases and ensuring synthesized annotation properties.

• getAnnotationValueTest2
  This test validates the retrieval of annotation values from the ClassWithAnnotation class, specifically the names attribute of AnnotationForTest, ensuring it matches an expected array.

• getAnnotationValueTest
  This test checks if the value retrieved from the annotation AnnotationForTest on the ClassWithAnnotation class is equal to the expected string.

• getCombinationAnnotationsTest
  This test checks if the getAnnotations() method of AnnotationUtil returns the expected array of annotations for a given class.

• getCombinationAnnotationsWithClassTest
  This test verifies if the getCombinationAnnotations() method of AnnotationUtil correctly retrieves annotations of a specific type for a given class.

• scanMetaAnnotationTest
  This test scans for meta-annotations (annotations on other annotations) starting from the RootAnnotation class.

3. Why the tests fail?

The issue with the first 5 tests is that we are comparing the equality of declaredAnnotations and annotations without paying attention to the order of the annotations. If we compare them without considering the order of the elements then the annotationMap could sometimes be initialized to an empty TableMap which is incorrect. The following line is where the mentioned comparison happens.

hutool/hutool-core/src/main/java/cn/hutool/core/annotation/CombinationAnnotationElement.java

Line 105 in 20f0c72

if (Arrays.equals(declaredAnnotations, annotations)) {

Because of this underlying issue, each of the 5 tests fail in the following lines:

• getAnnotationSyncAlias
  

  hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

  Line 55 in 20f0c72

  Assert.assertEquals("测试", annotation.retry());

• getAnnotationValueTest2
  

  hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

  Line 45 in 20f0c72

  Assert.assertTrue(ArrayUtil.equals(names, new String[]{"测试1", "测试2"}));

• getAnnotationValueTest
  

  hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

  Line 38 in 20f0c72

  Assert.assertEquals("测试", value);

• getCombinationAnnotationsTest
  

  hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

  Line 24 in 20f0c72

  Assert.assertEquals(2, annotations.length);

• getCombinationAnnotationsWithClassTest
  

  hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

  Line 32 in 20f0c72

  Assert.assertEquals("测试", annotations[0].value());

The issue with the 6th test is that the getAnnotationsFromTargetClass() method returns the targetAnnotations in reverse order. This causes the order of annotations being compared in the test to be incorrect, causing the assertion to fail.

hutool/hutool-core/src/test/java/cn/hutool/core/annotation/AnnotationUtilTest.java

Line 81 in 20f0c72

Assert.assertEquals(RootMetaAnnotation3.class, annotations.get(0).annotationType());

4. How I fixed these tests?

This PR fixes the first 5 tests by sorting both declaredAnnotations and annotations before comparing their equality. It also fixes the 6th test by sorting the annotations in reverse order.

You can run the following commands to run the tests using the NonDex tool:

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationSyncAlias

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationValueTest2

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationValueTest

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getCombinationAnnotationsTest

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getCombinationAnnotationsWithClassTest

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -pl hutool-core -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#scanMetaAnnotationTest

(Optional) You can also run the following commands to run the test:

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationSyncAlias

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationValueTest2

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getAnnotationValueTest

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getCombinationAnnotationsTest

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#getCombinationAnnotationsWithClassTest

mvn -pl hutool-core test -Dtest=cn.hutool.core.annotation.AnnotationUtilTest#scanMetaAnnotationTest

Test Environment:

java version "1.8.0_202"
Apache Maven 3.6.3

Kindly let me know if these fixes are acceptable and if not, I would appreciate it if you could provide feedback/suggestions on my fixes. Thank you.

[Nemesis123925]

Simple fix and looks good to me. But what's confusing is that if the original tests is using arrays, are the orders of the arrays also a element to test?

[harshith2000]

Hi, The fix looks good to me. But can you please confirm if these changes do not break the other tests in this module? You can check this by doing a test on the entire module.

[yesh385]

Thank you for the feedback. These changes do not break other tests.

[zzjas]

https://campuswire.com/c/G7A0E96FD/feed/501 good to go