[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT

[opensearch-ci-bot]

Flaky Test Report for ClientYamlTestSuiteIT

Noticed the ClientYamlTestSuiteIT has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	25 61047 62309 62314 62323 62327 62335 49451 74366 57351 60802 62063 62124 62524 62838 62900 63398 63488 63855 64209 65063 65179 65544 65778 66405 66492 66732 66752 66755 66802 66844 67198 67225 68032 68340 68696 69088 69841 69998 70002 70120 70519 70919 71020 71489 71638 71648 71882 72520 72572 72757 73565 73664 73748 74095 75199 78522 79670	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=index/111_false_allow_templates/Index documents with setting dynamic parameter to false_allow_templates in the mapping of the index} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/date_histogram profiler} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/95_search_after_shard_doc/accept _shard_doc with PIT + paginate with search_after (multi-shard)} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs} org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/histogram profiler} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/10_source_filtering/_source as an empty array} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/10_source_filtering/_source as an empty object} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/10_source_filtering/_source as object with an empty excludes array} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/10_source_filtering/_source as object with an empty includes array} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fields - pattern} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.refresh/10_basic/Indices refresh test no-match wildcard} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fielddata fields - all metric} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fielddata fields - one} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.stats/50_noop_update/Test noop_update_total metric can be updated by both update API and bulk API} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/170_cardinality_metric_unsigned/Missing field with missing param} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.profile/10_fetch_phase/Combined fetch sub-phases profiling}
bccad99	21874	78792	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.profile/10_fetch_phase/Combined fetch sub-phases profiling}
e361637	21441	75458	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/histogram profiler}
ee40b5f	21345	75065	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler}
ec4dd15	21244	74625	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler}
669ffc3	21154	74188	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
63f64c7	21017	74042	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/histogram profiler}
be092d1	21068	73813	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler}
e52e135	20833	72539	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.profile/10_fetch_phase/Combined fetch sub-phases profiling}
c12aa29	20606	71803	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler}
af1ecd3	20160	68915	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
7d97f77	20106	68135	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.profile/10_fetch_phase/Combined fetch sub-phases profiling}
9f28ef8	18825	62305	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=index/111_false_allow_templates/Index documents with setting dynamic parameter to false_allow_templates in the mapping of the index}
bf5b43d	18910	61893	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs}
2e027ee	18939	61825	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs}
9b79f19	18887	61740	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
46a0045	18636	59978	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/330_auto_date_histogram/profile at top level}
0e2ec6f	18541	59739	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/330_auto_date_histogram/profile at top level}
3c3d380	18072	57136	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/date_histogram profiler}
481fd38	16098	48529	org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.segments/10_basic/basic segments test}
c85ee68	15988	48303	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
d64c5f8	15866	47801	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
2742e74	15700	47023	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
0df6e9e	15098	43940	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.delete_index_template/10_basic/Delete index template which is not used by data stream but index pattern matches}
fdd3ea7	15023	43647	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
ed24a14	14881	43113	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
7a54f71	14707	42236	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
8b62948	13953	39797	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the ClientYamlTestSuiteIT class are:

• 19176
• 18814
• 15024
• 17329
• 16366
• 17718
• 16803
• 18631
• 14443
• 19209
• 18842
• 21954
• 15124
• 15568
• 17383
• 21607
• 16294
• 21632
• 22237
• 15131
• 18683
• 19009
• 22207
• 21693
• 22240
• 13853
• 13829
• 22187
• 21637
• 22159
• 18568
• 21831
• 21899
• 19304
• 20346
• 20742
• 21642
• 19321
• 20831
• 18424
• 21253
• 19699
• 19115
• 22189
• 20221
• 20145
• 21904
• 20140
• 21732
• 22359
• 21463
• 21814
• 15333
• 20442
• 22243
• 20648
• 20132
• 20847
• 22277
• 20526
• 19352
• 21457
• 20779
• 21867
• 19098
• 20909
• 16884
• 19862
• 18277
• 21365
• 15258
• 18632
• 18901
• 19885
• 20014
• 21209
• 15922
• 19008
• 20503
• 13865
• 17756
• 20683
• 21723
• 13788
• 19573
• 18523
• 20680
• 17772
• 13821
• 13620
• 17638
• 20230
• 17009
• 22306
• 22030
• 22186
• 14446
• 13232
• 18195
• 18777
• 13461
• 18693
• 19683
• 17740
• 22348
• 17023
• 19793
• 20700
• 20551
• 18929
• 20802
• 14383
• 15039
• 17978
• 18673
• 20067
• 21417
• 15012
• 22280
• 19524
• 15365
• 18455
• 18707
• 18996
• 19033
• 19255
• 22338
• 18504
• 19152
• 22340
• 14069
• 14464
• 15545
• 18161
• 18215
• 14454
• 15289
• 18618
• 20483
• 21009
• 14718
• 14774
• 15882
• 17039
• 18597
• 18879
• 19776
• 20785
• 14613
• 20204
• 20417
• 20515
• 14711
• 19054
• 19661
• 19704
• 19765
• 21029
• 21133
• 21418
• 20432
• 20844
• 20855
• 13573
• 14486
• 15194
• 15410
• 15509
• 15607
• 16297
• 16731
• 16960
• 17447
• 17919
• 17951
• 18718
• 18936
• 19046
• 19057
• 19958
• 19979
• 20959
• 21318
• 19939
• 14817
• 15145
• 15256
• 15557
• 15603
• 18498
• 18601
• 18874
• 20260
• 13190
• 13929
• 14611
• 14630
• 15100
• 15153
• 15358
• 15381
• 15578
• 16038
• 16726
• 17273
• 17712
• 17750
• 18106
• 18194
• 18491
• 18664
• 18896
• 18924
• 18966
• 19076
• 19396
• 21834
• 22131
• 14513
• 20176
• 14415
• 17855
• 18544
• 19648
• 19685
• 19936
• 21178
• 21860
• 22045
• 12782
• 13772
• 13888
• 14118
• 14195
• 14684
• 15128
• 15326
• 15430
• 15649
• 15679
• 16021
• 16904
• 18184
• 18346
• 18467
• 18479
• 18511
• 18668
• 18889
• 18923
• 18961
• 18978
• 19136
• 19174
• 19187
• 19283
• 19306
• 19307
• 19331
• 19403
• 19436
• 19455
• 19607
• 19630
• 19668
• 19675
• 19698
• 19783
• 19937
• 19985
• 20009
• 20065
• 20112
• 20113
• 20192
• 20371
• 20372
• 20411
• 20434
• 20440
• 20549
• 20570
• 20617
• 20792
• 20810
• 20834
• 20859
• 20873
• 20887
• 20908
• 20949
• 21110
• 21145
• 21149
• 21225
• 21237
• 21474
• 21544
• 21563
• 21660
• 21703
• 21803
• 21819
• 21921
• 21944
• 21962
• 22002
• 22007
• 22025
• 22146

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[getsaurabh02]

@prudhvigodithi can we confirm if these failures are pointing to an individual test or the overall test class?

[getsaurabh02]

Also, can we highlight the branch the test is failing on, in the issue description?

[prudhvigodithi]

Hey @getsaurabh02 today the gradle check runs on a commit ref, the issue description, the markdown table created has the commit that has the flaky failure. Here for this issue the commit 8b62948 has a flaky failure for the test class ClientYamlTestSuiteIT and the failing test is org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}.

Sometimes there are can be more than one test failing for the same commit or for the different one, here is the example #14381, in that case we group them under one issue created for a specific test class name.