[FLINK-39565] Eliminate maven build warnings across all modules

[Dennis-Mircea]

What is the purpose of the change

This pull request cleans up the maven build so that mvn clean install produces zero [WARNING] lines across all modules, while preserving runtime behavior of the operator, autoscaler, webhook, and example modules.

The build was emitting hundreds of warnings spread across the RAT plugin, CycloneDX plugin, the shade plugin (operator, webhook, autoscaler-standalone, three example modules), and the assembly plugin (operator + webhook). None of them were blocking the build, but they masked legitimate signal during code review and CI runs.

This change is build-plumbing only: no production class, no CRD, no Helm value, no SPI contract, and no public API was modified.

Brief change log

• apache-rat-plugin 0.16.1 → 0.18, migrated to the v0.17 schema:
  • Custom ruleset extracted to tools/rat/rat-config.xml and wired in via the non-deprecated <configs> parameter (the old inline <families> / <licenses> / <approvedLicenses> are deprecated for removal in 0.17+).
  • <configurationNoDefaults>true</configurationNoDefaults> disables apache-rat-core/org/apache/rat/default.xml, which would otherwise also recognise AL1.0 / AL1.1 (legacy Apache License) and MIT / BSD-3 / CDDL1 / OASIS / W3C families. The new ruleset declares a single family AL containing exactly one license AL2.0 matched on the canonical ASF source-header opener Licensed to the Apache Software Foundation (ASF) under one, and approves only that family.
  • <numUnapprovedLicenses>0</numUnapprovedLicenses> → <counterMaxs><counterMax>Unapproved:0</counterMax></counterMaxs>.
  • <excludes> → <inputExcludes>. Added two pre-existing third-party files (LICENSE.rsync-deployments, e2e-tests/cert-manager.yaml) to the exclude list, they had been silently passing under RAT 0.16's permissive defaults.
• cyclonedx-maven-plugin 2.9.0 → 2.9.1, plus .mvn/maven.config setting -Dorg.slf4j.simpleLogger.log.com.networknt.schema=error to silence the validator's Unknown keyword meta:enum / Unknown keyword deprecated chatter while keeping every other logger at default level.
• maven-shade-plugin 3.1.1 → 3.6.2 (centrally pinned in the root pom; required for MSHADE-460 / MSHADE-441 fixes around module-info.class filtering). Configuration overhauled in flink-kubernetes-operator, flink-kubernetes-webhook, flink-autoscaler-standalone, examples/flink-beam-example, examples/flink-sql-runner-example, examples/autoscaling:
  • Comprehensive *:* metadata <filters> block dropping JPMS module descriptors (module-info.class, META-INF/**/module-info.class), signature files (META-INF/*.SF / *.DSA / *.RSA), per-dep maven metadata (META-INF/maven/**), per-dep LICENSE / NOTICE / DEPENDENCIES, Kotlin *.kotlin_module files, ProGuard rules, GraalVM native-image hints, jandex indices, netty release metadata, and META-INF/MANIFEST.MF (regenerated by ManifestResourceTransformer).
  • Deterministic per-artifact "winner" filters for previously-overlapping intentional forks: operator/autoscaler forks of org.apache.flink.runtime.rest.messages.{job.JobDetailsInfo, JobDetailsInfo$*, job.metrics.AggregatedMetric, job.metrics.IOMetricsInfo, JobExceptionsInfoWithHistory, JobExceptionsInfoWithHistory$*, json.SlotSharingGroupID(De)Serializer} win over flink-runtime; operator's FlinkConfMountDecorator wins over flink-kubernetes; operator's log4j2.properties wins over flink-kubernetes-operator-api's; okio-jvm wins over okio for META-INF/okio.kotlin_module.
  • Added Log4j2PluginCacheFileTransformer (extension org.apache.logging.log4j:log4j-transform-maven-shade-plugin-extensions:0.2.0) to merge Log4j2Plugins.dat from log4j-core and log4j-1.2-api. Previously this resource was overwritten last-write-wins, which could silently drop log4j plugin definitions.
  • Added <createDependencyReducedPom>false</createDependencyReducedPom> to the flink-kubernetes-operator shade execution. The unshaded jar is the published main artifact (downstream modules like flink-kubernetes-webhook compile against it as a regular Maven dep), so the dep-reduced pom, which strips compile-scope deps such as flink-core, slf4j, fabric8, jackson, was breaking transitive resolution. Stale dependency-reduced-pom.xml files removed.
• maven-assembly-plugin pinned to 3.8.0 (operator + webhook); <finalName>test-plugins</finalName> moved from execution-scope to plugin-scope <configuration> (3.4+ rejects it as read-only at execution scope). Empty / leading-slash <outputDirectory> entries removed from both modules' src/test/assembly/test-plugins-assembly.xml to drop the "*nix-specific root-relative reference" warning.
• Beam example: excluded org.apache.beam:beam-sdks-java-extensions-ml from beam-examples-java to silence the upstream "POM is invalid, transitive dependencies will not be available" warning (Beam 2.62.0 ships a malformed POM for that artifact). The bundled examples don't reference the ML extensions.

Verifying this change

This change is a build/packaging cleanup with no functional code changes. It is verifiable by build and inspection rather than unit tests:

• mvn clean install -DskipTests from repo root → BUILD SUCCESS, zero [WARNING] lines.
• mvn -pl flink-kubernetes-webhook,flink-autoscaler-standalone -am package -DskipTests → BUILD SUCCESS, zero [WARNING] lines.
• mvn -N apache-rat:check with a probe Java file carrying an MIT-style header dropped into flink-autoscaler/src/main/java/... is rejected with Unknown license (Unapproved) and trips the Unapproved:0 counter, confirming the RAT policy is strictly AL2.0 (regression vs. the previous setup where the entire AL family, including legacy AL1.0/AL1.1, was approved by default).
• unzip -l flink-kubernetes-operator/target/flink-kubernetes-operator-*-shaded.jar | grep -E "FlinkConfMountDecorator|AggregatedMetric|JobDetailsInfo|Log4j2Plugins\.dat|module-info" → confirms forks present, no module-info.class, single merged Log4j2Plugins.dat.
• All existing operator IT and unit tests continue to pass with no changes.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): yes (build-time only). apache-rat-plugin 0.16.1 → 0.18; cyclonedx-maven-plugin 2.9.0 → 2.9.1; maven-shade-plugin 3.1.1 → 3.6.2; maven-assembly-plugin pinned to 3.8.0 (was 3.2.0 inherited); shade-plugin extension org.apache.logging.log4j:log4j-transform-maven-shade-plugin-extensions:0.2.0 added. No runtime dependency was added, removed, or upgraded, the set of classes on the shaded classpath is unchanged.
• The public API, i.e., is any changes to the CustomResourceDescriptors: no
• Core observer or reconciler logic that is regularly executed: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[gyfora]

This is a somewhat large change that touches the build in a lot of places. Could we please open a jira for it?

[Dennis-Mircea]

This is a somewhat large change that touches the build in a lot of places. Could we please open a jira for it?

Thanks for the review! Sure, I opened FLINK-39565. I also applied a small fix for 2 failing e2e tests. Plase re-run the CI workflow, now it should be all fine.

[gyfora]

I am not sure if it's a good idea to enumerate and exclude the overlapping classes this way. When the overlapping classes are removed from the operator this will silently break the output uber jar. Maybe the warning is better than that scenario :)

[gyfora]

Overall looks good, the comments are a bit verbose but should be okay. My only concern is regarding the exclusion of the overlapping classes. Those come and go and this can lead to silent errors in the future