Attribute shaded Maven classes to their dependency packages

[chinyeungli]

Issues

• Closes: maven-heaven: Correlate relocated classes in shaded JARs to their originating packages #2247

Changes

Maven Shade relocates dependency classes under a new package prefix, so they have no matching source in the development codebase. These classes were previously flagged as missing sources, which is misleading because they come from a dependency.

This PR parses the main POM for maven-shade-plugin entries and reverses them to recover the original class names. Then, it matches each unmatched .class against the Maven dependency packages using the groupId, artifactId, and the class's package path (with a scoring system). If the match meets the required score, it will then assign the matched resource's status as "shaded-class".

Notes

Classes that cannot be matched unambiguously are intentionally left as requires-review rather than guessed.

Checklist

• I have read the contributing guidelines
• I have linked an existing issue above
• I have added unit tests covering the new code
• I have reviewed and understood every line of this PR

[tdruez]

@chinyeungli The relocation parsing makes sense, but the scoring runs on every unmapped .class of the to/ side, not only on the shaded ones. Unmapped classes of the main project get attributed to any dependency from the same group. For example, org.apache.hadoop.mapreduce.Job matches pkg:maven/org.apache.hadoop/hadoop-common with a score of 5 and is flagged as shaded-class, which is worse than leaving it as requires-review.

• What about limiting the matching to the classes under a shadedPattern when relocations are available, and excluding the main package namespace otherwise?
• In which cases are the dependency DiscoveredPackage present in the project? Dependencies declared in the POM are created as DiscoveredDependency. The tests create those packages directly, so I'm not sure this path runs on a real shaded JAR.
• How were the weights and MIN_MATCH_SCORE = 5 chosen? Were they tested on shaded JARs other than htrace-core?
• The new step is never executed by the test suite: test_scanpipe_scan_maven_package_single_file runs without D2D, and the step only runs with D2D enabled. We still need a pipeline test with D2D, ideally on a real shaded JAR, so the relocation reversal is covered too (test_scanpipe_maven_map_shaded_classes_attributions creates no main POM).
• Missing tests for: the POM parsing failure in get_maven_shade_relocations, the "no POM found" case in get_main_maven_pom, and the early return without dependency packages in map_shaded_classes_to_maven_packages.

See my inline comments for the rest.

[chinyeungli]

What about limiting the matching to the classes under a shadedPattern when relocations are available, and excluding the main package namespace otherwise?

It is now handled in _get_original_fqn() that check if relocations are available and only work on classes under shadedPattern.

In which cases are the dependency DiscoveredPackage present in the project? Dependencies declared in the POM are created as DiscoveredDependency. The tests create those packages directly, so I'm not sure this path runs on a real shaded JAR.

Right. I updated the code and tests.

How were the weights and MIN_MATCH_SCORE = 5 chosen? Were they tested on shaded JARs other than htrace-core?

It's chosen by the following minimum scenario:

• a second matching artifact segment (1 group + 2 artifact = 1 + 4 = 5)
• two extra matching group segments (3 group + 1 artifact = 3 + 2 = 5)
• a fuzzy match (1 group + 1 artifact + fuzzy = 1 + 2 + 3 = 6)

I also checked pkg:maven/org.apache.activemq/artemis-commons@2.31.1 and the score 5 seems to work.
Note that this package not only using shadedPattern, but also artifactSet for shaded files

<artifactSet>
   <includes>
      <include>org.apache.johnzon:johnzon-core</include>
      <include>jakarta.json:jakarta.json-api</include>
   </includes>
</artifactSet>

https://repo1.maven.org/maven2/org/apache/activemq/artemis-commons/2.31.1/artemis-commons-2.31.1.pom

Created #2251 for follow up.

The new step is never executed by the test suite: test_scanpipe_scan_maven_package_single_file runs without D2D, and the step only runs with D2D enabled. We still need a pipeline test with D2D, ideally on a real shaded JAR, so the relocation reversal is covered too (test_scanpipe_maven_map_shaded_classes_attributions creates no main POM).

Added test_scanpipe_scan_maven_package_d2d_shaded_classes, and it uses the real shaded-pom.xml

Missing tests for: the POM parsing failure in get_maven_shade_relocations, the "no POM found" case in get_main_maven_pom, and the early return without dependency packages in map_shaded_classes_to_maven_packages.

I think I have these covered.