[HIVE-30027] Upgrade avro from 1.12.0 to 1.12.2 vul fix

[Akanksha-kedia]

What changes were proposed in this pull request?

Bumps the avro.version property from 1.12.0 to 1.12.2 — a patch-level release within the same minor line.

Why are the changes needed?

Picks up upstream Avro bug fixes from the 1.12.x line. All modules in this repo declare the bare avro/avro-mapred artifacts with no version override, relying on the root pom's dependencyManagement via ${avro.version}, so this single property change covers the whole build consistently.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Verified no module declares an independent avro version that would need to stay in sync (checked standalone-metastore/pom.xml specifically, which has its own copies of some other version properties, but not avro). Could not run a full Maven build in my environment to compile-verify.

[Akanksha-kedia]

Found and fixed the Jenkins failure (AvroBytesConverterTest), but flagging a bigger implication for review.

Root cause: Avro's 1.12.x line added org.apache.avro.util.ClassSecurityValidator, which closes an unrestricted-deserialization CVE class by refusing to resolve any class as a SpecificRecord unless explicitly trusted via org.apache.avro.SERIALIZABLE_CLASSES/SERIALIZABLE_PACKAGES. 1.12.0 didn't enforce this; 1.12.2 does by default. AvroBytesConverterTest hit this because KafkaAvroSerializer.serialize() (Confluent) internally builds a SpecificDatumWriter, which calls SpecificData.getClass() to resolve the test's SimpleRecord fixture class — and that's now forbidden by default.

Test fix pushed: trust the test's own fixture class via System.setProperty("org.apache.avro.SERIALIZABLE_CLASSES", SimpleRecord.class.getName()) in @BeforeClass, scoped to this test only.

Bigger concern: KafkaSerDe.AvroBytesConverter.getWritable() (the production deserialization path, not just this test) also constructs a SpecificDatumReader, which goes through the identical SpecificData.getClass() check. That means any real user querying Kafka Avro data via a schema that resolves to a generated SpecificRecord class (the standard pattern with Confluent Schema Registry + code-generated Avro classes) will hit the same SecurityException in production after this bump — not just in this test.

I deliberately did not try to work around that in production code, since picking a trust boundary there (e.g. blanket-trusting a package) is a real design/security decision that risks reintroducing the exact class of vulnerability this Avro change closes, and affects real users' arbitrary class names — not something to decide unilaterally in a version-bump PR. Wanted to surface it explicitly so a maintainer can weigh in on whether this needs a migration note, a Hive-side default-trust mechanism, or should hold until that's resolved.

@ayushtkn @uros-b flagging for your take given the security angle.

[Akanksha-kedia]

Correction: my first fix (`System.setProperty(...)` in `@BeforeClass`) did not actually work — Jenkins re-ran and hit the identical `SecurityException` at the shifted line number, confirming it. `ClassSecurityValidator` reads its trusted-classes allowlist once when it's first loaded, not per-call, and something in the same Surefire fork can trigger that load before `@BeforeClass` runs — so setting it at test runtime is too late regardless of which method it's in.

Fixed properly now: moved the trust declaration into `kafka-handler/pom.xml`'s own `maven-surefire-plugin` `systemPropertyVariables`, so it's present as a JVM property from fork startup. Reverted the ineffective test-file change.

(Separately, the `TestLlapZookeeperRegistryImpl` failure in the same run is under Jenkins' own "Existing failures" bucket — pre-existing, unrelated to this change.)

[Akanksha-kedia]

cc @abstractdog

[Akanksha-kedia]

Rebased on latest master, fresh CI run triggered. @ayushtkn @uros-b @abstractdog could you take a look, especially the production-path SecurityException concern flagged above?

[ayushtkn]

Not very straight forward, do you have idea what introduced this? I am even not sure if this is only test only fix and blast during runtime, did you reproduced it locally and verified it works by this change? I doubt

[Akanksha-kedia]

What introduced this: Avro 1.12.x added org.apache.avro.util.ClassSecurityValidator, which as of 1.12.2 rejects resolving any class as SpecificRecord unless explicitly trusted via org.apache.avro.SERIALIZABLE_CLASSES/SERIALIZABLE_PACKAGES. AvroBytesConverterTest hits this because KafkaAvroSerializer.serialize() builds a SpecificDatumWriter, which resolves this test's SimpleRecord fixture — now forbidden by default.

Reproduced locally: reverting this systemPropertyVariables change and rerunning AvroBytesConverterTest fails deterministically with:

java.lang.SecurityException: Forbidden org.apache.hadoop.hive.kafka.SimpleRecord! This class is not trusted...
  at org.apache.avro.specific.SpecificDatumWriter.<init>
  at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl

With it restored, mvn -pl kafka-handler test -Dtest=AvroBytesConverterTest passes 5/5.

To your "test-only" question — it's not. KafkaSerDe.AvroBytesConverter.getWritable() (the production deserialization path) builds a SpecificDatumReader, going through the identical SpecificData.getClass() check. Any real user querying Kafka Avro data via Confluent Schema Registry + code-generated SpecificRecord classes will hit the same SecurityException at runtime after this bump — this test just happens to be what caught it in CI. I didn't add a production-side trust workaround because picking that trust boundary is a real security decision (risks reintroducing the CVE this Avro change closes) and affects arbitrary user class names — not something to decide unilaterally in a version-bump PR. Wanted your take on whether this needs a Hive-side default-trust mechanism, a migration note, or should block until resolved.

[Akanksha-kedia]

@ayushtkn — what introduced this: Avro 1.12.x added org.apache.avro.util.ClassSecurityValidator, which as of 1.12.2 rejects resolving any class as SpecificRecord unless explicitly trusted via org.apache.avro.SERIALIZABLE_CLASSES/SERIALIZABLE_PACKAGES. AvroBytesConverterTest hits this because KafkaAvroSerializer.serialize() builds a SpecificDatumWriter, which resolves the test's SimpleRecord fixture — now forbidden by default.

Reproduced locally just now: reverting the maven-surefire-plugin systemPropertyVariables change and rerunning AvroBytesConverterTest fails deterministically with:

java.lang.SecurityException: Forbidden org.apache.hadoop.hive.kafka.SimpleRecord! This class is not trusted...
  at org.apache.avro.specific.SpecificDatumWriter.<init>
  at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl

With the property restored, mvn -pl kafka-handler test -Dtest=AvroBytesConverterTest passes 5/5.

To your point — this is not just a test-only fix. KafkaSerDe.AvroBytesConverter.getWritable() (production deserialization path) builds a SpecificDatumReader, which goes through the identical SpecificData.getClass() check. So any real user querying Kafka Avro data via Confluent Schema Registry + code-generated SpecificRecord classes will hit the same SecurityException at runtime after this bump — the test just happens to be the thing that caught it in CI. I didn't add a production-side trust workaround because picking that trust boundary (e.g. blanket-trusting a package) is a real security decision that risks reintroducing the CVE this Avro change closes, and affects arbitrary user class names — not something to decide unilaterally in a version-bump PR. Wanted your take on whether this needs a Hive-side default-trust mechanism, a migration note, or should block until resolved.

[Akanksha-kedia]

Found and fixed another instance of the same root cause, and confirmed the TestHBaseSerDe diff floating around isn't actually needed — details below.

itests/qtest — TestMiniHiveKafkaCliDriver (kafka_storage_handler.q): QTestMiniClusters#getAvroRows() calls new SpecificDatumWriter<>(Wikipedia.getClassSchema()), which goes through the identical SpecificData.getForSchema() → getClass() path already fixed for SimpleRecord in AvroBytesConverterTest — same SecurityException, this time for org.apache.hive.kafka.Wikipedia. Fixed the same way: maven-surefire-plugin systemPropertyVariables in itests/qtest/pom.xml (JVM-startup property, not @BeforeClass — already shown unreliable for this exact failure mode earlier in this thread). Pushed.

Caveat: I could not run this specific test end-to-end locally — building the full itests profile reactor here pulls in hive-hcatalog, whose legacy test deps (commons-httpclient:3.1, jetty:6.1.26) are blocked by a local network policy unrelated to this change. Confident in the fix by code-path analysis and by direct analogy to the already-reproduced SimpleRecord case, but flagging that I'm relying on Jenkins for the actual green check here rather than a local run.

TestHBaseSerDe: ran it unmodified against avro 1.12.2 — 17/17 pass, no SecurityException, no trust property needed. It uses new SpecificDatumWriter<Employee>(Employee.class) (class-direct constructor), which never goes through the schema-name→class resolution path that ClassSecurityValidator guards. So the @BeforeClass/SERIALIZABLE_PACKAGES change proposed for it isn't fixing a real problem, and carries the same reliability risk we already ruled out for AvroBytesConverterTest. Didn't apply it.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud