Hive 29869 beeline size reduce - #6748
Conversation
asf-ci-hive
added
tests pending
tests unstable
tests passed
and removed
tests pending
tests unstable
labels
Aggarwal-Raghav
left a comment
Aggarwal-Raghav
left a comment
There was a problem hiding this comment.
Also I can still see the kerby and curator classes in beeline-standalone.jar !! Is this expected?
jar tvf beeline-standalone.jar | grep -i "curator" | head -n 10
0 Fri Feb 01 00:00:00 IST 1980 META-INF/maven/org.apache.curator/
0 Fri Feb 01 00:00:00 IST 1980 META-INF/maven/org.apache.curator/curator-recipes/
68 Fri Feb 01 00:00:00 IST 1980 META-INF/maven/org.apache.curator/curator-recipes/pom.properties
3783 Fri Feb 01 00:00:00 IST 1980 META-INF/maven/org.apache.curator/curator-recipes/pom.xml
0 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/
0 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/framework/
0 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/framework/recipes/
1770 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/framework/recipes/AfterConnectionEstablished$1.class
1817 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/framework/recipes/AfterConnectionEstablished.class
0 Fri Feb 01 00:00:00 IST 1980 org/apache/curator/framework/recipes/atomic/
jar tvf beeline-standalone.jar | grep -i "kerby" | head -n 10
0 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/
0 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/
0 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/
2987 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/KrbInputStream.class
1971 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/KrbOutputStream.class
0 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/ccache/
7645 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/ccache/CredCacheInputStream.class
6456 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/ccache/CredCacheOutputStream.class
8287 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/ccache/Credential.class
12508 Sun Dec 11 19:18:10 IST 2022 org/apache/hadoop/shaded/org/apache/kerby/kerberos/kerb/ccache/CredentialCache.class
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.thrift</groupId> |
There was a problem hiding this comment.
Why this is excluded? there is direct dependency in beeline module
There was a problem hiding this comment.
It's true, but that's just a clean up as thrift is pulled transitively.
| </exclusion> | ||
| <exclusion> | ||
| <groupId>org.apache.kerby</groupId> | ||
| <artifactId>kerb-util</artifactId> |
There was a problem hiding this comment.
maybe
<exclusion>
<groupId>org.apache.kerby</groupId>
<artifactId>*</artifactId>
</exclusion>
There was a problem hiding this comment.
updated
|
| <groupId>org.apache.hive</groupId> | ||
| <artifactId>hive-standalone-metastore-server</artifactId> | ||
| <version>${standalone-metastore.version}</version> | ||
| <scope>provided</scope> |
There was a problem hiding this comment.
why provided? isn't it only needed for tests?
There was a problem hiding this comment.
Not exactly. It's needed for SchemaTool. And that's is the part of open questions. Do we want native beeline installer to contain the SchemaTool in it? It can't be used anyways but it's the part of beeline module as of now. So I suggested to remove that dependency from compile time if the SchemaTool distribution remains the same and native beeline can go on its own.
There was a problem hiding this comment.
SchemaTool?? Could you please check if it's even possible to trigger schema-tool from beeline?? beeline is an HS2 client and not supposed to init/recreate DB schema
There was a problem hiding this comment.
To what I see no. SchemaTool has it's own main method entry point and Beeline doesn't reference SchemaTool at any place. Also cli client only refers to beeline. For some reason SchemaTool is just inside beeline module. However they don't have anything in common to what I can judge.
| </exclusion> | ||
| <!-- kerby/kerberos auth chain pulled in transitively via hadoop-auth; beeline | ||
| does not exercise hadoop-common's Kerberos/UGI code paths --> | ||
| <exclusion> |
There was a problem hiding this comment.
do we package optional deps?
<artifactId>hadoop-common</artifactId>
<optional>true</optional>
There was a problem hiding this comment.
Yes they are packaged.
| <arg value="${project.build.directory}/apache-hive-beeline-${project.version}/${installer.arch}"/> | ||
| <!-- Reduces the size of the output artifact --> | ||
| <arg value="--add-modules"/> | ||
| <arg value="java.base,java.compiler,java.desktop,java.instrument,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql.rowset,jdk.httpserver,jdk.jfr,jdk.management,jdk.security.auth,jdk.unsupported"/> |
There was a problem hiding this comment.
does this list come from
jdeps \
--multi-release 21 \
--ignore-missing-deps \
--print-module-deps \
beeline-standalone.jar
?
There was a problem hiding this comment.
Yes, this command is the source for the list
4 participants
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?