[ASTERIXDB-3154] Adds apache iceberg table read support
- user model changes: yes
- Introduced two new DDL parameters “table-format” and “metadata-path” specific to iceberg
- storage format changes: no
- interface changes: no
Details:
These changes add support in AsterixDB to read apache iceberg tables (version 1) as external data.
Iceberg table details can be specified in the external dataset DDL and all queries fetch the data from the
latest iceberg snapshot.
APE link: https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+1%3A+Iceberg+API+Integration
* External Adapters modified: AWS S3 and Hadoop
* Removed kite-sdk dependency and extracted the schema inferrence utilities from kite-sdk
Change-Id: I12df589a6dffdc5af4a5cace68a11729995ea9af
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17419
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
diff --git a/hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml b/hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml
index e5c1d06..6dd822b 100644
--- a/hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml
+++ b/hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml
@@ -104,6 +104,10 @@
<artifactId>nimbus-jose-jwt</artifactId>
</exclusion>
<exclusion>
+ <groupId>org.apache.avro</groupId>
+ <artifactId>avro</artifactId>
+ </exclusion>
+ <exclusion>
<groupId>javax.servlet.jsp</groupId>
<artifactId>jsp-api</artifactId>
</exclusion>
@@ -126,6 +130,10 @@
<artifactId>hadoop-mapreduce-client-core</artifactId>
<exclusions>
<exclusion>
+ <groupId>org.apache.avro</groupId>
+ <artifactId>avro</artifactId>
+ </exclusion>
+ <exclusion>
<groupId>com.sun.jersey.jersey-test-framework</groupId>
<artifactId>jersey-test-framework-grizzly2</artifactId>
</exclusion>