arempter / hive-metastore-docker Goto Github PK

docker-compose build
[+] Building 0.9s (7/10)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/openjdk:8u242-jre 0.5s
=> [1/6] FROM docker.io/library/openjdk:8u242-jre@sha256:f437249498cd370546035d6dc73e03a00ec637bcf47b740e60283b2fa7e00fb4 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 137B 0.0s
=> CACHED [2/6] WORKDIR /opt 0.0s
=> ERROR [3/6] RUN curl -L https://www-us.apache.org/dist/hive/hive-standalone-metastore-3.0.0/hive-standalone-metastore-3.0.0-bin.tar.gz | tar zxf - 0.3s

[3/6] RUN curl -L https://www-us.apache.org/dist/hive/hive-standalone-metastore-3.0.0/hive-standalone-metastore-3.0.0-bin.tar.gz | tar zxf - && curl -L https://archive.apache.org/dist/hadoop/common/hadoop-3.2.0/hadoop-3.2.0.tar.gz | tar zxf - && curl -L https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-8.0.19.tar.gz | tar zxf - && cp mysql-connector-java-8.0.19/mysql-connector-java-8.0.19.jar /opt/apache-hive-metastore-3.0.0-bin/lib/ && rm -rf mysql-connector-java-8.0.19:
#6 0.296 % Total % Received % Xferd Average Speed Time Time Time Current
#6 0.298 Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: www-us.apache.org
#6 0.326
#6 0.326 gzip: stdin: unexpected end of file
#6 0.331 /bin/tar: Child returned status 1
#6 0.332 /bin/tar: Error is not recoverable: exiting now

failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c curl -L https://www-us.apache.org/dist/hive/hive-standalone-metastore-${METASTORE_VERSION}/hive-standalone-metastore-${METASTORE_VERSION}-bin.tar.gz | tar zxf - && curl -L https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz | tar zxf - && curl -L https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-8.0.19.tar.gz | tar zxf - && cp mysql-connector-java-8.0.19/mysql-connector-java-8.0.19.jar ${HIVE_HOME}/lib/ && rm -rf mysql-connector-java-8.0.19]: exit code: 2

Hive crash when triying to create tables at start and print this error

Error: Table 'CTLGS' already exists (state=42S01,code=1050)

Closing: 0: jdbc:mysql://192.168.5.249:3306/hive?createDatabaseIfNotExist=true

org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

Underlying cause: java.io.IOException : Schema script failed, errorcode 2

org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:594)

at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)

at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:323)

at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

Caused by: java.io.IOException: Schema script failed, errorcode 2

at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1226)

at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1204)

at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:590)

... 8 more

*** schemaTool failed ***

Attaching my log for review. kindly guide
$ docker-compose up
Recreating hive-metastore-docker-master_mariadb_1 ...
Starting hive-metastore-docker-master_minio_1 ...
Starting hive-metastore-docker-master_minio_1 ... done
Recreating hive-metastore-docker-master_mariadb_1 ... done
Recreating hive-metastore-docker-master_hive-metastore_1 ...
Recreating hive-metastore-docker-master_hive-metastore_1 ... done
Attaching to hive-metastore-docker-master_minio_1, hive-metastore-docker-master_mariadb_1, hive-metastore-docker-master_hive-metastore_1
mariadb_1 | 2021-04-09 18:16:24+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+mariafocal started.
mariadb_1 | 2021-04-09 18:16:24+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mariadb_1 | 2021-04-09 18:16:24+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.9+mariafocal started.
mariadb_1 | 2021-04-09 18:16:25 0 [Note] mysqld (mysqld 10.5.9-MariaDB-1:10.5.9+mariafocal) starting as process 1 ...
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Uses event mutexes
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Number of pools: 1
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mariadb_1 | 2021-04-09 18:16:25 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Using Linux native AIO
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: 128 rollback segments are active.
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: 10.5.9 started; log sequence number 448658; transaction id 626
mariadb_1 | 2021-04-09 18:16:25 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadb_1 | 2021-04-09 18:16:25 0 [Note] Server socket created on IP: '::'.
mariadb_1 | 2021-04-09 18:16:25 0 [Warning] 'proxies_priv' entry '@% root@a1661c140fd9' ignored in --skip-name-resolve mode.
mariadb_1 | 2021-04-09 18:16:25 0 [Note] InnoDB: Buffer pool(s) load completed at 210409 18:16:25
mariadb_1 | 2021-04-09 18:16:25 0 [Note] Reading of all Master_info entries succeeded
mariadb_1 | 2021-04-09 18:16:25 0 [Note] Added new Master_info '' to hash table
mariadb_1 | 2021-04-09 18:16:25 0 [Note] mysqld: ready for connections.
mariadb_1 | Version: '10.5.9-MariaDB-1:10.5.9+mariafocal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
minio_1 | Endpoint: http://172.18.0.2:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Browser Access:
minio_1 | http://172.18.0.2:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Object API (Amazon S3 compatible):
minio_1 | Go: https://docs.min.io/docs/golang-client-quickstart-guide
minio_1 | Java: https://docs.min.io/docs/java-client-quickstart-guide
minio_1 | Python: https://docs.min.io/docs/python-client-quickstart-guide
minio_1 | JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
minio_1 | .NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
minio_1 | IAM initialization complete
hive-metastore_1 | SLF4J: Class path contains multiple SLF4J bindings.
hive-metastore_1 | SLF4J: Found binding in [jar:file:/opt/apache-hive-metastore-3.0.0-bin/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
hive-metastore_1 | SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
hive-metastore_1 | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
hive-metastore_1 | SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
hive-metastore_1 | Metastore connection URL: jdbc:mysql://mariadb:3306/metastore_db
hive-metastore_1 | Metastore Connection Driver : com.mysql.cj.jdbc.Driver
hive-metastore_1 | Metastore connection User: admin
hive-metastore_1 | Starting metastore schema initialization to 3.0.0
hive-metastore_1 | Initialization script hive-schema-3.0.0.mysql.sql
hive-metastore_1 | 1/506 -- MySQL dump 10.13 Distrib 5.5.25, for osx10.6 (i386)
hive-metastore_1 | 2/506 --
hive-metastore_1 | 3/506 -- Host: localhost Database: test
hive-metastore_1 | 4/506 -- ------------------------------------------------------
hive-metastore_1 | 5/506 -- Server version 5.5.25
hive-metastore_1 | 6/506
hive-metastore_1 | 7/506 /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT /;
hive-metastore_1 | No rows affected (0.013 seconds)
hive-metastore_1 | 8/506 /!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 9/506 /!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 10/506 /!40101 SET NAMES utf8 /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 11/506 /!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 12/506 /!40103 SET TIME_ZONE='+00:00' /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 13/506 /!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 /;
hive-metastore_1 | No rows affected (0.003 seconds)
hive-metastore_1 | 14/506 /!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 15/506 /!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 16/506 /!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 17/506
hive-metastore_1 | 18/506 --
hive-metastore_1 | 19/506 -- Table structure for table BUCKETING_COLS
hive-metastore_1 | 20/506 --
hive-metastore_1 | 21/506
hive-metastore_1 | 22/506 /!40101 SET @saved_cs_client = @@character_set_client /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 23/506 /!40101 SET character_set_client = utf8 /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 24/506 CREATE TABLE IF NOT EXISTS BUCKETING_COLS (
hive-metastore_1 | SD_ID bigint(20) NOT NULL,
hive-metastore_1 | BUCKET_COL_NAME varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
hive-metastore_1 | INTEGER_IDX int(11) NOT NULL,
hive-metastore_1 | PRIMARY KEY (SD_ID,INTEGER_IDX),
hive-metastore_1 | KEY BUCKETING_COLS_N49 (SD_ID),
hive-metastore_1 | CONSTRAINT BUCKETING_COLS_FK1 FOREIGN KEY (SD_ID) REFERENCES SDS (SD_ID)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 25/506 /!40101 SET character_set_client = @saved_cs_client /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 26/506
hive-metastore_1 | 27/506 --
hive-metastore_1 | 28/506 -- Table structure for table CDS
hive-metastore_1 | 29/506 --
hive-metastore_1 | 30/506
hive-metastore_1 | 31/506 /!40101 SET @saved_cs_client = @@character_set_client /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 32/506 /!40101 SET character_set_client = utf8 /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 33/506 CREATE TABLE IF NOT EXISTS CDS (
hive-metastore_1 | CD_ID bigint(20) NOT NULL,
hive-metastore_1 | PRIMARY KEY (CD_ID)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 34/506 /!40101 SET character_set_client = @saved_cs_client /;
hive-metastore_1 | No rows affected (0.004 seconds)
hive-metastore_1 | 35/506
hive-metastore_1 | 36/506 --
hive-metastore_1 | 37/506 -- Table structure for table COLUMNS_V2
hive-metastore_1 | 38/506 --
hive-metastore_1 | 39/506
hive-metastore_1 | 40/506 /!40101 SET @saved_cs_client = @@character_set_client /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 41/506 /!40101 SET character_set_client = utf8 /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 42/506 CREATE TABLE IF NOT EXISTS COLUMNS_V2 (
hive-metastore_1 | CD_ID bigint(20) NOT NULL,
hive-metastore_1 | COMMENT varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
hive-metastore_1 | COLUMN_NAME varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
hive-metastore_1 | TYPE_NAME MEDIUMTEXT DEFAULT NULL,
hive-metastore_1 | INTEGER_IDX int(11) NOT NULL,
hive-metastore_1 | PRIMARY KEY (CD_ID,COLUMN_NAME),
hive-metastore_1 | KEY COLUMNS_V2_N49 (CD_ID),
hive-metastore_1 | CONSTRAINT COLUMNS_V2_FK1 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
hive-metastore_1 | No rows affected (0.003 seconds)
hive-metastore_1 | 43/506 /!40101 SET character_set_client = @saved_cs_client /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 44/506
hive-metastore_1 | 45/506 --
hive-metastore_1 | 46/506 -- Table structure for table DATABASE_PARAMS
hive-metastore_1 | 47/506 --
hive-metastore_1 | 48/506
hive-metastore_1 | 49/506 /!40101 SET @saved_cs_client = @@character_set_client /;
hive-metastore_1 | No rows affected (0.001 seconds)
hive-metastore_1 | 50/506 /!40101 SET character_set_client = utf8 /;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 51/506 CREATE TABLE IF NOT EXISTS DATABASE_PARAMS (
hive-metastore_1 | DB_ID bigint(20) NOT NULL,
hive-metastore_1 | PARAM_KEY varchar(180) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
hive-metastore_1 | PARAM_VALUE varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
hive-metastore_1 | PRIMARY KEY (DB_ID,PARAM_KEY),
hive-metastore_1 | KEY DATABASE_PARAMS_N49 (DB_ID),
hive-metastore_1 | CONSTRAINT DATABASE_PARAMS_FK1 FOREIGN KEY (DB_ID) REFERENCES DBS (DB_ID)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
hive-metastore_1 | No rows affected (0.003 seconds)
hive-metastore_1 | 52/506 /!40101 SET character_set_client = @saved_cs_client */;
hive-metastore_1 | No rows affected (0.002 seconds)
hive-metastore_1 | 53/506
hive-metastore_1 | 54/506 CREATE TABLE CTLGS (
hive-metastore_1 | CTLG_ID BIGINT PRIMARY KEY,
hive-metastore_1 | NAME VARCHAR(256),
hive-metastore_1 | DESC VARCHAR(4000),
hive-metastore_1 | LOCATION_URI VARCHAR(4000) NOT NULL,
hive-metastore_1 | UNIQUE KEY UNIQUE_CATALOG (NAME)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
hive-metastore_1 | Warning: Table 'CTLGS' already exists (state=42S01,code=1050)
hive-metastore_1 | java.sql.SQLWarning: Table 'CTLGS' already exists
hive-metastore_1 | at com.mysql.cj.protocol.a.NativeProtocol.convertShowWarningsToSQLWarnings(NativeProtocol.java:2065)
hive-metastore_1 | at com.mysql.cj.jdbc.StatementImpl.getWarnings(StatementImpl.java:1733)
hive-metastore_1 | at sqlline.Commands.execute(Commands.java:849)
hive-metastore_1 | at sqlline.Commands.sql(Commands.java:733)
hive-metastore_1 | at sqlline.SqlLine.dispatch(SqlLine.java:795)
hive-metastore_1 | at sqlline.SqlLine.runCommands(SqlLine.java:1706)
hive-metastore_1 | at sqlline.Commands.run(Commands.java:1317)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
hive-metastore_1 | at sqlline.SqlLine.dispatch(SqlLine.java:791)
hive-metastore_1 | at sqlline.SqlLine.initArgs(SqlLine.java:595)
hive-metastore_1 | at sqlline.SqlLine.begin(SqlLine.java:643)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.runSqlLine(MetastoreSchemaTool.java:1034)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.runSqlLine(MetastoreSchemaTool.java:1007)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.doInit(MetastoreSchemaTool.java:596)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.doInit(MetastoreSchemaTool.java:574)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.run(MetastoreSchemaTool.java:1273)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.main(MetastoreSchemaTool.java:1178)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
hive-metastore_1 | Error: Table 'CTLGS' already exists (state=42S01,code=1050)
hive-metastore_1 | java.sql.SQLSyntaxErrorException: Table 'CTLGS' already exists
hive-metastore_1 | at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
hive-metastore_1 | at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
hive-metastore_1 | at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
hive-metastore_1 | at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:764)
hive-metastore_1 | at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648)
hive-metastore_1 | at sqlline.Commands.execute(Commands.java:823)
hive-metastore_1 | at sqlline.Commands.sql(Commands.java:733)
hive-metastore_1 | at sqlline.SqlLine.dispatch(SqlLine.java:795)
hive-metastore_1 | at sqlline.SqlLine.runCommands(SqlLine.java:1706)
hive-metastore_1 | at sqlline.Commands.run(Commands.java:1317)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
hive-metastore_1 | at sqlline.SqlLine.dispatch(SqlLine.java:791)
hive-metastore_1 | at sqlline.SqlLine.initArgs(SqlLine.java:595)
hive-metastore_1 | at sqlline.SqlLine.begin(SqlLine.java:643)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.runSqlLine(MetastoreSchemaTool.java:1034)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.runSqlLine(MetastoreSchemaTool.java:1007)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.doInit(MetastoreSchemaTool.java:596)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.doInit(MetastoreSchemaTool.java:574)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.run(MetastoreSchemaTool.java:1273)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.main(MetastoreSchemaTool.java:1178)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
hive-metastore_1 | Aborting command set because "force" is false and command failed: "CREATE TABLE CTLGS (
hive-metastore_1 | CTLG_ID BIGINT PRIMARY KEY,
hive-metastore_1 | NAME VARCHAR(256),
hive-metastore_1 | DESC VARCHAR(4000),
hive-metastore_1 | LOCATION_URI VARCHAR(4000) NOT NULL,
hive-metastore_1 | UNIQUE KEY UNIQUE_CATALOG (NAME)
hive-metastore_1 | ) ENGINE=InnoDB DEFAULT CHARSET=latin1;"
hive-metastore_1 | Closing: com.mysql.cj.jdbc.ConnectionImpl
hive-metastore_1 | Schema initialization FAILED! Metastore state would be inconsistent !!
hive-metastore_1 | Underlying cause: java.io.IOException : Schema script failed, errorcode OTHER
hive-metastore_1 | Use --verbose for detailed stacktrace.
hive-metastore_1 | *** schemaTool failed ***
hive-metastore_1 | 2021-04-09 18:17:04,507 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore_1 | 2021-04-09 18:17:04,524 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore_1 | 2021-04-09 18:17:04,554 shutdown-hook-0 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger
hive-metastore_1 | 2021-04-09 18:17:04: Starting Metastore Server
hive-metastore_1 | SLF4J: Class path contains multiple SLF4J bindings.
hive-metastore_1 | SLF4J: Found binding in [jar:file:/opt/apache-hive-metastore-3.0.0-bin/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
hive-metastore_1 | SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
hive-metastore_1 | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
hive-metastore_1 | SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
hive-metastore_1 | 2021-04-09 18:17:10,977 main INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore_1 | MetaException(message:Got exception: java.io.FileNotFoundException Bucket spark does not exist)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8541)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8536)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8806)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8723)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
hive-metastore_1 | Caused by: MetaException(message:Got exception: java.io.FileNotFoundException Bucket spark does not exist)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:163)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:114)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:138)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:144)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:157)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultCatalog(HiveMetaStore.java:723)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:755)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:525)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:80)
hive-metastore_1 | ... 11 more
hive-metastore_1 | Exception in thread "main" MetaException(message:Got exception: java.io.FileNotFoundException Bucket spark does not exist)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:84)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8541)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8536)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8806)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8723)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
hive-metastore_1 | at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
hive-metastore_1 | Caused by: MetaException(message:Got exception: java.io.FileNotFoundException Bucket spark does not exist)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:163)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:114)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:138)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:144)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:157)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultCatalog(HiveMetaStore.java:723)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:755)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:525)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
hive-metastore_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
hive-metastore_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
hive-metastore_1 | at java.lang.reflect.Method.invoke(Method.java:498)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
hive-metastore_1 | at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:80)
hive-metastore_1 | ... 11 more
hive-metastore_1 | 2021-04-09 18:17:17,312 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore_1 | 2021-04-09 18:17:17,318 shutdown-hook-0 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
hive-metastore_1 | 2021-04-09 18:17:17,336 shutdown-hook-0 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger
mariadb_1 | 2021-04-09 18:17:17 24 [Warning] Aborted connection 24 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 18 [Warning] Aborted connection 18 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 20 [Warning] Aborted connection 20 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 19 [Warning] Aborted connection 19 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 21 [Warning] Aborted connection 21 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 22 [Warning] Aborted connection 22 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 23 [Warning] Aborted connection 23 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 17 [Warning] Aborted connection 17 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 15 [Warning] Aborted connection 15 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 14 [Warning] Aborted connection 14 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 16 [Warning] Aborted connection 16 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 13 [Warning] Aborted connection 13 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 12 [Warning] Aborted connection 12 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 11 [Warning] Aborted connection 11 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 10 [Warning] Aborted connection 10 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 6 [Warning] Aborted connection 6 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 5 [Warning] Aborted connection 5 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 9 [Warning] Aborted connection 9 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 8 [Warning] Aborted connection 8 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
mariadb_1 | 2021-04-09 18:17:17 7 [Warning] Aborted connection 7 to db: 'metastore_db' user: 'admin' host: '172.18.0.4' (Got an error reading communication packets)
hive-metastore-docker-master_hive-metastore_1 exited with code 1
Stopping hive-metastore-docker-master_mariadb_1 ...
Stopping hive-metastore-docker-master_minio_1 ...
Stopping hive-metastore-docker-master_mariadb_1 ... done
Stopping hive-metastore-docker-master_minio_1 ... done
Gracefully stopping... (press Ctrl+C again to force)

when i use this repo on windows, comand pull ok, but up container dont work. do you have any idea this result.
container mino and maria start up good but hive dont work and hay start after and dont generate error but dont work!

thanks

Hi,

thanks for this great work!

It did not run on my M2 Mac due to the usage of a non-ARM compatible Java8 base image, therefore I wanted to share my solution for others:

Dockerfile:

#FROM openjdk:8u242-jre <- comment this out and add next line
FROM arm64v8/openjdk:8
..

The bucket spark was not existing in minio initially and also its web UI was not reachable (because of its listening port being randomly chosen), therefore I added following tweaks taken from:

• https://stackoverflow.com/questions/70329649/docker-compose-minio-switches-port-when-viewed-in-browser
• https://stackoverflow.com/questions/66412289/minio-add-a-public-bucket-with-docker-compose

docker-compose.yml:

version: "2"

services:
  mariadb:
    image: mariadb:latest
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: admin
      MYSQL_USER: admin
      MYSQL_PASSWORD: admin
      MYSQL_DATABASE: metastore_db

  # make sure that you specify correct volume to be mounted
  minio:
    image: minio/minio
    environment:
      - MINIO_ACCESS_KEY=accesskey
      - MINIO_SECRET_KEY=secretkey
    volumes:
      - ./data1/minio:/data
    ports:
      - 9000:9000
      - 9001:9001
    command: server --address ":9000" --console-address ":9001" /data

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set myminio http://minio:9000 accesskey secretkey;
      /usr/bin/mc mb myminio/spark;
      /usr/bin/mc policy set public myminio/spark;
      exit 0;
      "

  hive-metastore:
    build: .
    image: hive-metastore:latest
    ports:
    - 9083:9083
    depends_on:
      - mariadb