Cfxtrjtrk

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration:

Ubuntu 18.04.1 LTS

Hadoop  version 3.0.3

Hive 3.1.0

Derby 10.14.2

When I execute the show tables; query I get the following error.

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Following is the detail error in the hive log file.

2018-09-05T11:38:25,952  INFO [main] conf.HiveConf: Found configuration file file:/usr/local/apache-hive-3.1.0-bin/conf/hive-site.xml

2018-09-05T11:38:30,549  INFO [main] SessionState: Hive Session ID = 826ec55c-7fca-4fff-baa5-b5a010e5af89

2018-09-05T11:38:35,948  INFO [main] SessionState:

Logging initialized using configuration in jar:file:/usr/local/apache-hive-3.1.0-bin/lib/hive-common-3.1.0.jar!/hive-log4j2.properties Asy$

2018-09-05T11:38:47,015  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop

2018-09-05T11:38:47,069  INFO [main] session.SessionState: Created local directory: /tmp/mydir

2018-09-05T11:38:47,096  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89

2018-09-05T11:38:47,104  INFO [main] session.SessionState: Created local directory: /tmp/mydir/826ec55c-7fca-4fff-baa5-b5a010e5af89

2018-09-05T11:38:47,122  INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/826ec55c-7fca-4fff-baa5-b5a010e5af89/_$

2018-09-05T11:38:47,125  INFO [main] conf.HiveConf: Using the default value passed in for log id: 826ec55c-7fca-4fff-baa5-b5a010e5af89

2018-09-05T11:38:47,126  INFO [main] session.SessionState: Updating thread name to 826ec55c-7fca-4fff-baa5-b5a010e5af89 main

2018-09-05T11:38:50,476  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.HiveMetaStore: 0: Opening raw store with implementatio$

2018-09-05T11:38:50,695  WARN [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set$

2018-09-05T11:38:50,714  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] metastore.ObjectStore: ObjectStore, initialize called

2018-09-05T11:38:50,717  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file file:/usr/local/apa$

2018-09-05T11:38:50,719  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file hivemetastore-sit$

2018-09-05T11:38:50,720  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null

2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Unable to find config file metastore-site.xml

2018-09-05T11:38:50,722  INFO [826ec55c-7fca-4fff-baa5-b5a010e5af89 main] conf.MetastoreConf: Found configuration file null

hive-site.xml

<property>

    <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:derby:;databaseName=metastore_db;create=true</value>

    <description>

      JDBC connect string for a JDBC metastore.

      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.

      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.

    </description>

  </property>

Environment variables in .profile (I am trying to configure an installation done by someone else so the environment variables are set up in .profile instead of .bashrc even though Hadoop is run manually)

#HIVE

export HIVE_HOME=/usr/local/apache-hive-3.1.0-bin

export HIVE_CONF_DIR=/usr/local/apache-hive-3.1.0-bin/conf

export PATH=$HIVE_HOME/bin:$PATH

export CLASSPATH=$CLASSPATH:/usr/local/hadoop/lib/*:.

export CLASSPATH=$CLASSPATH:/usr/local/apache-hive-3.1.0-bin/lib/*:.



#DERBY

DERBY_HOME=/usr/local/db-derby-10.14.2.0-bin

export PATH=$PATH:$DERBY_HOME/bin

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytool$

The error messages indicates about a metastore-site.xml file which I did not come across the configuration of hive.

I think Hive expect to find Derby database in current working directory. Hive will create the database, because create=true in javax.jdo.option.ConnectionURL property, but will not initialize schema in the database.

My configuration:

• Ubuntu 18.04.1 LTS


• Hadoop 3.1.1


• Hive 3.1.0


• 
  

  Apache Derby from Ubuntu repositories (Version: 10.14.1.0-1ubuntu1 currently)

  
  
  

  sudo apt install derby-tools libderby-java libderbyclient-java

  
  

According Hive Documentation we need to run ShemaTool explicity.

I decide to keep my Derby database into HIVE_HOME (/opt/hive-3.1.0-bin in my case) directory. You can setup Derby database location in hive-site.xml:

<property>

    <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:derby:/opt/hive-3.1.0-bin/metastore_db;databaseName=metastore_db;create=true</value>

</property>

Ensure that there is no Derby database yet:

$ ls $HIVE_HOME/meta*

ls: cannot access '/opt/hive-3.1.0-bin/meta*': No such file or directory

Start Hive:

$ pwd

/home/hadoop

$ hive

hive> show tables;

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

hive> exit;

Now we got Derby database, but it's not functional:

$ ls -d $HIVE_HOME/meta*

/opt/hive-3.1.0-bin/metastore_db

So, we need to create metada schema:

$ rm -rf $HIVE_HOME/metastore_db

$ cd $HIVE_HOME

$ schematool -initSchema -dbType derby

...

Initialization script completed

schemaTool completed

Now Hive is working as expected:

$ cd

$ hive

hive> show tables;

OK

Time taken: 0.803 seconds