Git Product home page Git Product logo

Comments (21)

beltran avatar beltran commented on August 30, 2024 1

Then you should be using NONE: connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration). You may get a different error now but hopefully it will be more clear

from gohive.

beltran avatar beltran commented on August 30, 2024

I'd assume there's something incorrectly set in hive, have you set the setting hive.server2.authentication = NOSASL? Can you connect using the beeline? For example with beeline -u "jdbc:hive2://hs2.example.com:10000"

from gohive.

yzhengwei avatar yzhengwei commented on August 30, 2024

I didn't set hive.server2.authentication = NOSASL, but I can connect to the hive use beeline

image

from gohive.

yzhengwei avatar yzhengwei commented on August 30, 2024

I'm using connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration) , then I can connected to the hive, thanks very much! I have another question, how do I set the database name in the connect ?

from gohive.

beltran avatar beltran commented on August 30, 2024

Happy to hear it's working, you should be able to do it with: cursor.Execute(context.Background(), "USE database", false) and then query tables in that database with the same cursor, like here for example.

from gohive.

yzhengwei avatar yzhengwei commented on August 30, 2024

The problem is perfectly solved. Thanks very much!

from gohive.

beltran avatar beltran commented on August 30, 2024

Sure! Happy to hear.

from gohive.

yzhengwei avatar yzhengwei commented on August 30, 2024
package main

import (
	"github.com/beltran/gohive"
	"log"
)

func main() {
	configuration := gohive.NewConnectConfiguration()
	configuration.Service = "hive"
	configuration.FetchSize = 1000
	connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration)

	if errConn != nil {
		log.Fatal(errConn)
	}

	connection.Close()
}

I was using CenOS 7, but I found that the program could only run use root, and when I was running with a regular user, it returned Bad SASL negotiation status: 3 (Error validating the login), I don't know what happend?

image

from gohive.

beltran avatar beltran commented on August 30, 2024

You shouldn't need to use root to run it. What error do you get when you run with out sudo? Also is hive listening at 10.21.89.89:10000? Also checking the hive logs to this would be useful.

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

hello! I'm also getting an "EOF" error from just a show databases query
Already checked my hive-site.xml, I'm using NONE for authentication.

Heres my code:

conf :=gohive.NewConnectConfiguration()
conf.Username = "default"
sess, err := gohive.Connect("localhost", 10000, "NONE", conf)
if err != nil {
	fmt.Println("sess err")
	fmt.Println(err)
}
cursor := sess.Cursor()
ctx := context.Background()
sess.Cursor().Exec(ctx, "show databases")
if cursor.Err != nil {
	fmt.Println("query err")
	log.Fatal(cursor.Err)
}

var s string
for cursor.HasMore(ctx) {
	cursor.FetchOne(ctx, &s)
	if cursor.Err != nil {
		fmt.Println("res err")
		log.Fatal(cursor.Err)
	}
	log.Println(s)
}

Heres my hive-site.xml:

<configuration>

  <property>
    <name>hive.support.concurrency</name>
    <value>true</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:postgresql://postgres:5432/metastore</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>org.postgresql.Driver</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>postgres</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>postgres</value>
  </property>
  <property>
    <name>datanucleus.autoCreateSchema</name>
    <value>true</value>
  </property>
  <property>
    <name>datanucleus.autoCreateTables</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.metastore.uris</name>
    <value>thrift://localhost:9083</value>
  </property>
  <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
  </property>
  <property>
    <name>hive.server2.webui.port</name>
    <value>10002</value>
  </property>
  <property>
    <name>hive.metastore.schema.verification</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.zookeeper.quorum</name>
    <value>zookeeper</value>
  </property>
    <property>
    <name>hive.server2.authentication</name>
    <value>NONE</value>
    <description>
      Client authentication types.
      NONE: no authentication check
      LDAP: LDAP/AD based authentication
      KERBEROS: Kerberos/GSSAPI authentication
      CUSTOM: Custom authentication provider
      (Use with property hive.server2.custom.authentication.class)
      PAM: Pluggable authentication module.
    </description>
  </property>
</configuration>

Heres the hive-server2 logs for the query:

2020-12-05 14:19:21,674 ERROR [HiveServer2-Handler-Pool: Thread-52]: server.TThreadPoolServer (TThreadPoolServer.java:run(295)) - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

I'm using Hive 1.1.0-cdh5.11.1, looks like there is a match in thrift protocol? Is there a way to fix this?

from gohive.

beltran avatar beltran commented on August 30, 2024

Hello, you're getting this error at gohive.Connect right? Can you connect with beehive? If so what's the command you issue to connect? If you have connect with beehive can you run SET hive.server2.authentication to verify the authentication you're setting is NONE

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

ah sorry I forgot to add earlier, its actually when receiving the results, at cursor.FetchOne(ctx, &s)

Trying with the beeline that came with the hive version works, but using a newer version of beeline gives this error.

Error: Could not establish connection to jdbc:hive2://ip:10000/default: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) (state=08S01,code=0)

using hive-jdbc 1.1.0 from remote works too, jdbc:hive2://ip:10000

I suspect its a compatibility issue between the client and server? but unfortunately I am unable to change the server hive version. Looks similar to: https://stackoverflow.com/questions/24694415/required-field-client-protocol-is-unset

Thanks for your help!

from gohive.

beltran avatar beltran commented on August 30, 2024

I've used an old thrift file previous to when the regression was introduced with the driver: https://github.com/beltran/gohive/tree/HIVE-6050. You could install this and give it a try. I haven't tested it, but the changes were pretty straightforward, you can see the last commit of that branch. The only relevant changes are in hive.go.

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

Thanks for the quick update! however it looks like it is still having the same problems.. maybe still incompatible?

2020-12-05 14:19:21,674 ERROR [HiveServer2-Handler-Pool: Thread-52]: server.TThreadPoolServer (TThreadPoolServer.java:run(295)) - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?

If it helps, there is a docker with hive 1.1.0-cdh5.11.1 here: https://github.com/parrot-stream/docker-hive/tree/1.1.0-cdh5.11.1

from gohive.

beltran avatar beltran commented on August 30, 2024

I haven't been able to make it work I believe because I don't have the externally linked hadoop and zookeeper images. If you provide me with access to your hive instance I'll try to make it work, you can send it to my email.

I think the hive version used in that cloudera bundle is apache 1.10. I've made the changes for the thrift file of that version here: https://github.com/beltran/gohive/tree/APACHE-1.10, can you give them a try?

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

Thanks for the help so far! Unfortunately it is not working and I am unable to provide you access to the instance. The docker images for external hadoop and zookeeper are available here:
https://github.com/parrot-stream/docker-zookeeper/tree/3.5.4
https://github.com/parrot-stream/docker-hadoop/tree/2.6.0-cdh5.11.1

A slight configuration can be done to put them on the same network in their docker-compose.yml

networks:
  default:
    external:
      name: parrot_default

from gohive.

beltran avatar beltran commented on August 30, 2024

I intend to try this, I haven't had the time yet.

from gohive.

beltran avatar beltran commented on August 30, 2024

I've successfully started 1.1.0-cdh5.11.1 but was able to connect with the driver in the current state, more or less this is what I've done:

docker create network parrot_default
git checkout https://github.com/parrot-stream/docker-hadoop/ && pushd docker-hadoop &&  docker-compose -p parrot up && popd
git checkout https://github.com/parrot-stream/docker-zookeeper/ && pushd docker-zookeeper &&  docker-compose -p parrot up && popd
git checkout https://github.com/parrot-stream/docker-hive/ && pushd docker-hive && git checkout 1.1.0-cdh5.11.1 &&  docker-compose -p parrot up && popd

Then change KERBEROS to NONE here.

Then run example/main.go:

go run example/main.go

And the scripts finishes successfully. I can give you access to the hive instance I've set up.

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

that is strange. Are you able to run a simple query like "Show tables" and retrieve results?

from gohive.

beltran avatar beltran commented on August 30, 2024

Yes, I can insert, "show tables", and read the results.

from gohive.

yilun-zhu avatar yilun-zhu commented on August 30, 2024

It seems that we have followed the same steps, yet I am encountering an error. Let me check again on this, maybe there is something wrong with my environment. Thanks for your help!

from gohive.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.