Comments (21)
Then you should be using NONE
: connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration)
. You may get a different error now but hopefully it will be more clear
from gohive.
I'd assume there's something incorrectly set in hive, have you set the setting hive.server2.authentication = NOSASL
? Can you connect using the beeline? For example with beeline -u "jdbc:hive2://hs2.example.com:10000"
from gohive.
I didn't set hive.server2.authentication = NOSASL, but I can connect to the hive use beeline
from gohive.
I'm using connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration)
, then I can connected to the hive, thanks very much! I have another question, how do I set the database name in the connect ?
from gohive.
Happy to hear it's working, you should be able to do it with: cursor.Execute(context.Background(), "USE database", false)
and then query tables in that database with the same cursor, like here for example.
from gohive.
The problem is perfectly solved. Thanks very much!
from gohive.
Sure! Happy to hear.
from gohive.
package main
import (
"github.com/beltran/gohive"
"log"
)
func main() {
configuration := gohive.NewConnectConfiguration()
configuration.Service = "hive"
configuration.FetchSize = 1000
connection, errConn := gohive.Connect("10.21.89.89", 10000, "NONE", configuration)
if errConn != nil {
log.Fatal(errConn)
}
connection.Close()
}
I was using CenOS 7, but I found that the program could only run use root, and when I was running with a regular user, it returned Bad SASL negotiation status: 3 (Error validating the login)
, I don't know what happend?
from gohive.
You shouldn't need to use root to run it. What error do you get when you run with out sudo
? Also is hive listening at 10.21.89.89:10000
? Also checking the hive logs to this would be useful.
from gohive.
hello! I'm also getting an "EOF" error from just a show databases
query
Already checked my hive-site.xml, I'm using NONE
for authentication.
Heres my code:
conf :=gohive.NewConnectConfiguration()
conf.Username = "default"
sess, err := gohive.Connect("localhost", 10000, "NONE", conf)
if err != nil {
fmt.Println("sess err")
fmt.Println(err)
}
cursor := sess.Cursor()
ctx := context.Background()
sess.Cursor().Exec(ctx, "show databases")
if cursor.Err != nil {
fmt.Println("query err")
log.Fatal(cursor.Err)
}
var s string
for cursor.HasMore(ctx) {
cursor.FetchOne(ctx, &s)
if cursor.Err != nil {
fmt.Println("res err")
log.Fatal(cursor.Err)
}
log.Println(s)
}
Heres my hive-site.xml:
<configuration>
<property>
<name>hive.support.concurrency</name>
<value>true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:postgresql://postgres:5432/metastore</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.postgresql.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>postgres</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>postgres</value>
</property>
<property>
<name>datanucleus.autoCreateSchema</name>
<value>true</value>
</property>
<property>
<name>datanucleus.autoCreateTables</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://localhost:9083</value>
</property>
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
</property>
<property>
<name>hive.server2.webui.port</name>
<value>10002</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>true</value>
</property>
<property>
<name>hive.zookeeper.quorum</name>
<value>zookeeper</value>
</property>
<property>
<name>hive.server2.authentication</name>
<value>NONE</value>
<description>
Client authentication types.
NONE: no authentication check
LDAP: LDAP/AD based authentication
KERBEROS: Kerberos/GSSAPI authentication
CUSTOM: Custom authentication provider
(Use with property hive.server2.custom.authentication.class)
PAM: Pluggable authentication module.
</description>
</property>
</configuration>
Heres the hive-server2 logs for the query:
2020-12-05 14:19:21,674 ERROR [HiveServer2-Handler-Pool: Thread-52]: server.TThreadPoolServer (TThreadPoolServer.java:run(295)) - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I'm using Hive 1.1.0-cdh5.11.1, looks like there is a match in thrift protocol? Is there a way to fix this?
from gohive.
Hello, you're getting this error at gohive.Connect
right? Can you connect with beehive? If so what's the command you issue to connect? If you have connect with beehive can you run SET hive.server2.authentication
to verify the authentication you're setting is NONE
from gohive.
ah sorry I forgot to add earlier, its actually when receiving the results, at cursor.FetchOne(ctx, &s)
Trying with the beeline that came with the hive version works, but using a newer version of beeline gives this error.
Error: Could not establish connection to jdbc:hive2://ip:10000/default: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default}) (state=08S01,code=0)
using hive-jdbc 1.1.0 from remote works too, jdbc:hive2://ip:10000
I suspect its a compatibility issue between the client and server? but unfortunately I am unable to change the server hive version. Looks similar to: https://stackoverflow.com/questions/24694415/required-field-client-protocol-is-unset
Thanks for your help!
from gohive.
I've used an old thrift file previous to when the regression was introduced with the driver: https://github.com/beltran/gohive/tree/HIVE-6050. You could install this and give it a try. I haven't tested it, but the changes were pretty straightforward, you can see the last commit of that branch. The only relevant changes are in hive.go.
from gohive.
Thanks for the quick update! however it looks like it is still having the same problems.. maybe still incompatible?
2020-12-05 14:19:21,674 ERROR [HiveServer2-Handler-Pool: Thread-52]: server.TThreadPoolServer (TThreadPoolServer.java:run(295)) - Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
If it helps, there is a docker with hive 1.1.0-cdh5.11.1 here: https://github.com/parrot-stream/docker-hive/tree/1.1.0-cdh5.11.1
from gohive.
I haven't been able to make it work I believe because I don't have the externally linked hadoop and zookeeper images. If you provide me with access to your hive instance I'll try to make it work, you can send it to my email.
I think the hive version used in that cloudera bundle is apache 1.10. I've made the changes for the thrift file of that version here: https://github.com/beltran/gohive/tree/APACHE-1.10, can you give them a try?
from gohive.
Thanks for the help so far! Unfortunately it is not working and I am unable to provide you access to the instance. The docker images for external hadoop and zookeeper are available here:
https://github.com/parrot-stream/docker-zookeeper/tree/3.5.4
https://github.com/parrot-stream/docker-hadoop/tree/2.6.0-cdh5.11.1
A slight configuration can be done to put them on the same network in their docker-compose.yml
networks:
default:
external:
name: parrot_default
from gohive.
I intend to try this, I haven't had the time yet.
from gohive.
I've successfully started 1.1.0-cdh5.11.1 but was able to connect with the driver in the current state, more or less this is what I've done:
docker create network parrot_default
git checkout https://github.com/parrot-stream/docker-hadoop/ && pushd docker-hadoop && docker-compose -p parrot up && popd
git checkout https://github.com/parrot-stream/docker-zookeeper/ && pushd docker-zookeeper && docker-compose -p parrot up && popd
git checkout https://github.com/parrot-stream/docker-hive/ && pushd docker-hive && git checkout 1.1.0-cdh5.11.1 && docker-compose -p parrot up && popd
Then change KERBEROS
to NONE
here.
Then run example/main.go:
go run example/main.go
And the scripts finishes successfully. I can give you access to the hive instance I've set up.
from gohive.
that is strange. Are you able to run a simple query like "Show tables" and retrieve results?
from gohive.
Yes, I can insert, "show tables", and read the results.
from gohive.
It seems that we have followed the same steps, yet I am encountering an error. Let me check again on this, maybe there is something wrong with my environment. Thanks for your help!
from gohive.
Related Issues (20)
- LDAP authentication with HTTPS Does Not Work HOT 8
- Exec PENDING_STATE HOT 2
- go build error HOT 1
- some problems to access to hiveserver2 without kinit HOT 2
- Cursor.totalRows is 0
- panic: runtime error: invalid memory address or nil pointer dereference HOT 1
- How to solve "Incorrect FrameSize"? HOT 6
- could we generate cpp bindings
- What to do if the hive service is actively disconnected due to timeout? HOT 5
- Incorrect frame size HOT 6
- ConnectZookeeper panic HOT 1
- use in hive 1.1.0 fetch data uncompleted HOT 1
- hive-site.xml location HOT 1
- panic: runtime error: invalid memory address or nil pointer dereference HOT 1
- Assignment Mismatch | Thrift HOT 5
- pollUntilData func panic HOT 1
- how to specify mote than one host HOT 1
- I want to connect to the Hive database with KerberOS authentication enabled on Windows. How to install GSSAPI C library on Windows? HOT 1
- HTTP(S) plain/ldap authentication connection does not support cookies and issue with the cookie handling HOT 1
- Query never sent to server if it contains set statements HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gohive.