Comments (8)
Hello, I've tested this here: https://github.com/beltran/gohive/pull/121/files and tests are passing (https://travis-ci.org/github/beltran/gohive/builds/738294406) so I haven't been able to reproduce (I'm getting the expected result x.a: 1 x.b:2 y.a: nil y.b:nil
)
Could you verify you are indeed getting a:nil b.nil
, also gohive simple surfaces whatever is coming from hive so I'm not sure how this could be happening, what happening when you execute "select * from t as x left join t as y on x.a=y.b"
with other clients like beehive?
from gohive.
My reproduce steps:
- start spark thrift server
./sbin/start-thriftserver.sh - use go hive to execute sql
drop table if exists t
create table t(a int, b int)
insert into t values(1,2)
select * from t as x left join t as y on x.a=y.b
the result of RowMap
is :
map[a:<nil> b:<nil>]
3. The expected result is
- use beeline:
+----+----+-------+-------+--+
| a | b | a | b |
+----+----+-------+-------+--+
| 1 | 2 | NULL | NULL |
+----+----+-------+-------+--+
- use PyHive:
[(1, 2, None, None)]
from gohive.
from gohive.
Hello, thank you for the repro script, I see now what the problem is but I don't see a solution with RowMap to this but print a warning that two columns have the same name (spark returns the raw column but hive returns tableName + columnName).
Your best option is to use FetchOne
, you can specify interfaces if you don't want to write down all the variables and it's type like it's done here if it's more convenient.
from gohive.
Thanks for your reply. The combination cursor.Description()
and FetchOne
is a good option to achieve it.
from gohive.
The problem is that RowMap
returns a map from TableName.ColumnName
to the values so if TableName
is not available from cursor.Description
there's no way to build this map. Also since several columns share the same name we'd need more than one key for the two columns in the map.
I see PyHive doesn't run into this because they return an array, for example, in the script you provided it returns [(1, 2, None, None)]
from gohive.
When using FetchOne
, it can't get null
values. The reproduce script is as follows:
cursor.Exec(ctx,"drop table if exists t")
cursor.Exec(ctx,"create table t(a int,b double ,c char(10),d date)")
cursor.Exec(ctx,"insert into t values(null,null,null,null)")
cursor.Exec(ctx, "select * from t")
desc := cursor.Description()
columnValues := make([]interface{}, len(desc))
cursor.FetchOne(ctx, columnValues...)
fmt.Println(columnValues)
The result is:
[0 0 ]
But the expected result is:
[nil nil nil nil]
from gohive.
If you care about nil values there are some docs here https://github.com/beltran/gohive#null-values, but the bottom line is that you won't be able to use
columnValues := make([]interface{}, len(desc))
you'll have to define all the pointers and then pass them to FetchOne
:
var a *int32 = new(int32)
var b *float64 = new(float64)
var c *string = new(string)
var d *string = new(string)
cursor.FetchOne(ctx, a, b, c, d)
from gohive.
Related Issues (20)
- Exec PENDING_STATE HOT 2
- go build error HOT 1
- some problems to access to hiveserver2 without kinit HOT 2
- Cursor.totalRows is 0
- panic: runtime error: invalid memory address or nil pointer dereference HOT 1
- How to solve "Incorrect FrameSize"? HOT 6
- could we generate cpp bindings
- What to do if the hive service is actively disconnected due to timeout? HOT 5
- Incorrect frame size HOT 6
- ConnectZookeeper panic HOT 1
- use in hive 1.1.0 fetch data uncompleted HOT 1
- hive-site.xml location HOT 1
- panic: runtime error: invalid memory address or nil pointer dereference HOT 1
- Assignment Mismatch | Thrift HOT 5
- pollUntilData func panic HOT 1
- how to specify mote than one host HOT 1
- I want to connect to the Hive database with KerberOS authentication enabled on Windows. How to install GSSAPI C library on Windows? HOT 1
- HTTP(S) plain/ldap authentication connection does not support cookies and issue with the cookie handling HOT 1
- Query never sent to server if it contains set statements HOT 3
- Should I connect to metastore once an operation? Is it necessary to use connection pool? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gohive.