ufukomer / node-impala Goto Github PK

View Code? Open in Web Editor NEW

31.0 5.0 12.0 225 KB

Node Client for Impala using Apache Thrift

Home Page: https://www.npmjs.com/package/node-impala

License: MIT License

JavaScript 20.41% Thrift 79.59%

impala thrift node beeswax node-impala

node-impala's People

Contributors

Stargazers

Watchers

Forkers

deryakopsen harunurhan gdtm86 asdw741111 aleub abdelhamidhenni ofrebourg vjsingh1984 coderade jonapgar szhorizon apapacy

node-impala's Issues

Add ESLint to Travis

Add npm run eslint to Travis CI, when Travis is fixed.

Weird data formatting being returned

Hi,

Thanks again for this piece of software.

When executing the following query:

SELECT 1 as foo, 2 as foo

Data returned is:

[
    [ '1\t2' ],
    [ { name: 'foo', type: 'tinyint', comment: '' }, { name: 'foo', type: 'tinyint', comment: '' } ]
]

I'm not sure i quite understand why 1\t2, i was expecting ['1', '2'].

Is this a bug ?

Regards

Cannot get data of the sql query.

Hi, ufukomer,

I use Node.js to run the code, here is my code,
`var impalaDB = require('node-impala');
const client = impalaDB.createClient();

client.connect({
host: '10.2.18.185',
port: 21050,
resultType: 'json-array'
})
.then(function (message) {
console.log(message); //this output 'Connection established'
})
.catch(function (err) {
console.log(err);
});

client.query('show databases')
.then(result => console.log(result))
.catch(err => console.error(err))
.done(() => client.close().catch(err => console.error(err))); //here output nothing
`

Could you please explain why the query doesn't output anything, result or err? Thank you so much!

Port documentation to ESDoc

Since ESDoc is one of the best options when it comes to generate documentation for ES6 codebase, I think it would be good to have proper documentation format that follows the rules of ESDoc (Syntax is almost same as JSDoc).

Also while porting, types for parameters should be added like described in tutorial.

ESDoc Tutorial

how to set a username?

I find that only host and port can be set, but how to set a username of the impala.

how to get the progress of the query ?

can it get to the progress of the query?

Connecting to remote host with password

Is there a way to connect to a remote host thats user/password protected?

Does node-impala work on Impala with Kerberos enabled

Hi, there,

We have a cluster of Impala daemon nodes which requires Kerberos auth. I tried to use the lib to connect to those nodes and it was not successful. Here is the stack trace,

{ Error: read ECONNRESET at exports._errnoException (util.js:1026:11) at TCP.onread (net.js:572:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }

The stack trace may not help much. I am not sure if it is something related to the Kerberos auth requirement. As least, I don't know how to configure this lib to use GSSAPI to authenticate against the cluster.

So my question is if this lib support GSSAPI or Kerberos for auth mech? If so, how to configure it.

Thanks,
Shuo

Settings for Connection Pool

The Impala driver I need to connect to has a connection pool set up. I need to connect to this pool when using the node-impala client (I need to specify the pool to which I want to connect). How can I do this?

I will appreciate the help and feedback!

Unexpected behaviour using promises on closing

When I use then on connection close it seems not to executed. But the connection seems to be closed and there is no error raised. Is that intentional behaviour?

client.close()
    .then( () => {
       console.log('closed.'); // never logged
       this.closed = true; 
    })
    .catch(err => console.error(err));

My use case: I wanted to keep the connection open while firing requests parallel. Therefore I need the state whether the connection is already open or not. Is there a built in method to do so?

All my queries return only 1024 lines and then stops?

I am connecting to an impala db via node-impala from my node server, but every time I do a query (unless the limit is less than) the result return 1024 line from the db. Even if I SELECT * it returns 1024 lines form a db with over 6 million lines. I understand that beeswax reads 1024 lines at a time when querying, but why does it stop after the first run?

Please please help.
And thank you in advance to whomever gets back to me :)

Node.js Impala client TapplicationException error

Hi ,

I am trying to connect to impala using the package and i'm getting the TapplicationExcpetion error.

In server.js

var impala=require('node-impala');
app.post('/submit_dsp_block',function(req,res){
var day=req.body.day;
var limit=req.body.limit;
console.log(day);
console.log(limit);
console.log(impala);
var client = impala.createClient();
console.log(client);
console.log("trying to connect");
client.connect(
{
host:'10.201.50.11',
port: 21050,
resultType: 'json-array'
});
console.log("client connected");
client.query('select * from rpt.rptdaily limit 100')
.then(result => console.log(result))
.catch(err => console.error(err));
.done(() => client.close().catch(err => console.error(err)));
});

This is the console result in git bash:

node server.js
Example app listening on port 3000!
2016-08-15
100
{ createClient: [Function: createClient] }
ImpalaClient {}
trying to connect
client connected
{ [TApplicationException: Invalid method name: 'query']
name: 'TApplicationException',
message: 'Invalid method name: 'query'',
type: 1 }

Guess my client is getting connected, but there is problem with the query. Also how do i execute prepared query using node-impala , for example something like this

var day=req.body.day;
var limit=req.body.limit;
client.execute('select* from portal.pubdomainstats where day=? limit ?',[day,limit], function (err, result) {
if (err) {
return console.error('There was while trying to retrieve data from system.local', err);
}

I'm pretty new to node.js and would be helpful if I can get a layman's answer to my problem

Thanks

Support TLS/SSL

Hi @ufukomer, thanks for this piece of software !

Would be nice to support TLS/SSL configuration as well

Cf. https://www.cloudera.com/documentation/enterprise/latest/topics/impala_ssl.html
Cf. https://github.com/apache/thrift/blob/c8e0207053a26f206d6515313747b7e1999a01d5/lib/nodejs/lib/thrift/connection.js#L249

TApplicationException: Invalid method name

Using the example there: from https://www.npmjs.com/package/node-impala I got the following error:

{ TApplicationException: Invalid method name: 'query'
    at BeeswaxServiceClient.recv_query ([...]/node_modules/node-impala/lib/thrift/BeeswaxService.js:1527:13)
    at [...]/node_modules/thrift/lib/nodejs/lib/thrift/connection.js:157:41
    at Socket.<anonymous> [...]/node_modules/thrift/lib/nodejs/lib/thrift/buffered_transport.js:48:5)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:172:18)
    at Socket.Readable.push (_stream_readable.js:130:10)
    at TCP.onread (net.js:542:20)
  name: 'TApplicationException',
  message: 'Invalid method name: \'query\'',
  type: 1 }

The connection to the server seems to be established, but I can not execute the query.

query 1024 row limit

Hi,

When I want to paging a query with LIMIT and OFFSET clauses,
I only have 24 rows when LIMIT = 1000 and OFFSET = 1000

the result of request count is 2152.

I think this is an issue.

Invalid method name 'query'

i installed the node-impala by npm and copy to a server,
but cannot query sql.
the query err is: TApplicationException: Invalid method name 'query'

query returns one row only, my code style is es5

Hi ufukomer,

Here ais my es5 codes, it always return the first row of results
i think it may caused by the 'pending' state, but how can i get all 50 rows after the pending finished?



> var sql = "select system_name from itm.system group by system_name";

> var client = require('node-impala').createClient({"host": "hadoop3"});

> client.query(sql, function(err, data){ console.log('err', err, 'data', data)  })

{ state: 'pending' }

> err null data [ [ 'CASCECUP01:KUX' ],

[ { name: 'system_name', type: 'string', comment: '' } ] ]



> client.resultType = 'map'

'map'

> client.query(sql, function(err, data){ console.log('err', err, 'data', data)  })

{ state: 'pending' }

> err null data Map { 'system_name' => [ 'CASCECUP01:KUX' ] }



> var client = require('node-impala').createClient({"host": "hadoop3", "resultType": 'map'})

undefined

> client.query(sql, function(err, data){ console.log('err', err, 'data', data)  })

{ state: 'pending' }

> err null data Map { 'system_name' => [ 'CASCECUP01:KUX' ] }

Please check same query under impala-shell :

Not able to get Arraysize more than 1024 on a query

Hi,
Is there a way where we can increase the query array size,right now its limiting to 1024.