tuplejump / cash Goto Github PK

Hi,

It seems there's no way to specify a username/password when trying to connect to a Cassandra server using the PasswordAuthenticator. When trying to create my EXTERNAL TABLE, i.e.

CREATE EXTERNAL TABLE example (
example_data bigint
)
STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler'
WITH SERDEPROPERTIES ("cassandra.host" = "1.2.3.4")
TBLPROPERTIES ("cassandra.ks.name" = "test", "cassandra.cf.name" = "example");

`
I get this error message:

FAILED: Error in metadata: MetaException(message:Unable to connect to the server InvalidRequestException(why:You have not logged in))
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

My external table works if I change the server back to AllowAllAuthenticator, but unfortunately that's not really an option.

Would it be possible to add username and password to SERDEPROPERTIES? Or is there a different workaround that I'm missing?

Does this plugin also support querying dynamic column families?

Hi,

We download the code and were able to make it work both on a localhost deployment of cassandra and a remote deployment. Thanks a lot of the great piece of work that you have shared, and it has saved us a lot of time and effort.

Please find my questions below on performance.

1. About 275 map jobs are started in hadoop when a simple select count(*) is issued on the hive. This slows down the query enormously if the query is issued on hive on a external table which is located on cassandra. ( about 30 minutes for 150 records)
2. If I create hive table from external cassandra table it is very slow. ( About 30 minutes.

Is there a work around or something to be expected from hive side.

thanks

Sagar

I realize this is probably not an issue that is directly related to this repo, but I'm really at a loss and am hoping someone is willing to help me anyway.

I followed the steps stated within the README.md, copied the target/hive-cassandra-1.2.9.jar, target/dependency/cassandra-all-1.2.9.jar and target/dependency/cassandra-thrift-1.2.9.jar to the hive lib directory, added hive/lib to HIVE_AUX_JARS_PATH and also added HIVE_CLASSPATH=$HIVE_HOME/lib/*.jar to the hadoop user's .bashrc

After that I was able to create an external table with command:
CREATE EXTERNAL TABLE service.feeds(asset_id int, created_date string, created_timestamp string, type_id int, value string) STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler' WITH SERDEPROPERTIES ("cassandra.host"="external.cassandra.host", "cassandra.port"="9160", "cql.primarykey"="asset_id, created_date", "caching"="all") TBLPROPERTIES ("cassandra.ks.name"="service", "cassandra.cf.name"="feeds", "cassandra.ks.strategy"="SimpleStrategy");

resulting in:
OK
Time taken: 0.758 seconds

When I query that table, I get an exception within the main thread upon which hive crashes:

hive>select * from service.feeds limit 10;
OK
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/ArrayList;
at org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:73)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:471)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:561)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:534)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:137)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1519)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:285)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

I am not really sure where it is going wrong, so could someone please help me out?

Cassandra version: 2.0.9
Hive version: 0.13.1
Hadoop version: 1.2.1