appliedinfrastructure / cassandra-snapshot-tools Goto Github PK

It will be good to add SSL , keystore and truststore support for the sstableloader options. Or you can pick up from the Cassandra.yaml as well

We hit a confusing bug recently where getSnapshot was printing No data directories, or no cassandra.yaml file found on a new Cassandra installation, even though all our paths were the same as on prior working installations, and our cassandra.yaml looked the same. What had happened was that the YAML had been autoformatted, setting the indentation to our usual in-house value of 2 spaces. It turns out that the awk-based YAML quasi-parser in getSnapshot didn't correctly read the data directories line when it was only indented by two. 3 or 4 worked fine, 2 or 1 didn't. I'd try to offer a fix but don't have enough awk skills to feel safe modifying this.

Failing snippet of cassandra.yaml:

data_file_directories:
  - /var/lib/cassandra/data

$ ./getSnapshot -k the_keyspace -s the_snapshot
No data directories, or no cassandra.yaml file found

Working snippet of cassandra.yaml:

data_file_directories:
    - /var/lib/cassandra/data

$ ./getSnapshot -k the_keyspace -s the_snapshot
Using provided snapshot name the_snapshot
...

Need a friendly way to use SSL in clusters that enforce SSL client connections

I believe there is a slight error on the tar command:

tar --directory "$DUMPDIR" \
            -zcvf $FILENAME \
                  $KEYSPACE \
                  $SCHEMA \
                  $KEYSPFILE \
                  $SNAPSFILE \
                  $HOSTSFILE \
                  $DATESFILE >/dev/null 2>&1

$KEYSPACE should not be there, I think.

When I run the script Im seeing the following error. I have placed the script in /Users/ksachdeva/dse/bin directory. Any idea ?

Kunals-Mac:bin ksachdeva$ ./getSnapshot -k admin -s admin.backup -y ../resources/cassandra/conf/cassandra.yaml
Error finding required executables: cqlsh

May be a simple diagram will make it easy to understand as how it works.

Currently the NetworkTopologyStrategy and its options are assumed (because that's what we currently use) - need to support (and test for) SimpleStrategy as well.

Do you have the plan to support the new version of Cassandra? When I run it on Cassandra 3.11.2, putSnapshot gives lots of errors.

diff getSnapshot.orig getSnapshot
169c169
< for directory in ${data_file_directories_[@]}; do

        for directory in ${data_file_directories[@]}; do

ie. seems to be a trailing underscore. Works for me with it removed.

Need to support authentication for clusters that use it.

Need a simple way to view details about a snapshot file. For example, it would be nice to be able to view the datacenter/replication factor configuration in a given snapshot.

On a node where more than one data_file_directory is defined in cassandra.yaml, the getSnapshot script searches for the appropriate keyspace in all data_file_directory entries, but uses only the first directory containing the keyspace and snapshot. This will likely not include all of the necessary SSTables required to successfully restore the keyspace.

Need to implement a loop that searches in all data file directories for the keyspace and snapshot, and includes them in the snapshot dump.