[deprecated] Data Warehouse Assessment Extraction Client

This repository is no longer maintained. Assessment Extraction Client has been moved to the dwh-migration-tools repository as part of BigQuery Migration Service. Please follow the BigQuery Migration Assessment documentation for more details on how to use a new tool.

This tool allows to extract meta information from a data warehouse that allows to make an assessment for migration.

License

Copyright 2021 Google LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Building

Install bazel:

sudo apt-get update
sudo apt-get install bazel

Build an executable of the extraction client:

Make sure the teradata driver jar file terajdbc4.jar is in the same directory of the ExtractionTool_deploy.jar.

Run the extraction tool with the teradata driver

java -cp ExtractionTool_deploy.jar:terajdbc4.jar com/google/cloud/bigquery/dwhassessment/extractiontool/ExtractionTool td-extract --db-address jdbc:teradata://localhost/DBS_PORT=1025,DATABASE=dbc --output tmp  --db-user dbc --db-password dbc --schema-filter db:dbc --skip-sql-scripts users

Extraction tool user guide

The extraction tool is currently intended for approved users that are engaging with GCP technical sales teams.

Step 1: Follow the user guide from the GCP technical sales team and download the Extraction tool binary (i.e., ExtractionTool_deploy.jar) and its run script.

Step 2: Download Teradata JDBC driver into the same directory with the Extraction tool binary.

Step 3: Create a folder for output files and run the Extraction tool:

./run-td-extract.sh -j <terajdbc driver> --db-address <database address> --output <output path> --db-user <db user>

You can also append "--" to specify optional parameters, as

./run-td-extract.sh -j <terajdbc driver> --db-address <database address> --output <output path> --db-user <db user> -- --<optional parameter> <parameter value>

See Optional parameters section for details about those parameters.

Alternatively, ensure that your terajdbc driver is exactly named terajdbc4.jar and run

./dwh-assessment-execution-tool.sh td-extract --db-address <database address> --output <output path> --db-user <db user> --<optional parameter> <parameter value>

Optional Parameters:

--schema-filter The schema filter to apply when extracting schemas from the database. By default, all schemas will be extracted.

• Example usage: --schema-filter db:(abc|def),table:public_. Only take schemas from tables in the database abc or from tables whose names have the prefix public_.
• Multiple filters can be defined by repeating the option. Each filter has to match (i.e. AND logic).

--sql-scripts The list of scripts to execute. By default, all available scripts will be executed.

• Available scripts extract metadata from tables and views:
  • AllTempTablesVX
  • All_RI_ChildrenV
  • All_RI_ParentsV
  • ColumnsV
  • DatabasesV
  • DiskSpaceV
  • FunctionsV
  • IndicesV
  • PartitioningConstraintsV
  • DBQLObjTbl
  • QryLogV
  • QryLogSQLV
  • RoleMembersV
  • StatsV
  • TablesV
  • TableSizeV
  • TableTextV
  • UsersV
• Click to view the scripts.

--skip-sql-scripts The list of scripts to skip. By default, all available scripts will be executed.

For additional options to control running, apply filtering, skip schema extraction, etc., please run

./dwh-assessment-extraction-tool.sh td-extract -h

Step 4: The extraction process may take from a few minutes to hours to finish, depending on the amount of data in your database. Share the output files with the PSO Cloud Consultant.