Git Product home page Git Product logo

database_loader's Introduction

Database Loader

Python Loader to upload files to database using pandas interface

database_loader is a module to load information to a database table. It identify the field from the header fields, so those fields are used to map the table columns

Supported Databases

  • Oracle
  • MySql

Formats supported

  • EXCEL
  • CSV
  • TSV

Instalation

You can install database_loader by cloning this Git repository

python -m pip install git+https://github.com/marinellirubens/database_loader#egg=database_loader==1.0.34

In order for database_loader to work you will have to install the appropriate database driver.

  • Oracle: cx-Oracle version 7.0.0+
  • Mysql: mysql-connector-python version 8.0.0+

You can install this drivers via pip:

python -m pip install cx-Oracle
python -m pip install mysql-connector-python

Usage

$ database_loader -h
usage: __main__.py [-h] [-c COMMIT] [-t TABLE] [-v] [-l] [-d DATABASE] [-f FILE_LOAD] [-u USER] [-p PASSWORD]
                   [-T TYPE] [-V]

Parses command.

optional arguments:
  -h, --help            show this help message and exit
  -c COMMIT, --commit COMMIT
                        Commit every X lines.
  -t TABLE, --table TABLE
                        Table name.
  -v, --verbose         Prints information.
  -l, --clean           Clean table before inserts.
  -d DATABASE, --database DATABASE
                        Database TNS.
  -f FILE_LOAD, --file_load FILE_LOAD
                        File to be load on the table
  -u USER, --user USER  Database User
  -p PASSWORD, --password PASSWORD
                        Database Password
  -T TYPE, --type TYPE  Types CSV, EXCEL, TSV
  -V, --version         Show version
  -B DATABASE_TYPE, --database_type DATABASE_TYPE
                        Database type [ORACLE, MYSQL]
  -C CONNECTION_TYPE, --connection_type CONNECTION_TYPE
                        Connection type [TNS, STRING]
  -H HOST, --host HOST  Database host
  -P PORT, --port PORT  Database port

Process finished with exit code 0

How to use database_loader

Check Version

$ database_loader --version
v1.0.34

Loading files on database by TNSNAMES(ORACLE Only)

Executed on windows

$ database_loader --database TESTDB ^
                  --user root ^
                  --password"123456" ^
                  --table tb_salary ^
                  --file_load C:\insert_files\insert.tsv ^
                  --commit 5000 ^
                  --type TSV ^
                  --database_type ORACLE ^
                  --connection_type TNS ^
                  --clean

Executed on linux

$ database_loader --database TESTDB \
                  --user root \
                  --password"123456" \
                  --table tb_salary \
                  --file_load C:\insert_files\insert.tsv \
                  --commit 5000 \
                  --type TSV \
                  --database_type ORACLE \
                  --connection_type TNS \
                  --clean

Loading files on Oracle database by Connection String

Executed on windows

$ database_loader --connection STRING ^
                  --database TESTDB ^
                  --host 192.168.15.168 ^
                  --port 1601 ^
                  --user root ^
                  --password"123456" ^
                  --database_type ORACLE ^
                  --table tb_salary ^
                  --file_load C:\insert_files\insert.tsv ^
                  --commit 5000 ^
                  --type TSV ^
                  --clean

Executed on linux

$ database_loader --connection STRING \
                  --database TESTDB \
                  --host 192.168.15.168 \
                  --port 1601 \
                  --user root \
                  --database_type ORACLE \
                  --password"123456" \
                  --table tb_salary \
                  --file_load /insert_files/insert.tsv \
                  --commit 5000 \
                  --type TSV \
                  --clean

database_loader's People

Contributors

marinellirubens avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.