Git Product home page Git Product logo

takenoko's Introduction

#Takenoko ##Description Rails gem for getting data from Google spreedsheet then export to files or database via simple interface and rake tasks

##What's new in versions 0.1.1

  • Simpler config, easier to use by using spreadsheet as config
  • Add validator and filter for each row
  • Allow authorization by both service account and personal account
  • Allow skip rows, worksheets by '#'
  • Fix minor bugs

##Install

1.Gemfile

gem "takenoko"

2.Generate initializer

#Generate config/initializers/takenoko.rb
rails generate takenoko:config

Update config/initializers/takenoko.rb

conf.sheet_id = "SPREADSHEET_ID"    

Other global options, you can overwrite these options in mapping file for each table

Option Require Default Description Value Overwrite(*)
google_cridential_file Optional Only use when using service account authorizaion. If not set, Takenoko will use persional cridential auto matically and as you for permission at the first time via command line String No
mapping_file Optional Use when you want customize exporting, overwrite default setting for each tables, if not set takenoko will export all worksheet except skipped one.All tables must be listed here String No
sheet_id Optional Spreadsheet id, optain via url of your spreadsheet,IMPORTANT, Must set this value here or in mapping file String Yes
file_extension Optional csv Export file format csv,yaml,json Yes
export_file_location Optional db/spreadsheet Location for exported files String Yes
truncate_all_data Optional false Truncate table before saving data bool Yes
allow_overwrite Optional true Overwrite duplicated row bool Yes
enable_postprocess Optional false Enable validator and filter bool Yes
postprocess_class Optional nil Class for post processing, nil for Class = table class name String Yes

You can refer HERE for how to get cridential_file when using service account permission

"*": Allow overwrite in mapping_file.yaml

###3. Create mapping_file.yaml

tables:
  table1:
    worksheet_id: #
    columns_mapping:
      table_col_1:spread_sheet_colA
      table_col_2:spread_sheet_colB
      table_col_3:spread_sheet_colC

  table2:
    spreadsheet_id: Example
...

Tables: Hash of tables

Table option

Option Require Default Description Value
worksheet_id Optional Worksheet gid Integer
worksheet Optional Work sheet name Use to find worksheet when worksheet_id is not set String
columns_mapping Optional Use when worksheet column difference from database column, if spreadsheet be set to false, the column will be skipped Hash
table_name Optional key name Table name String
class_name Optional singular camel form of table_name Model class name String
find_column Optional id By default takenoko use worksheet row number as id of row, you have to set this option for find and replace duplicated rows String
sheet_id
file_extension
export_file_location
truncate_all_data
allow_overwrite
enable_postprocess
postprocess_class
Optional Overwrite global config

##Usage

Google spreadsheet format

First row will be header definition(Use # to skip rows)

name code price description ...
Beer 1 1000 Super beer ...
Drug 2 9999 Super drug ...
... ... ... ... ...

Takenoko usage

For all tables

Takenoko.all_to_db
Takenoko.all_to_yaml
...    

For single table

Takenoko.table_to_db(table_name)
Takenoko.table_to_yaml(table_name)

Rake task

For all tables

rake takenoko:all_to_db
rake takenoko:all_to_yaml
...    

For single table

rake takenoko:table_to_db[table_name]
rake takenoko:table_to_yaml[table_name]

Valation and filtering

Takenoko uses three method for post processing. Defined them in your postprocess_class. By default, postprocess_class will be class_name in mapping_file.yml

  • spreadsheet_row_valid?(row) => bool : skip invalid rows (Optional)
  • postprocess_spreadsheet_row(row) => row : modify row, return processed row(Optional)
  • postprocess_spreadsheet_table(table) => row : modify whole dowloaded table, return processed table(Optional)

Skipping worksheet and column

Just put '#' before your worksheet name or column name to skip it

takenoko's People

Contributors

khiem-nguyen avatar khiemns54 avatar

Watchers

James Cloos avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.