Git Product home page Git Product logo

tap-airtable's Introduction

tap-airtable

tap-airtable is a Singer tap for Airtable usable with Meltano.

Built with the Meltano Tap SDK for Singer Taps.

Installation

Standalone

You can install tap-airtable using pipx to install it globally on your system:

pipx install tap-airtable

With Meltano

If you're using this tap with Meltano, you can install it as an extractor. The tap is the default extractor for Airtable on MeltanoHub and can be installed using the following command:

meltano add extractor tap-airtable

You can also install the extractor directly from the source.

meltano add extractor tap-airtable --from-ref https://raw.githubusercontent.com/tomasvotava/tap-airtable/master/tap-airtable.yml

Configuration

Accepted Config Options

Property Required Description
token Yes Airtable API token
base_ids No List of base IDs to extract data from (if not given, all bases are extracted)
table_mapping No Mapping of Airtable table ids to target table names (if not given table names from Airtable are used)

If table_mapping is not provided, the tap will use the table names from Airtable as target table names. This may not be ideal if the table names are not human-readable or if they contain special characters.

Standalone Configuration

{
    "token": "<your airtable token>",
    "base_ids": ["<base_id_1>", "<base_id_2>"],
    "table_mapping": {
        "<table_id_1>": "target_table_name_1",
        "<table_id_2>": "target_table_name_2"
    }
}

Run the tap with:

tap-airtable --config config.json

Meltano Configuration

Run meltano config tap-airtable set --interactive to set the configuration interactively.

tap-airtable's People

Contributors

tomasvotava avatar dependabot[bot] avatar tayloramurphy avatar

Stargazers

Luke Evers avatar Rene Hlavova avatar

Watchers

 avatar

tap-airtable's Issues

Type error on field

Hey,

I was testing this out and ran into an issue. I have a field in airtable that is a formula and returns a number. It looks like it's mad it's not a string (and was not cast to a string):

Screenshot 2024-07-29 at 2 29 43 PM

Here are my logs:

root@docker-desktop:/project# meltano run tap-airtable target-jsonl
2024-07-29T18:25:46.303044Z [info     ] Environment 'dev' is active   
2024-07-29T18:25:47.103235Z [warning  ] No state was found, complete import.
2024-07-29T18:25:49.473064Z [info     ] 2024-07-29 18:25:49,472 | INFO     | tap-airtable.active_journeys | Beginning full_table sync of 'active_journeys'... cmd_type=elb consumer=False name=tap-airtable producer=True stdio=stderr string_id=tap-airtable
2024-07-29T18:25:49.473868Z [info     ] 2024-07-29 18:25:49,472 | INFO     | tap-airtable.active_journeys | Tap has custom mapper. Using 1 provided map(s). cmd_type=elb consumer=False name=tap-airtable producer=True stdio=stderr string_id=tap-airtable
2024-07-29T18:25:49.915153Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.915836Z [info     ]   File "/project/.meltano/loaders/target-jsonl/venv/bin/target-jsonl", line 8, in <module> cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.916610Z [info     ]     sys.exit(main())           cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.917122Z [info     ]   File "/project/.meltano/loaders/target-jsonl/venv/lib/python3.10/site-packages/target_jsonl.py", line 92, in main cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.917589Z [info     ]     state = persist_messages(  cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.917983Z [info     ]   File "/project/.meltano/loaders/target-jsonl/venv/lib/python3.10/site-packages/target_jsonl.py", line 54, in persist_messages cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.919601Z [info     ]     validators[o['stream']].validate((o['record'])) cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.920150Z [info     ]   File "/project/.meltano/loaders/target-jsonl/venv/lib/python3.10/site-packages/jsonschema/validators.py", line 130, in validate cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.920646Z [info     ]     raise error                cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.921338Z [info     ] jsonschema.exceptions.ValidationError: 6 is not of type 'string', 'null' cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.921995Z [info     ]                                cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.922677Z [info     ] Failed validating 'type' in schema['properties']['days_idle']: cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.923281Z [info     ]     {'type': ['string', 'null']} cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.923736Z [info     ]                                cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.924801Z [info     ] On instance['days_idle']:      cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.925646Z [info     ]     6                          cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2024-07-29T18:25:49.949321Z [error    ] Loader failed                 
2024-07-29T18:25:49.949836Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.