tap-sharepointsites is a Singer tap for Microsoft Graph Sharepoint lists.

Built with the Meltano Tap SDK for Singer Taps.

• catalog
• state
• discover
• about
• stream-maps
• schema-flattening

A full list of supported settings and capabilities is available by running: tap-sharepointsites --about

The file configuration accepts an array of objects, with keys:

• name: Name given to the stream/table
• file_pattern: regex-like pattern for filenames to load
• folder: Subfolder where the files are located
• file_type: Type (format) of file to load, either csv or excel.
• delimiter: Field delimiter for CSV files. default ,
• clean_colnames: Whether to convert column names to snake_case. default false
• sheet_name: Sheet name to pull from. default: Sheet1
• min_row: starting row in sheet. optional
• max_row: last row in sheet. optional
• min_col: starting column in sheet. optional
• max_col: last column in sheet. optional

Example config:

...
  config:
    ...
    files:
    - name: employees
      file_pattern: employees_.*\.xlsx
      folder: hr_data/raw
      file_type: excel
      clean_colnames: true
      sheet_name: Sheet1
      min_row: 1
      max_row: 10
      min_col: 1
      max_col: 10
  ...

You can sync the content of sharepoint web pages, typically relevant for LLM/RAG type of use cases. The Microsoft Graph endpoint for pages is still in Beta, and does not work when logged in as a personal user. In order for it to work, you need to use a Managed Identity.

Example config:

...
  config:
    ...
    pages: true
  ...

A full list of supported settings and capabilities for this tap is available by running:

tap-sharepointsites --about

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

You can easily run tap-sharepointsites by itself or in a pipeline using Meltano.

tap-sharepointsites --version
tap-sharepointsites --help
tap-sharepointsites --config CONFIG --discover > ./catalog.json

Follow these instructions to contribute to this project.

pipx install poetry
poetry install

Create tests within the tap_sharepointsites/tests subfolder and then run:

poetry run pytest

You can also test the tap-sharepointsites CLI interface directly using poetry run:

poetry run tap-sharepointsites --help

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-sharepointsites
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-sharepointsites --version
# OR run a test `elt` pipeline:
meltano elt tap-sharepointsites target-jsonl

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.