Git Product home page Git Product logo

dbt-data-reliability's Introduction

Logo

Data Observability for Analytics Engineers

Elementary enables you to monitor your data and dbt operation.

To learn more, refer to our main repo, and live demo.

For reporting issues, feature requests and contributions, refer to issues in the main repo.

Quick start

After adding the package to packages.yml and running dbt deps, add to your dbt_project.yml:

models:

## elementary models will be created in the schema '<your_schema>_elementary'
## see docs: https://docs.elementary-data.com/
  elementary:
    +schema: 'elementary'

And run dbt run --select elementary.

Check out the full documentation.

Run Results and dbt artifacts

The package automatically uploads the dbt artifacts and run results to your tables in your data warehouse.

Here you can find additional details.

Data anomalies detection as dbt tests

Elementary dbt tests collect metrics and metadata over time, such as freshness, volume, schema changes, distribution, cardinality, etc. Executed as any other dbt tests, the Elementary tests alert on anomalies and outliers.

Elementary tests are configured and executed like native tests in your project!

Example of Elementary test config in properties.yml:

models:
  - name: your_model_name
    config:
      elementary:
        timestamp_column: updated_at
    tests:
        - elementary.table_anomalies
        - elementary.all_columns_anomalies

Data observability report

UI

Checkout the live demo.

Slack alerts

UI

High level architecture

UI

Data warehouse support

This package has been tested on Snowflake, BigQuery and Redshift. Additional integrations coming soon!

Community & Support

Contributions

Thank you 🧑 Whether it’s a bug fix, new feature, or additional documentation - we greatly appreciate contributions!

Check out the contributions guide and open issues in the main repo.

dbt-data-reliability's People

Contributors

maayan-s avatar oravi avatar elongl avatar idoneshaveit avatar ecalzo avatar civitaspo avatar gthesheep avatar irinasel avatar jtalmi avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.