Git Product home page Git Product logo

dataengineering's Introduction

dataengineering

https://awesomedataengineering.com/

https://github.com/letthedataconfess/Data-Engineering-Books

https://github.com/oleg-agapov/data-engineering-book

https://github.com/Nunie123/data_engineering_on_gcp_book

data lakehouse & data warehouse

https://www.altexsoft.com/blog/data-lakehouse/

https://www.sqlhammer.com/cloud-data-warehouse/

https://www.oracle.com/autonomous-database/autonomous-data-warehouse/

https://www.qlik.com/us/cloud-data-migration/cloud-data-warehouse

https://www.scnsoft.com/analytics/data-warehouse/cloud

https://solutionsreview.com/data-management/the-best-cloud-data-warehouse-solutions-2/

https://www.stitchdata.com/resources/compare-on-premises-and-cloud-data-warehouse/

https://greenplum.org/

bigdata platform

cloudera

mapr

azure hdinsight

databricks

google dataproc

Opensource Data Lake

https://delta.io/

https://iceberg.apache.org/

https://hudi.apache.org/

Data Lake Comparisions

https://airbyte.com/blog/data-lake-lakehouse-guide-powered-by-table-formats-delta-lake-iceberg-hudi

https://www.databricks.com/session_na20/a-thorough-comparison-of-delta-lake-iceberg-and-hudi

https://www.dremio.com/subsurface/comparison-of-data-lake-table-formats-iceberg-hudi-and-delta-lake/

https://lakefs.io/hudi-iceberg-and-delta-lake-data-lake-table-formats-compared/

https://databeans-blogs.medium.com/delta-vs-iceberg-vs-hudi-reassessing-performance-cb8157005eb0

https://www.onehouse.ai/blog/apache-hudi-vs-delta-lake-vs-apache-iceberg-lakehouse-feature-comparison?830fc854_page=3

CDC (Log based,table based)

https://hevodata.com/learn/7-best-cdc-tools/

debezium

golden gate

streamsets

ETL/ELT tools

informatica

apache nifi

dbt

apache airflow

Talend

FiveTran

airbyte

airflow

Stitch

Xplenty

Matillion

dataengineering's People

Contributors

msaltan avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.