Topic: bigdata Goto Github

Some thing interesting about bigdata

👇 Here are 1983 public repositories matching this topic...

absaoss / spline

bigdata,Data Lineage Tracking And Visualization Solution

Organization: absaoss

Home Page: https://absaoss.github.io/spline/

spark lineage tracking visualization bigdata hadoop scala

apache / amoro

bigdata,Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.

Organization: apache

Home Page: https://amoro.apache.org/

bigdata datalake lakehouse

apache / avro

bigdata,Apache Avro is a data serialization system.

Organization: apache

Home Page: https://avro.apache.org/

csharp ruby cplusplus python php java c avro bigdata dotnet

apache / celeborn

bigdata,Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

Organization: apache

Home Page: https://celeborn.apache.org/

shuffle spark bigdata

apache / hudi

bigdata,Upserts, Deletes And Incremental Processing on Big Data.

Organization: apache

Home Page: https://hudi.apache.org/

hudi apachehudi datalake bigdata apachespark incremental-processing stream-processing data-integration apacheflink

apache / incubator-livy

bigdata,Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

Organization: apache

Home Page: https://livy.apache.org/

livy bigdata spark apachelivy

apache / shardingsphere

bigdata,Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.

Organization: apache

database distributed-database distributed-sql-database distributed-transactions sql shard database-cluster mysql postgresql database-plus

bigartm / bigartm

bigdata,Fast topic modeling platform

Organization: bigartm

Home Page: http://bigartm.org/

bigartm bigdata c-plus-plus machine-learning python python-api regularizer text-mining topic-modeling

byzer-org / byzer-lang

bigdata,Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.

Organization: byzer-org

Home Page: https://www.byzer.org

machine-learning mlsql bigdata sql-like-dsl

collabh / bigdata-growth

bigdata,大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

User: collabh

flink kafka hive mapreduce spark olap kudu hadoop hbase debezium

dataexpert-io / data-engineer-handbook

bigdata,This is a repo with links to everything you'd ever want to learn about data engineering

Organization: dataexpert-io

apachespark awesome bigdata data dataengineering sql

datafuselabs / databend

bigdata,𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com

Organization: datafuselabs

Home Page: https://docs.databend.com

ai bigdata database rust serverless snowflake

dotnet / spark

bigdata,.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Organization: dotnet

Home Page: https://dot.net/spark

spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core

douban / dpark

bigdata,Python clone of Spark, a MapReduce alike framework in Python

Organization: douban

bigdata mapreduce dpark stream-processing spark python

dtstack / chunjun

bigdata,A data integration framework

Organization: dtstack

Home Page: https://dtstack.github.io/chunjun/

flink bigdata data-integration framework java

dtstack / flinkstreamsql

bigdata,基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法

Organization: dtstack

flink sql stream bigdata

fdv / running-elasticsearch-fun-profit

bigdata,A book about running Elasticsearch

User: fdv

Home Page: https://fdv.github.io/running-elasticsearch-fun-profit/

elasticsearch documentation sysadmin ebook bigdata

gangly / datafaker

bigdata,Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

User: gangly

mysql bigdata postgresql kafka datafaker testing hive hbase python fakedata

gearpump / gearpump

bigdata,Lightweight real-time big data streaming engine over Akka

Organization: gearpump

Home Page: https://gearpump.github.io/gearpump/

akka stream-processing bigdata scala

grailbio / bigslice

bigdata,A serverless cluster computing system for the Go programming language

Organization: grailbio

Home Page: https://bigslice.io/

cluster computing go golang mapreduce bigdata machinelearning etl

griddb / griddb

bigdata,GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.

Organization: griddb

Home Page: https://griddb.org/

database griddb nosql iot fast time-series newsql sql bigdata timeseries

heibaiying / bigdata-notes

bigdata,大数据入门指南 :star:

User: heibaiying

hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka

hi-primus / optimus

bigdata,:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Organization: hi-primus

Home Page: https://hi-optimus.com

spark pyspark data-wrangling bigdata big-data-cleaning data-science data-cleansing data-cleaner data-transformation machine-learning

igaowei / bigdataview

bigdata,100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中

User: igaowei

Home Page: https://igaowei.github.io/BigDataView/

bigdataviewer html-template echarts bigdata viewmodel

bigdata,懒松鼠Flink-Boot 脚手架让Flink全面拥抱Spring生态体系，使得开发者可以以Java WEB开发模式开发出分布式运行的流处理程序，懒松鼠让跨界变得更加简单。懒松鼠旨在让开发者以更底上手成本（不需要理解分布式计算的理论知识和Flink框架的细节）便可以快速编写业务代码实现。为了进一步提升开发者使用懒松鼠脚手架开发大型项目的敏捷的度，该脚手架默认集成Spring框架进行Bean管理，同时将微服务以及WEB开发领域中经常用到的框架集成进来，进一步提升开发速度。比如集成Mybatis ORM框架，Hibernate Validator校验框架,Spring Retry重试框架等，具体见下面的脚手架特性。

User: intsmaze

flink bigdata java mcv sping spring-boot mybatis spring-retry flink-boot java-flink

jadianes / spark-movie-lens

bigdata,An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

User: jadianes

movie-recommendation movielens-dataset spark flask python big-data bigdata

jadianes / spark-py-notebooks

bigdata,Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

User: jadianes

Home Page: http://jadianes.github.io/spark-py-notebooks

spark python pyspark data-analysis mllib ipython-notebook notebook ipython data-science machine-learning

josonle / coding-now

bigdata,学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等

User: josonle

ebook-collection notes coding java hadoop-hive spark linux bigdata

juicedata / juicefs

bigdata,JuiceFS is a distributed POSIX file system built on top of Redis and S3.

Organization: juicedata

Home Page: https://juicefs.com

filesystem cloud-native golang redis distributed-systems storage object-storage posix hdfs s3

kubernetes-retired / kube-batch

bigdata,A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC

Organization: kubernetes-retired

kubernetes k8s-sig-scheduling machine-learning bigdata hpc

liyupi / sql-generator

bigdata,🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~

User: liyupi

Home Page: http://sql.yupi.icu

bigdata hive javascript json spark sql mysql typescript vite vue

microsoft / mobius

bigdata,C# and F# language binding and extensions to Apache Spark

Organization: microsoft

spark apache-spark rdd dataframe dstream dataset streaming csharp mobius kafka-streaming

moran1607 / bigdataguide

bigdata,大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

User: moran1607

hadoop zookeeper hive flume kafka hbase spark flink scala bigdata

netflix / genie

bigdata,Distributed Big Data Orchestration Service

Organization: netflix

Home Page: https://netflix.github.io/genie

big-data bigdata orchestration configuration configuration-management java spring-boot distributed-systems netflixoss cloud

newtendermint / awesome-bigdata

bigdata,A curated list of awesome big data frameworks, ressources and other awesomeness.

User: newtendermint

Home Page: https://github.com/onurakpolat/awesome-bigdata

awesome bigdata data-warehouse data-analytics data-visualization data-stream database data-science data awesome-list

opendatadiscovery / odd-platform

bigdata,First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

Organization: opendatadiscovery

Home Page: https://opendatadiscovery.org

oss data-platform metadata metadata-management data-pipelines data-engineering observability data-catalog datacatalog data-discovery

pingcap / tispark

bigdata,TiSpark is built for running Apache Spark on top of TiDB/TiKV

Organization: pingcap

bigdata spark tidb tikv

seagate / cortx

bigdata,CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

Organization: seagate

Home Page: https://github.com/Seagate/cortx

s3-storage software-defined-storage storage storage-api object-storage object-storage-service objectstorage objectstore opensource open-source

shzlw / poli

bigdata,An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.

User: shzlw

Home Page: https://shzlw.github.io/poli

business-intelligence sql dashboard data-visualization sql-editor jdbc bigdata reporting spring-boot reactjs

taosdata / tdengine

bigdata,TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.

Organization: taosdata

Home Page: https://tdengine.com

iot bigdata time-series database industrial-iot connected-vehicles monitoring tsdb tdengine sql

tensorbase / tensorbase

bigdata,TensorBase is a new big data warehousing with modern efforts.

Organization: tensorbase

Home Page: https://tensorbase.io/

rust bigdata database analytics modern infrastructure data data-infrastructure high-performance engineering

vaexio / vaex

bigdata,Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

Organization: vaexio

Home Page: https://vaex.io

dataframe python bigdata tabular-data visualization memory-mapped-file hdf5 machine-learning machinelearning data-science