datamechanics Goto Github PK

aws-glue-data-catalog-client-for-apache-hive-metastore

The AWS Glue Data Catalog is a fully managed, Apache Hive Metastore compatible, metadata repository. Customers can use the Data Catalog as a central repository to store structural and operational metadata for their data. AWS Glue provides out-of-box integration with Amazon EMR that enables customers to use the AWS Glue Data Catalog as an external Hive Metastore. This is an open-source implementation of the Apache Hive Metastore client on Amazon EMR clusters that uses the AWS Glue Data Catalog as an external Hive Metastore. It serves as a reference implementation for building a Hive Metastore-compatible client that connects to the AWS Glue Data Catalog. It may be ported to other Hive Metastore-compatible platforms such as other Hadoop and Apache Spark distributions

certified-operators

Production catalog for Red Hat Certified Operator Bundles

charts

Curated applications for Kubernetes

datamechanics_airflow_plugin

delight

A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.

docker-demo

A demo of the Docker dev workflow with Data Mechanics

eks-spark-benchmark

Performance optimization for Spark running on Kubernetes

enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

jupyter-hub-example

DataMechanics / JupyterHub integration example

jupyter_server

The backend—i.e. core services, APIs, and REST endpoints—to Jupyter web applications.

notebook

Jupyter Interactive Notebook

prophet

Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.

scala-project-demo

An example project to run a Scala app on Data Mechanics

spark

Apache Spark - A unified analytics engine for large-scale data processing

spark-on-k8s-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

spark-snowflake

Snowflake Data Source for Apache Spark.

spark-sql-perf

terraform-spotinst-ocean-aws-k8s

A Terraform module to create an Ocean Cluster.