Easy-to-use Databricks Notebooks for Admin Tasks.
Made with โค๏ธ by Dotlas Inc
This repository contains a directory of Databricks notebooks that assists with administrative tasks for Databricks, or otherwise helps as a supporting utility.
For example, consider the following use-cases:
- ๐ View a calendar of scheduled jobs to resolve conflicts in Databricks workflows.
- ๐ผ Upload a Pandas DataFrame to Delta Lake
- ๐ Update Delta Lake table Documentation using Pydantic Models
- โฟ Migrate Jobs between Databricks workspaces
- โ๏ธ Mass-edit Job Clusters in Existing Jobs
Notebook | Description |
---|---|
Workflow Calendar | Visualize scheduled Jobs on a calendar, eyeball conflicts and view historic runs as a Gantt chart |
Delta Docs with Pydantic | If you have pydantic models with fields containing description and tags that are used as data models, transfer these field descriptions to Delta lake columns as comments and tags. |
Pandas to Delta | Use databricks-sql-python and SQLAlchemy to upload a Pandas DataFrame to Delta Lake from outside a Databricks environment |
Workspace Jobs Migration | Migrate Workflows from one Databricks workspace to another |
Job Cluster Update | Use the Databricks API to mass-update Job and Task configs |
Workflow Config Exporter | Export existing workflow configuration and save it for future consumption |
- Check out the launch discussion on this LinkedIn Release Post, with a highlight from Databricks CEO, Ali Ghodsi.
- Feel free to raise an issue on this repository to start a discussion about new features, bug fixes or enhancements.
- See CONTRIBUTING.md for guidelines when adding or modifying notebooks in this repository.