This is a sample data engineering project that tries to answer the question if cow best friends also take their meals together.

This project has been used for the talk "Bridging the Production Gap: Develop and Deploy Code Easily With IDEs" as the DATA+AI Summit 2023.

The notebook /notebooks/Generate Cow Data.ipynb contains the code to generate the sample data. In order to create the sample data import this notebook into your Databricks workspace and execute it there.

Alternatively you can open the notebook in VS Code with the Databricks extension installed and execute the Databricks: Run File as Workflow on Databricks command.

Configure Python virtual environment

python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

pytest -v tests --disable-warnings