Exercise Staters and solutions for cd0581-building-a-reproducible-model-workflow by Giacomo Vianello

License: Other

Python 0.72% Jupyter Notebook 99.28%

cd0581-building-a-reproducible-model-workflow-exercises's Introduction

Build a Reproducible Model Workflow - Exercises

This repo contains the code for demos, exercises, and exercise solutions.

This repository organizes the code by the lessons that they are used in. Each set of code is located in their respective lessons.

Please note that certain instructions for each exercise, as well as any relevant environment setup, are only provided within the Udacity classroom.

Example:

All lesson 2 files are in /lesson-2-data-exploration-and-preparation/.

This directory contains: demo, exercises, with the exercises directory organized by the exercise number, and therein containing an exercise README.md file and starter and solution directories.

cd0581-building-a-reproducible-model-workflow-exercises's People

Contributors

Stargazers

Watchers

Forkers

prateek-01 siddjain24 lsm0729 mxagar lamjoun rfajri27 marcelortizv prajwalcr95 alangrosso fsq2 tuscar2001 yhbyhb rccmodena ozangenc nikohunt jakrapuns ruchitagarde gariciodaro agiedke prashdash112 mfurquimdev savirkkula arthurrenault flyingsamosa ahmedalaa24494 h-rim cs48a bialymt92 beatrizalbiero yoovraj mustaphabenhajm dangstan satwiknedurumalli ghulamzada samisouabni seghelicious acadena-repo damiano-glassform 7pandeys kunle-xy vinay-singh27 pedramhashemi quadri-olamilekan sadrafh nezihakin smetal1 khaledbakri soubc carlosjimenez88m dhedderich anwesend-tui ondrej-ploteny phidesigner rahul-ahuja taoge64 marcusreaiche prayat-pu arunchavan89 lhamnett manualrg lightuse prestondewell quynhpt98 catherinemartini fabryandrea erwin-ma neerajadsul huyenbui117 btwendel ranjitgopi korinnestegall jeffc05 prathmesh-dali blake-belpedio adityakush1 march-08 sheridanq swatt45 deepbluefuzzie cherfongfoo carmend-wgu shape9190

cd0581-building-a-reproducible-model-workflow-exercises's Issues

Exercise 11 - Issue

Exercise 11 fails with the following return error:
$ mlflow run . -P hydra_options="random_forest_pipeline.random_forest.max_depth=5"
2023/05/20 18:18:32 INFO mlflow.utils.conda: Conda environment mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 already exists.
2023/05/20 18:18:32 INFO mlflow.projects.utils: === Created directory C:\Users\giannis\AppData\Local\Temp\tmpftu8jmwr for downloading remote URIs passed to arguments of type 'path' ===
2023/05/20 18:18:32 INFO mlflow.projects.backend.local: === Running command 'conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 && python main.py $(echo random_forest_pipeline.random_forest.max_depth=5)' in run with ID 'fabf4ebaf7fc40649a25570246409951' ===
C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py:49: FutureWarning: In the future np.object will be defined as the corresponding NumPy scalar.
binary = (7, np.dtype("bytes"), "BinaryType", np.object)
Traceback (most recent call last):
File "C:\Users\giannis\Desktop\ML-DevOps-2\lesson-4-training-validation-experiment-tracking\exercises\exercise_11\starter\main.py", line 3, in
import mlflow
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow_init_.py", line 52, in
import mlflow.fastai as fastai # noqa: E402
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\fastai.py", line 22, in
from mlflow import pyfunc
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\pyfunc_init_.py", line 219, in
import mlflow.pyfunc.model
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\pyfunc\model.py", line 17, in
from mlflow.models import Model
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\models_init_.py", line 25, in
from .signature import ModelSignature, infer_signature
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\models\signature.py", line 12, in
from mlflow.types.schema import Schema
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types_init_.py", line 6, in
from .schema import DataType, ColSpec, Schema, TensorSpec
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py", line 20, in
class DataType(Enum):
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py", line 49, in DataType
binary = (7, np.dtype("bytes"), "BinaryType", np.object)
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\numpy_init_.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'object'.
np.object was a deprecated alias for the builtin object. To avoid this error in existing code, use object by itself. Doing this will not modify any behavior and is safe.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
2023/05/20 18:18:35 ERROR mlflow.cli: === Run (ID 'fabf4ebaf7fc40649a25570246409951') failed ===
(venv)

To Whoever Facing Problems Running the Exercises on WSL (non-Docker Workaround)

To whoever taking this course in year 2024 (and beyond)

Until Udacity updates the contents, the course is pretty stuck in year 2022. I'm sharing what works for me on WSL (didn't try on *nix or Mac yet), but it probably will work for them too. I gave up "fighting" the versions issue after many hair-pulling evenings.

Miniconda

After installing the latest and the greatest, create a new conda environment with Python version:

3.7 as follows (I named it as py37, to make it obvious), if you are using WSL.
3.8 (change the arguments according), if you are using Mac.

$ conda create -n py37 python=3.7
$ conda activate py37

Then install some essential packages as specified as follows with pip (don't worry, this won't "taint" you system/base Python's site packages):

# if you are using WSL
$ pip install protobuf==3.20.3 mlflow=1.14.1 numpy=1.18.5 wandb==0.10.21 

# if you are using Mac
$ pip install protobuf==3.20.3 mlflow=1.14.1 wandb==0.10.21
$ pip uninstall numpy
$ conda install numpy=1.19.5   

# install the following regardless of the OS you are using
$ pip install hydra-core==1.3.2

Exercises

In every conda.yml file (remember those in sub-directories too), add the lines as stated in comments:

name: whatever
dependencies:
  - python=3.7        # add this, always (or python=3.8 if you are using Mac)
  - numpy=1.18.5      # add this when mlflow or pandas is listed as a dependency (or numpy=1.19.5 if you are using Mac)
  - Jinja2=3.0.3      # add this when pandas-profiling is listed as a dependency
  - jupyterlab=3.0.12 # remove this when it is listed as a dependency
  - notebook          # add this when jupyterlab is listed as a dependency; no need to specify the version
  - pip:
    - protobuf==3.20.3   # add this when wandb is listed as a dependency

With these, most of the dependencies stated in the exercise's conda.yml should just work (as of Mar 2024). For those that don't (especially the transitive dependencies), search in conda forge or pypi for versions that were published in 2022 (preferably in Mar 2022 or earlier).

In the exercise where jupyterlab dependency is replaced with notebook, change the command in the MLProject to
jupyter notebook.

Misc

~~When I have time, I'll try the same setup directly on Windows (instead of WSL) and Mac.~~ Directly/Natively on Windows won't work 'cos of the use of $(echo {hydra_options}) in some of the MLProject files.
Mac users may have to make some package version updates discovered along the way. I only randomly chose the exercises to test the setup on Mac, and discovered that some dependencies won't work, e.g., matplotlib 3.2.2 needs to be replaced with 3.5.3 (for exercise 12).

Also, whoever having successes in setting things differently, please add to this "issue" to help future learners.

Udacity

The test in exercise_9/solution is supposed to fail for ks_alpha=0.9, instead it succeeds

As per the README file, the test should fail when ks_alpha is set to 0.9 (or higher), instead it succeeds.

-> mlflow run . -P reference_artifact=exercise_6/data_train.csv:latest -P sample_artifact=exercise_6/data_test.csv:latest -P ks_alpha=0.9
2022/09/04 10:02:35 INFO mlflow.utils.conda: Conda environment mlflow-d8f23e2b65d43a5000d54c81519b844ddb5d8310 already exists.
2022/09/04 10:02:35 INFO mlflow.projects.utils: === Created directory /tmp/tmpc0japcqw for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 10:02:35 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-d8f23e2b65d43a5000d54c81519b844ddb5d8310 1>&2 && pytest -s -vv . --reference_artifact exercise_6/data_train.csv:latest \
                --sample_artifact exercise_6/data_test.csv:latest \
                --ks_alpha 0.9' in run with ID 'b4492cd98a504bada366ed9b1ae6fe53' === 
wandb: Currently logged in as: fantauzzi. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.13.2
wandb: Run data is saved locally in /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_9/solution/wandb/run-20220904_100237-3g3wtwt4
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run leafy-deluge-33
wandb: ⭐️ View project at https://wandb.ai/fantauzzi/exercise_9
wandb: 🚀 View run at https://wandb.ai/fantauzzi/exercise_9/runs/3g3wtwt4
=================================================================================================== test session starts ====================================================================================================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 -- /home/fanta/.local/virtualenv/python3/bin/python
cachedir: .pytest_cache
rootdir: /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_9/solution
collected 1 item                                                                                                                                                                                                           

test_data.py::test_kolmogorov_smirnov PASSED

==================================================================================================== 1 passed in 2.10s =====================================================================================================
wandb: Waiting for W&B process to finish... (success).
wandb:                                                                                
wandb: Synced leafy-deluge-33: https://wandb.ai/fantauzzi/exercise_9/runs/3g3wtwt4
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20220904_100237-3g3wtwt4/logs
2022/09/04 10:02:48 INFO mlflow.projects: === Run (ID 'b4492cd98a504bada366ed9b1ae6fe53') succeeded ===

exercise_11 stops with error "TypeError: Descriptors cannot not be created directly."

Steps to reproduce:

cd into cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter
run mlflow run . -P hydra_options="random_forest_pipeline.random_forest.max_depth=5" (as per instructions in README.md)

This is the output

2022/09/05 09:30:58 INFO mlflow.utils.conda: === Creating conda environment mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 ===
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt']
Pip subprocess output:
Collecting hydra-joblib-launcher==1.1.2
  Using cached hydra_joblib_launcher-1.1.2-py3-none-any.whl (5.2 kB)
Requirement already satisfied: hydra-core==1.0.* in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (1.0.6)
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: Click>=7.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (8.1.3)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: requests<3,>=2.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: configparser>=3.8.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (5.3.0)
Requirement already satisfied: GitPython>=1.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.1.27)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: protobuf>=3.12.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (4.21.5)
Requirement already satisfied: PyYAML in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (6.0)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (4.8)
Requirement already satisfied: omegaconf<2.1,>=2.0.5 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (2.0.6)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.0.5)
Collecting joblib>=0.14.0
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Requirement already satisfied: typing-extensions in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from omegaconf<2.1,>=2.0.5->hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (4.3.0)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting psutil>=5.0.0
  Using cached psutil-5.9.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Requirement already satisfied: idna<3,>=2.5 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2022.6.15)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.6-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.5-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.4-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.3-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.2-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.1-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.0-py2.py3-none-any.whl (156 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Installing collected packages: subprocess32, shortuuid, sentry-sdk, psutil, promise, pathtools, joblib, wandb, hydra-joblib-launcher
Successfully installed hydra-joblib-launcher-1.1.2 joblib-1.1.0 pathtools-0.1.2 promise-2.3 psutil-5.9.2 sentry-sdk-1.9.0 shortuuid-1.0.9 subprocess32-3.5.4 wandb-0.10.21
                                                                                                                                                                                                                          done
#
# To activate this environment, use
#
#     $ conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/05 09:31:49 INFO mlflow.projects.utils: === Created directory /tmp/tmp9nvxxkaf for downloading remote URIs passed to arguments of type 'path' ===
2022/09/05 09:31:49 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 1>&2 && python main.py $(echo random_forest_pipeline.random_forest.max_depth=5)' in run with ID '81bf8f661ec34669929321b0048accf9' === 
2022/09/05 09:31:50 INFO mlflow.utils.conda: === Creating conda environment mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f ===
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt']
Pip subprocess output:
Collecting omegaconf==2.0.6
  Using cached omegaconf-2.0.6-py3-none-any.whl (36 kB)
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt (line 1)) (2.8.2)
Collecting Click>=7.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
  Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
  Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
  Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
  Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
  Using cached psutil-5.9.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (284 kB)
Collecting PyYAML>=5.1.*
  Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting requests<3,>=2.0.0
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
  Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Collecting typing-extensions
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, typing-extensions, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb, omegaconf
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 omegaconf-2.0.6 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.2 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 typing-extensions-4.3.0 urllib3-1.26.12 wandb-0.10.21
                                                                                                                                                                                                                          done
#
# To activate this environment, use
#
#     $ conda activate mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/05 09:32:48 INFO mlflow.projects.utils: === Created directory /tmp/tmpl8ujq6se for downloading remote URIs passed to arguments of type 'path' ===
2022/09/05 09:32:48 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f 1>&2 && python run.py --train_data exercise_6/data_train.csv:latest \
              --model_config /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/outputs/2022-09-05/09-31-50/random_forest_config.yml' in run with ID '9f795a4d42d7448296934713201fc13a' === 
Traceback (most recent call last):
  File "run.py", line 15, in <module>
    import wandb
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/__init__.py", line 38, in <module>
    from wandb import sdk as wandb_sdk
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/__init__.py", line 12, in <module>
    from .wandb_init import init  # noqa: F401
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
    from .backend.backend import Backend
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
    from ..interface import interface
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
    from wandb.proto import wandb_internal_pb2 as pb
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
    from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
    _descriptor.FieldDescriptor(
  File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/main.py", line 26, in go
    _ = mlflow.run(
  File "/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 307, in run
    _wait_for(submitted_run_obj)
  File "/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 324, in _wait_for
    raise ExecutionException("Run (ID '%s') failed" % run_id)
mlflow.exceptions.ExecutionException: Run (ID '9f795a4d42d7448296934713201fc13a') failed

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
2022/09/05 09:32:49 ERROR mlflow.cli: === Run (ID '81bf8f661ec34669929321b0048accf9') failed ===

Note: almost all of the exercises are broken. I will stop filing bugs with this one.

Incorrect use of random seeds in the solution to exercise_14

With reference to file https://github.com/udacity/cd0581-building-a-reproducible-model-workflow-exercises/blob/main/lesson-5-final-pipeline-release-and-deploy/exercises/exercise_14/solution/main.py

the call to the sub-project segregate omits passing the random seed, I think the call should pass it.

Also, at line 91 it is passing the wrong random seed to the random_forest sub-project. I believe it can be corrected like this

"random_seed": config['random_forest_pipeline']['random_forest']['random_state'],

Also, the URL provided in the 2 courses is not working the course and repository is all outdated please update the repository

exercise_4 solution stops with Jupyter command `jupyter-notebook` not found.

With reference to
https://github.com/udacity/cd0581-building-a-reproducible-model-workflow-exercises/tree/main/lesson-2-data-exploration-and-preparation/exercises/exercise_4/solution

After running
mlflow run .
the code stops with an error containing

Jupyter command jupyter-notebook not found.

I believe there are multiple issues with the provided conda.yml. The same problem is in the starter code for the same exercise.

Here there is a conda.yml that I have used to complete the exercise and also to run the solution.


name: exercise_4
channels:
  - conda-forge
  - defaults
dependencies:
  - jupyter=1.0.0
  - seaborn=0.11.2
  - pip=22.1.2
  - pandas=1.4.4
  - pandas-profiling=3.2.0
  - fastparquet=0.8.3
  - pyarrow=8.0.0
  - pip:
      - wandb==0.13.2

exercise_3 solution stops with an error (and the starter code has the same problem)

From the exercise_3 solution run

mlflow run .

and you get an error containing TypeError: Descriptors cannot not be created directly.

The issue is that the conda.yml directs MLFlow to use incompatible versions of some python packages.
Note that just forcing protobuf to version 3.20.*, as indicated in Udacity's Knowledge, still brings about errors.

At this time, a fix is to replace the - pip section at the end of files

download_data/conda.yml
process_data/conda.yml

with the following

  - pip:
      - wandb == 0.13.2
      - protobuf==3.20.*

The starter code of the same exercise needs to be fixed too.
Also see here.

exercise_6 solution stops with an error "TypeError: Descriptors cannot not be created directly."

Steps to reproduce:

cd into cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution
run MLFlow as per instructions provided in the README

mlflow run . -P input_artifact="exercise_5/preprocessed_data.csv:latest" \
             -P artifact_root="data" \
             -P test_size=0.3 \
             -P stratify="genre"

Execution stop with an error. The same must affect the starter code.

2022/09/03 10:32:07 INFO mlflow.utils.conda: === Creating conda environment mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e ===
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
python-3.9.13        | 26.7 MB   | ############################################################################################################################# | 100% 
scikit-learn-0.24.1  | 7.6 MB    | ############################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
  Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
  Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
  Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
  Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
  Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Collecting requests<3,>=2.0.0
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
  Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
  Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
                                                                                                                                                                      done
#
# To activate this environment, use
#
#     $ conda activate mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/03 10:33:24 INFO mlflow.projects.utils: === Created directory /tmp/tmp3onf6gyi for downloading remote URIs passed to arguments of type 'path' ===
2022/09/03 10:33:24 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e 1>&2 && python run.py --input_artifact exercise_5/preprocessed_data.csv:latest \
              --artifact_root data \
              --artifact_type raw_data \
              --test_size 0.3 \
              --random_state 42 \
              --stratify genre' in run with ID '509c6be194774515ac0139f174adcb82' === 
Traceback (most recent call last):
  File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/run.py", line 8, in <module>
    import wandb
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/__init__.py", line 38, in <module>
    from wandb import sdk as wandb_sdk
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/__init__.py", line 12, in <module>
    from .wandb_init import init  # noqa: F401
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
    from .backend.backend import Backend
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
    from ..interface import interface
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
    from wandb.proto import wandb_internal_pb2 as pb
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
    from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
    _descriptor.FieldDescriptor(
  File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
2022/09/03 10:33:25 ERROR mlflow.cli: === Run (ID '509c6be194774515ac0139f174adcb82') failed ===

The solution of exercise_7 stops with an error

The same issue should affect also the starter code.

Steps to reproduce

cd into cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution
run mlops run .

Get the error:

> mlflow run .
2022/09/03 13:55:31 INFO mlflow.utils.conda: === Creating conda environment mlflow-a3af6a16a95faee04af2c40a1852613ae4973185 ===
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
pluggy-0.13.1        | 29 KB     | ############################################################################################################################# | 100% 
more-itertools-8.14. | 45 KB     | ############################################################################################################################# | 100% 
pytest-6.2.2         | 432 KB    | ############################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
  Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
  Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
  Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
  Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
  Using cached psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Collecting requests<3,>=2.0.0
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
  Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
  Using cached PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
                                                                                                                                                                      done
#
# To activate this environment, use
#
#     $ conda activate mlflow-a3af6a16a95faee04af2c40a1852613ae4973185
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/03 13:56:53 INFO mlflow.projects.utils: === Created directory /tmp/tmpewqiu835 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/03 13:56:53 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-a3af6a16a95faee04af2c40a1852613ae4973185 1>&2 && pytest -s -vv .' in run with ID 'e69d9c038757490e847e2fe3d2f28cc2' === 
========================================================================= test session starts ==========================================================================
platform linux -- Python 3.9.13, pytest-6.2.2, py-1.11.0, pluggy-0.13.1 -- /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/bin/python
cachedir: .pytest_cache
rootdir: /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution
collected 0 items / 1 error                                                                                                                                            

================================================================================ ERRORS ================================================================================
____________________________________________________________________ ERROR collecting test_data.py _____________________________________________________________________
test_data.py:2: in <module>
    import wandb
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/__init__.py:38: in <module>
    from wandb import sdk as wandb_sdk
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/__init__.py:12: in <module>
    from .wandb_init import init  # noqa: F401
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/wandb_init.py:29: in <module>
    from .backend.backend import Backend
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/backend/backend.py:17: in <module>
    from ..interface import interface
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/interface/interface.py:18: in <module>
    from wandb.proto import wandb_internal_pb2 as pb
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/proto/wandb_internal_pb2.py:15: in <module>
    from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/proto/wandb_telemetry_pb2.py:34: in <module>
    _descriptor.FieldDescriptor(
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in __new__
    _message.Message._CheckCalledFromGeneratedFile()
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E   
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
======================================================================= short test summary info ========================================================================
ERROR test_data.py - TypeError: Descriptors cannot not be created directly.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================== 1 error in 0.30s ===========================================================================
2022/09/03 13:56:53 ERROR mlflow.cli: === Run (ID 'e69d9c038757490e847e2fe3d2f28cc2') failed ===

`conda.yml` of exercise 2 solution does not define a working conda environment.

When calling mflow using the provided solution conda.yml results in an error.
I use Python version 3.8.16 (followed instructions at beginning of the course).

The call in the solution folder:
mlflow run . -P file_url=https://raw.githubusercontent.com/scikit-learn/scikit-learn/4dfdfb4e1bb3719628753a4ece995a1b2fa5312a/sklearn/datasets/data/iris.csv -P artifact_name=iris -P artifact_description="This data sets consists of 3 different types of irises' (Setosa, Versicolour, and Virginica) petal and sepal length"

Output:

Traceback (most recent call last):
  File "XXX\Udacity\cd0581-building-a-reproducible-model-workflow-exercises\lesson-1-machine-learning-pipelines\exercises\exercise_2\solution\download_data.py", line 5, in <module>
    import wandb
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packag    from wandb import sdk as wandb_sdk
  File "XXX\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\__init__.py", line 12, in <module>
    from .wandb_init import init  # noqa: F401
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\wandb_init.py", line 29, in <module>
    from .backend.backend import Backend
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\backend\backend.py", line 17, in <module>
    from ..interface import interface
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\interface\interface.py", line 18, in <module>
    from wandb.proto import wandb_internal_pb2 as pb
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\proto\wandb_internal_pb2.py", line 15, in <module>
    from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\proto\wandb_telemetry_pb2.py", line 34, in <module>
    _descriptor.FieldDescriptor(
  File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\google\protobuf\descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.      
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).        

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
2023/05/23 14:11:33 ERROR mlflow.cli: === Run (ID '9b21011b2dd6459690460967d55c7ff2') failed ===

The proposed solution of downgrading protobuf to 3.20.x dit not fix this environment definition, but resulted in new errors (MutableSet not being defined in collections). Please fix/update the learning materials.

Exercise 2 of building a reproducible workflow model is outdated and not working

lesson_2/exercise_6 [wandb] artifact.file() raising an IndexError

even when running the solution, I get the following traceback:

Traceback (most recent call last):
  File "/path/to/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/starter/run.py", line 135, in <module>
    go(args)
  File "/path/to/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/starter/run.py", line 23, in go
    artifact_path = artifact.file()
  File "/path/to/anaconda3/envs/mlflow-9bfe8e3af68696f10e0ad9a623577b2921db6aba/lib/python3.9/site-packages/wandb/apis/public.py", line 3946, in file
    return self._download_file(list(manifest.entries)[0], root=root)
IndexError: list index out of range

Course 2 - Lesson 2 Bug - Deprecate library import

Pandas profiling library is deprecated.

Excercise 3 - conda.yml in /process_data/ has "name: download_data"

`name: download_data
channels:

conda-forge
defaults
....`

But it should probably be

`name: process_data
channels:

...`

exercise_10/solution stops with an error


> mlflow run .
2022/09/04 11:18:25 INFO mlflow.utils.conda: === Creating conda environment mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6 ===
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
greenlet-1.1.3       | 91 KB     | ################################################################################################################################################################################# | 100% 
click-8.1.3          | 146 KB    | ################################################################################################################################################################################# | 100% 
protobuf-4.21.5      | 342 KB    | ################################################################################################################################################################################# | 100% 
omegaconf-2.0.6      | 103 KB    | ################################################################################################################################################################################# | 100% 
gunicorn-20.1.0      | 119 KB    | ################################################################################################################################################################################# | 100% 
sqlalchemy-1.4.40    | 2.4 MB    | ################################################################################################################################################################################# | 100% 
mlflow-1.14.1        | 4.2 MB    | ################################################################################################################################################################################# | 100% 
chardet-3.0.4        | 170 KB    | ################################################################################################################################################################################# | 100% 
docker-py-5.0.3      | 188 KB    | ################################################################################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: / Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: GitPython>=1.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.1.27)
Requirement already satisfied: Click>=7.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (8.1.3)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: configparser>=3.8.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (5.3.0)
Requirement already satisfied: requests<3,>=2.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: protobuf>=3.12.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (4.21.5)
Requirement already satisfied: PyYAML in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (6.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.0.5)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting psutil>=5.0.0
  Using cached psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2022.6.15)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (1.25.11)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.6-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.5-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.4-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.3-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.2-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.1-py2.py3-none-any.whl (157 kB)
  Using cached sentry_sdk-1.9.0-py2.py3-none-any.whl (156 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Installing collected packages: subprocess32, shortuuid, sentry-sdk, psutil, promise, pathtools, wandb
Successfully installed pathtools-0.1.2 promise-2.3 psutil-5.9.1 sentry-sdk-1.9.0 shortuuid-1.0.9 subprocess32-3.5.4 wandb-0.10.21
                                                                                                                                                                                                                          done
#
# To activate this environment, use
#
#     $ conda activate mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/04 11:19:53 INFO mlflow.projects.utils: === Created directory /tmp/tmpfn3ljdp3 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 11:19:53 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6 1>&2 && python main.py $(echo '')' in run with ID '4dac944151114645ab75bbfaee227015' === 
2022/09/04 11:19:54 INFO mlflow.utils.conda: === Creating conda environment mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f ===
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
plotly-4.4.1         | 4.1 MB    | ################################################################################################################################################################################# | 100% 
retrying-1.3.3       | 11 KB     | ################################################################################################################################################################################# | 100% 
python-3.8.13        | 25.2 MB   | ################################################################################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
  Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
  Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
  Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
  Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
  Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
  Using cached psutil-5.9.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (284 kB)
Collecting requests<3,>=2.0.0
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
  Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
  Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
                                                                                                                                                                                                                          done
#
# To activate this environment, use
#
#     $ conda activate mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/04 11:20:56 INFO mlflow.projects.utils: === Created directory /tmp/tmpq_hnq_82 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 11:20:56 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f 1>&2 && python run.py --train_data exercise_6/data_train.csv:latest \
              --model_config /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/outputs/2022-09-04/11-19-54/random_forest_config.json' in run with ID '1a76630e6bbc434f9cf6a81a83f227e0' === 
Traceback (most recent call last):
  File "run.py", line 15, in <module>
    import wandb
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/__init__.py", line 38, in <module>
    from wandb import sdk as wandb_sdk
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/__init__.py", line 12, in <module>
    from .wandb_init import init  # noqa: F401
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
    from .backend.backend import Backend
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
    from ..interface import interface
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
    from wandb.proto import wandb_internal_pb2 as pb
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
    from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
    _descriptor.FieldDescriptor(
  File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
  File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/main.py", line 26, in go
    _ = mlflow.run(
  File "/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 307, in run
    _wait_for(submitted_run_obj)
  File "/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 324, in _wait_for
    raise ExecutionException("Run (ID '%s') failed" % run_id)
mlflow.exceptions.ExecutionException: Run (ID '1a76630e6bbc434f9cf6a81a83f227e0') failed

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
2022/09/04 11:20:57 ERROR mlflow.cli: === Run (ID '4dac944151114645ab75bbfaee227015') failed ===

udacity / cd0581-building-a-reproducible-model-workflow-exercises Goto Github PK