udacity / cd0581-building-a-reproducible-model-workflow-exercises Goto Github PK
View Code? Open in Web Editor NEWExercise Staters and solutions for cd0581-building-a-reproducible-model-workflow by Giacomo Vianello
License: Other
Exercise Staters and solutions for cd0581-building-a-reproducible-model-workflow by Giacomo Vianello
License: Other
To whoever taking this course in year 2024 (and beyond)
Until Udacity updates the contents, the course is pretty stuck in year 2022. I'm sharing what works for me on WSL (didn't try on *nix or Mac yet), but it probably will work for them too. I gave up "fighting" the versions issue after many hair-pulling evenings.
After installing the latest and the greatest, create a new conda environment with Python version:
$ conda create -n py37 python=3.7
$ conda activate py37
Then install some essential packages as specified as follows with pip (don't worry, this won't "taint" you system/base Python's site packages):
# if you are using WSL
$ pip install protobuf==3.20.3 mlflow=1.14.1 numpy=1.18.5 wandb==0.10.21
# if you are using Mac
$ pip install protobuf==3.20.3 mlflow=1.14.1 wandb==0.10.21
$ pip uninstall numpy
$ conda install numpy=1.19.5
# install the following regardless of the OS you are using
$ pip install hydra-core==1.3.2
In every conda.yml
file (remember those in sub-directories too), add the lines as stated in comments:
name: whatever
dependencies:
- python=3.7 # add this, always (or python=3.8 if you are using Mac)
- numpy=1.18.5 # add this when mlflow or pandas is listed as a dependency (or numpy=1.19.5 if you are using Mac)
- Jinja2=3.0.3 # add this when pandas-profiling is listed as a dependency
- jupyterlab=3.0.12 # remove this when it is listed as a dependency
- notebook # add this when jupyterlab is listed as a dependency; no need to specify the version
- pip:
- protobuf==3.20.3 # add this when wandb is listed as a dependency
With these, most of the dependencies stated in the exercise's conda.yml
should just work (as of Mar 2024). For those that don't (especially the transitive dependencies), search in conda forge or pypi for versions that were published in 2022 (preferably in Mar 2022 or earlier).
In the exercise where jupyterlab
dependency is replaced with notebook, change the command in the MLProject
to
jupyter notebook
.
When I have time, I'll try the same setup directly on Windows (instead of WSL) and Mac. Directly/Natively on Windows won't work 'cos of the use of $(echo {hydra_options})
in some of the MLProject files.
Mac users may have to make some package version updates discovered along the way. I only randomly chose the exercises to test the setup on Mac, and discovered that some dependencies won't work, e.g., matplotlib
3.2.2 needs to be replaced with 3.5.3 (for exercise 12).
Also, whoever having successes in setting things differently, please add to this "issue" to help future learners.
Steps to reproduce:
cd
into cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter
mlflow run . -P hydra_options="random_forest_pipeline.random_forest.max_depth=5"
(as per instructions in README.md
)This is the output
2022/09/05 09:30:58 INFO mlflow.utils.conda: === Creating conda environment mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 ===
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt']
Pip subprocess output:
Collecting hydra-joblib-launcher==1.1.2
Using cached hydra_joblib_launcher-1.1.2-py3-none-any.whl (5.2 kB)
Requirement already satisfied: hydra-core==1.0.* in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (1.0.6)
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: Click>=7.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (8.1.3)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: requests<3,>=2.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: configparser>=3.8.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (5.3.0)
Requirement already satisfied: GitPython>=1.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.1.27)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: protobuf>=3.12.0 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (4.21.5)
Requirement already satisfied: PyYAML in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (6.0)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (4.8)
Requirement already satisfied: omegaconf<2.1,>=2.0.5 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (2.0.6)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.0.5)
Collecting joblib>=0.14.0
Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Requirement already satisfied: typing-extensions in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from omegaconf<2.1,>=2.0.5->hydra-core==1.0.*->hydra-joblib-launcher==1.1.2->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 2)) (4.3.0)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting psutil>=5.0.0
Using cached psutil-5.9.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Requirement already satisfied: idna<3,>=2.5 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (1.25.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/condaenv.3t3s3ctv.requirements.txt (line 1)) (2022.6.15)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.6-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.5-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.4-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.3-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.2-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.1-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.0-py2.py3-none-any.whl (156 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Installing collected packages: subprocess32, shortuuid, sentry-sdk, psutil, promise, pathtools, joblib, wandb, hydra-joblib-launcher
Successfully installed hydra-joblib-launcher-1.1.2 joblib-1.1.0 pathtools-0.1.2 promise-2.3 psutil-5.9.2 sentry-sdk-1.9.0 shortuuid-1.0.9 subprocess32-3.5.4 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/05 09:31:49 INFO mlflow.projects.utils: === Created directory /tmp/tmp9nvxxkaf for downloading remote URIs passed to arguments of type 'path' ===
2022/09/05 09:31:49 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 1>&2 && python main.py $(echo random_forest_pipeline.random_forest.max_depth=5)' in run with ID '81bf8f661ec34669929321b0048accf9' ===
2022/09/05 09:31:50 INFO mlflow.utils.conda: === Creating conda environment mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f ===
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt']
Pip subprocess output:
Collecting omegaconf==2.0.6
Using cached omegaconf-2.0.6-py3-none-any.whl (36 kB)
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/random_forest/condaenv.sjr67ij7.requirements.txt (line 1)) (2.8.2)
Collecting Click>=7.0
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
Using cached psutil-5.9.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (284 kB)
Collecting PyYAML>=5.1.*
Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting requests<3,>=2.0.0
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Collecting typing-extensions
Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, typing-extensions, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb, omegaconf
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 omegaconf-2.0.6 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.2 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 typing-extensions-4.3.0 urllib3-1.26.12 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/05 09:32:48 INFO mlflow.projects.utils: === Created directory /tmp/tmpl8ujq6se for downloading remote URIs passed to arguments of type 'path' ===
2022/09/05 09:32:48 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f 1>&2 && python run.py --train_data exercise_6/data_train.csv:latest \
--model_config /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/outputs/2022-09-05/09-31-50/random_forest_config.yml' in run with ID '9f795a4d42d7448296934713201fc13a' ===
Traceback (most recent call last):
File "run.py", line 15, in <module>
import wandb
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/__init__.py", line 38, in <module>
from wandb import sdk as wandb_sdk
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/__init__.py", line 12, in <module>
from .wandb_init import init # noqa: F401
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
from .backend.backend import Backend
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
from ..interface import interface
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
from wandb.proto import wandb_internal_pb2 as pb
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
_descriptor.FieldDescriptor(
File "/home/fanta/.local/miniconda3/envs/mlflow-70e71a7afdc413046f59e7b87abd03dc50d8745f/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_11/starter/main.py", line 26, in go
_ = mlflow.run(
File "/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 307, in run
_wait_for(submitted_run_obj)
File "/home/fanta/.local/miniconda3/envs/mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 324, in _wait_for
raise ExecutionException("Run (ID '%s') failed" % run_id)
mlflow.exceptions.ExecutionException: Run (ID '9f795a4d42d7448296934713201fc13a') failed
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
2022/09/05 09:32:49 ERROR mlflow.cli: === Run (ID '81bf8f661ec34669929321b0048accf9') failed ===
Note: almost all of the exercises are broken. I will stop filing bugs with this one.
From the exercise_3 solution run
mlflow run .
and you get an error containing TypeError: Descriptors cannot not be created directly.
The issue is that the conda.yml
directs MLFlow to use incompatible versions of some python packages.
Note that just forcing protobuf to version 3.20.*, as indicated in Udacity's Knowledge, still brings about errors.
At this time, a fix is to replace the - pip
section at the end of files
download_data/conda.yml
process_data/conda.yml
with the following
- pip:
- wandb == 0.13.2
- protobuf==3.20.*
The starter code of the same exercise needs to be fixed too.
Also see here.
> mlflow run .
2022/09/04 11:18:25 INFO mlflow.utils.conda: === Creating conda environment mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6 ===
Collecting package metadata (repodata.json): done
Solving environment: done
Downloading and Extracting Packages
greenlet-1.1.3 | 91 KB | ################################################################################################################################################################################# | 100%
click-8.1.3 | 146 KB | ################################################################################################################################################################################# | 100%
protobuf-4.21.5 | 342 KB | ################################################################################################################################################################################# | 100%
omegaconf-2.0.6 | 103 KB | ################################################################################################################################################################################# | 100%
gunicorn-20.1.0 | 119 KB | ################################################################################################################################################################################# | 100%
sqlalchemy-1.4.40 | 2.4 MB | ################################################################################################################################################################################# | 100%
mlflow-1.14.1 | 4.2 MB | ################################################################################################################################################################################# | 100%
chardet-3.0.4 | 170 KB | ################################################################################################################################################################################# | 100%
docker-py-5.0.3 | 188 KB | ################################################################################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: / Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: GitPython>=1.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.1.27)
Requirement already satisfied: Click>=7.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (8.1.3)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: configparser>=3.8.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (5.3.0)
Requirement already satisfied: requests<3,>=2.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.24.0)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: protobuf>=3.12.0 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (4.21.5)
Requirement already satisfied: PyYAML in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (6.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.0.5)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting psutil>=5.0.0
Using cached psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (2022.6.15)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/condaenv.o2q2aa56.requirements.txt (line 1)) (1.25.11)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.6-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.5-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.4-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.3-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.2-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.1-py2.py3-none-any.whl (157 kB)
Using cached sentry_sdk-1.9.0-py2.py3-none-any.whl (156 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Installing collected packages: subprocess32, shortuuid, sentry-sdk, psutil, promise, pathtools, wandb
Successfully installed pathtools-0.1.2 promise-2.3 psutil-5.9.1 sentry-sdk-1.9.0 shortuuid-1.0.9 subprocess32-3.5.4 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/04 11:19:53 INFO mlflow.projects.utils: === Created directory /tmp/tmpfn3ljdp3 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 11:19:53 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6 1>&2 && python main.py $(echo '')' in run with ID '4dac944151114645ab75bbfaee227015' ===
2022/09/04 11:19:54 INFO mlflow.utils.conda: === Creating conda environment mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f ===
Collecting package metadata (repodata.json): done
Solving environment: done
Downloading and Extracting Packages
plotly-4.4.1 | 4.1 MB | ################################################################################################################################################################################# | 100%
retrying-1.3.3 | 11 KB | ################################################################################################################################################################################# | 100%
python-3.8.13 | 25.2 MB | ################################################################################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: - Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/random_forest/condaenv.lj2wi_mu.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
Using cached psutil-5.9.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (284 kB)
Collecting requests<3,>=2.0.0
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/04 11:20:56 INFO mlflow.projects.utils: === Created directory /tmp/tmpq_hnq_82 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 11:20:56 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f 1>&2 && python run.py --train_data exercise_6/data_train.csv:latest \
--model_config /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/outputs/2022-09-04/11-19-54/random_forest_config.json' in run with ID '1a76630e6bbc434f9cf6a81a83f227e0' ===
Traceback (most recent call last):
File "run.py", line 15, in <module>
import wandb
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/__init__.py", line 38, in <module>
from wandb import sdk as wandb_sdk
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/__init__.py", line 12, in <module>
from .wandb_init import init # noqa: F401
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
from .backend.backend import Backend
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
from ..interface import interface
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
from wandb.proto import wandb_internal_pb2 as pb
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
_descriptor.FieldDescriptor(
File "/home/fanta/.local/miniconda3/envs/mlflow-04dc18e87776aad288f8cabb3fd2dc65eef7059f/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Traceback (most recent call last):
File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-4-training-validation-experiment-tracking/exercises/exercise_10/solution/main.py", line 26, in go
_ = mlflow.run(
File "/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 307, in run
_wait_for(submitted_run_obj)
File "/home/fanta/.local/miniconda3/envs/mlflow-35c2fef811a8212520e1943fc147f38ec6f203c6/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 324, in _wait_for
raise ExecutionException("Run (ID '%s') failed" % run_id)
mlflow.exceptions.ExecutionException: Run (ID '1a76630e6bbc434f9cf6a81a83f227e0') failed
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
2022/09/04 11:20:57 ERROR mlflow.cli: === Run (ID '4dac944151114645ab75bbfaee227015') failed ===
As per the README file, the test should fail when ks_alpha is set to 0.9 (or higher), instead it succeeds.
-> mlflow run . -P reference_artifact=exercise_6/data_train.csv:latest -P sample_artifact=exercise_6/data_test.csv:latest -P ks_alpha=0.9
2022/09/04 10:02:35 INFO mlflow.utils.conda: Conda environment mlflow-d8f23e2b65d43a5000d54c81519b844ddb5d8310 already exists.
2022/09/04 10:02:35 INFO mlflow.projects.utils: === Created directory /tmp/tmpc0japcqw for downloading remote URIs passed to arguments of type 'path' ===
2022/09/04 10:02:35 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-d8f23e2b65d43a5000d54c81519b844ddb5d8310 1>&2 && pytest -s -vv . --reference_artifact exercise_6/data_train.csv:latest \
--sample_artifact exercise_6/data_test.csv:latest \
--ks_alpha 0.9' in run with ID 'b4492cd98a504bada366ed9b1ae6fe53' ===
wandb: Currently logged in as: fantauzzi. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.13.2
wandb: Run data is saved locally in /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_9/solution/wandb/run-20220904_100237-3g3wtwt4
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run leafy-deluge-33
wandb: ⭐️ View project at https://wandb.ai/fantauzzi/exercise_9
wandb: 🚀 View run at https://wandb.ai/fantauzzi/exercise_9/runs/3g3wtwt4
=================================================================================================== test session starts ====================================================================================================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 -- /home/fanta/.local/virtualenv/python3/bin/python
cachedir: .pytest_cache
rootdir: /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_9/solution
collected 1 item
test_data.py::test_kolmogorov_smirnov PASSED
==================================================================================================== 1 passed in 2.10s =====================================================================================================
wandb: Waiting for W&B process to finish... (success).
wandb:
wandb: Synced leafy-deluge-33: https://wandb.ai/fantauzzi/exercise_9/runs/3g3wtwt4
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20220904_100237-3g3wtwt4/logs
2022/09/04 10:02:48 INFO mlflow.projects: === Run (ID 'b4492cd98a504bada366ed9b1ae6fe53') succeeded ===
When calling mflow using the provided solution conda.yml
results in an error.
I use Python version 3.8.16 (followed instructions at beginning of the course).
The call in the solution folder:
mlflow run . -P file_url=https://raw.githubusercontent.com/scikit-learn/scikit-learn/4dfdfb4e1bb3719628753a4ece995a1b2fa5312a/sklearn/datasets/data/iris.csv -P artifact_name=iris -P artifact_description="This data sets consists of 3 different types of irises' (Setosa, Versicolour, and Virginica) petal and sepal length"
Output:
Traceback (most recent call last):
File "XXX\Udacity\cd0581-building-a-reproducible-model-workflow-exercises\lesson-1-machine-learning-pipelines\exercises\exercise_2\solution\download_data.py", line 5, in <module>
import wandb
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packag from wandb import sdk as wandb_sdk
File "XXX\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\__init__.py", line 12, in <module>
from .wandb_init import init # noqa: F401
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\wandb_init.py", line 29, in <module>
from .backend.backend import Backend
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\backend\backend.py", line 17, in <module>
from ..interface import interface
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\sdk\interface\interface.py", line 18, in <module>
from wandb.proto import wandb_internal_pb2 as pb
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\proto\wandb_internal_pb2.py", line 15, in <module>
from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\wandb\proto\wandb_telemetry_pb2.py", line 34, in <module>
_descriptor.FieldDescriptor(
File "XXX\AppData\Local\anaconda3\envs\mlflow-4b67c93d2a95df2e00cbf3c9f644d2e3dada00e0\Lib\site-packages\google\protobuf\descriptor.py", line 561, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
2023/05/23 14:11:33 ERROR mlflow.cli: === Run (ID '9b21011b2dd6459690460967d55c7ff2') failed ===
The proposed solution of downgrading protobuf to 3.20.x dit not fix this environment definition, but resulted in new errors (MutableSet
not being defined in collections
). Please fix/update the learning materials.
Exercise 11 fails with the following return error:
$ mlflow run . -P hydra_options="random_forest_pipeline.random_forest.max_depth=5"
2023/05/20 18:18:32 INFO mlflow.utils.conda: Conda environment mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 already exists.
2023/05/20 18:18:32 INFO mlflow.projects.utils: === Created directory C:\Users\giannis\AppData\Local\Temp\tmpftu8jmwr for downloading remote URIs passed to arguments of type 'path' ===
2023/05/20 18:18:32 INFO mlflow.projects.backend.local: === Running command 'conda activate mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76 && python main.py $(echo random_forest_pipeline.random_forest.max_depth=5)' in run with ID 'fabf4ebaf7fc40649a25570246409951' ===
C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py:49: FutureWarning: In the future np.object
will be defined as the corresponding NumPy scalar.
binary = (7, np.dtype("bytes"), "BinaryType", np.object)
Traceback (most recent call last):
File "C:\Users\giannis\Desktop\ML-DevOps-2\lesson-4-training-validation-experiment-tracking\exercises\exercise_11\starter\main.py", line 3, in
import mlflow
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow_init_.py", line 52, in
import mlflow.fastai as fastai # noqa: E402
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\fastai.py", line 22, in
from mlflow import pyfunc
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\pyfunc_init_.py", line 219, in
import mlflow.pyfunc.model
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\pyfunc\model.py", line 17, in
from mlflow.models import Model
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\models_init_.py", line 25, in
from .signature import ModelSignature, infer_signature
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\models\signature.py", line 12, in
from mlflow.types.schema import Schema
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types_init_.py", line 6, in
from .schema import DataType, ColSpec, Schema, TensorSpec
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py", line 20, in
class DataType(Enum):
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\mlflow\types\schema.py", line 49, in DataType
binary = (7, np.dtype("bytes"), "BinaryType", np.object)
File "C:\Users\giannis\anaconda3\envs\mlflow-7ff53cff2353e8a846a00d29e1b3cddc025e8f76\lib\site-packages\numpy_init_.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'object'.
np.object
was a deprecated alias for the builtin object
. To avoid this error in existing code, use object
by itself. Doing this will not modify any behavior and is safe.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
2023/05/20 18:18:35 ERROR mlflow.cli: === Run (ID 'fabf4ebaf7fc40649a25570246409951') failed ===
(venv)
With reference to file https://github.com/udacity/cd0581-building-a-reproducible-model-workflow-exercises/blob/main/lesson-5-final-pipeline-release-and-deploy/exercises/exercise_14/solution/main.py
the call to the sub-project segregate
omits passing the random seed, I think the call should pass it.
Also, at line 91 it is passing the wrong random seed to the random_forest
sub-project. I believe it can be corrected like this
"random_seed": config['random_forest_pipeline']['random_forest']['random_state'],
After running
mlflow run .
the code stops with an error containing
Jupyter command
jupyter-notebook
not found.
I believe there are multiple issues with the provided conda.yml
. The same problem is in the starter code for the same exercise.
Here there is a conda.yml
that I have used to complete the exercise and also to run the solution.
name: exercise_4
channels:
- conda-forge
- defaults
dependencies:
- jupyter=1.0.0
- seaborn=0.11.2
- pip=22.1.2
- pandas=1.4.4
- pandas-profiling=3.2.0
- fastparquet=0.8.3
- pyarrow=8.0.0
- pip:
- wandb==0.13.2
`name: download_data
channels:
But it should probably be
`name: process_data
channels:
even when running the solution, I get the following traceback:
Traceback (most recent call last):
File "/path/to/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/starter/run.py", line 135, in <module>
go(args)
File "/path/to/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/starter/run.py", line 23, in go
artifact_path = artifact.file()
File "/path/to/anaconda3/envs/mlflow-9bfe8e3af68696f10e0ad9a623577b2921db6aba/lib/python3.9/site-packages/wandb/apis/public.py", line 3946, in file
return self._download_file(list(manifest.entries)[0], root=root)
IndexError: list index out of range
Steps to reproduce:
cd
into cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution
mlflow run . -P input_artifact="exercise_5/preprocessed_data.csv:latest" \
-P artifact_root="data" \
-P test_size=0.3 \
-P stratify="genre"
Execution stop with an error. The same must affect the starter code.
2022/09/03 10:32:07 INFO mlflow.utils.conda: === Creating conda environment mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e ===
Collecting package metadata (repodata.json): done
Solving environment: done
Downloading and Extracting Packages
python-3.9.13 | 26.7 MB | ############################################################################################################################# | 100%
scikit-learn-0.24.1 | 7.6 MB | ############################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/condaenv.g6_6seht.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Collecting requests<3,>=2.0.0
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/03 10:33:24 INFO mlflow.projects.utils: === Created directory /tmp/tmp3onf6gyi for downloading remote URIs passed to arguments of type 'path' ===
2022/09/03 10:33:24 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e 1>&2 && python run.py --input_artifact exercise_5/preprocessed_data.csv:latest \
--artifact_root data \
--artifact_type raw_data \
--test_size 0.3 \
--random_state 42 \
--stratify genre' in run with ID '509c6be194774515ac0139f174adcb82' ===
Traceback (most recent call last):
File "/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-2-data-exploration-and-preparation/exercises/exercise_6/solution/run.py", line 8, in <module>
import wandb
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/__init__.py", line 38, in <module>
from wandb import sdk as wandb_sdk
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/__init__.py", line 12, in <module>
from .wandb_init import init # noqa: F401
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/wandb_init.py", line 29, in <module>
from .backend.backend import Backend
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/backend/backend.py", line 17, in <module>
from ..interface import interface
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/sdk/interface/interface.py", line 18, in <module>
from wandb.proto import wandb_internal_pb2 as pb
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/proto/wandb_internal_pb2.py", line 15, in <module>
from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/wandb/proto/wandb_telemetry_pb2.py", line 34, in <module>
_descriptor.FieldDescriptor(
File "/home/fanta/.local/miniconda3/envs/mlflow-b536c99cab3f07bfeb0a1ad9e153575c07cd087e/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
2022/09/03 10:33:25 ERROR mlflow.cli: === Run (ID '509c6be194774515ac0139f174adcb82') failed ===
The same issue should affect also the starter code.
Steps to reproduce
cd
into cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution
mlops run .
Get the error:
> mlflow run .
2022/09/03 13:55:31 INFO mlflow.utils.conda: === Creating conda environment mlflow-a3af6a16a95faee04af2c40a1852613ae4973185 ===
Collecting package metadata (repodata.json): done
Solving environment: done
Downloading and Extracting Packages
pluggy-0.13.1 | 29 KB | ############################################################################################################################# | 100%
more-itertools-8.14. | 45 KB | ############################################################################################################################# | 100%
pytest-6.2.2 | 432 KB | ############################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.21
Using cached wandb-0.10.21-py2.py3-none-any.whl (2.0 MB)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages (from wandb==0.10.21->-r /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution/condaenv.bz6j_xry.requirements.txt (line 1)) (1.16.0)
Collecting Click>=7.0
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting configparser>=3.8.1
Using cached configparser-5.3.0-py3-none-any.whl (19 kB)
Collecting docker-pycreds>=0.4.0
Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython>=1.0.0
Using cached GitPython-3.1.27-py3-none-any.whl (181 kB)
Collecting gitdb<5,>=4.0.1
Using cached gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting promise<3,>=2.0
Using cached promise-2.3-py3-none-any.whl
Collecting protobuf>=3.12.0
Using cached protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting psutil>=5.0.0
Using cached psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Collecting requests<3,>=2.0.0
Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting charset-normalizer<3,>=2
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sentry-sdk>=0.4.0
Using cached sentry_sdk-1.9.7-py2.py3-none-any.whl (157 kB)
Collecting shortuuid>=0.5.0
Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting smmap<6,>=3.0.1
Using cached smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting subprocess32>=3.5.3
Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting pathtools
Using cached pathtools-0.1.2-py3-none-any.whl
Collecting PyYAML
Using cached PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
Installing collected packages: smmap, urllib3, idna, gitdb, charset-normalizer, certifi, subprocess32, shortuuid, sentry-sdk, requests, PyYAML, psutil, protobuf, promise, pathtools, GitPython, docker-pycreds, configparser, Click, wandb
Successfully installed Click-8.1.3 GitPython-3.1.27 PyYAML-6.0 certifi-2022.6.15 charset-normalizer-2.1.1 configparser-5.3.0 docker-pycreds-0.4.0 gitdb-4.0.9 idna-3.3 pathtools-0.1.2 promise-2.3 protobuf-4.21.5 psutil-5.9.1 requests-2.28.1 sentry-sdk-1.9.7 shortuuid-1.0.9 smmap-5.0.0 subprocess32-3.5.4 urllib3-1.26.12 wandb-0.10.21
done
#
# To activate this environment, use
#
# $ conda activate mlflow-a3af6a16a95faee04af2c40a1852613ae4973185
#
# To deactivate an active environment, use
#
# $ conda deactivate
Retrieving notices: ...working... done
2022/09/03 13:56:53 INFO mlflow.projects.utils: === Created directory /tmp/tmpewqiu835 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/03 13:56:53 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-a3af6a16a95faee04af2c40a1852613ae4973185 1>&2 && pytest -s -vv .' in run with ID 'e69d9c038757490e847e2fe3d2f28cc2' ===
========================================================================= test session starts ==========================================================================
platform linux -- Python 3.9.13, pytest-6.2.2, py-1.11.0, pluggy-0.13.1 -- /home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/bin/python
cachedir: .pytest_cache
rootdir: /home/fanta/workspace/cd0581-building-a-reproducible-model-workflow-exercises/lesson-3-data-validation/exercises/exercise_7/solution
collected 0 items / 1 error
================================================================================ ERRORS ================================================================================
____________________________________________________________________ ERROR collecting test_data.py _____________________________________________________________________
test_data.py:2: in <module>
import wandb
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/__init__.py:38: in <module>
from wandb import sdk as wandb_sdk
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/__init__.py:12: in <module>
from .wandb_init import init # noqa: F401
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/wandb_init.py:29: in <module>
from .backend.backend import Backend
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/backend/backend.py:17: in <module>
from ..interface import interface
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/sdk/interface/interface.py:18: in <module>
from wandb.proto import wandb_internal_pb2 as pb
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/proto/wandb_internal_pb2.py:15: in <module>
from wandb.proto import wandb_telemetry_pb2 as wandb_dot_proto_dot_wandb__telemetry__pb2
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/wandb/proto/wandb_telemetry_pb2.py:34: in <module>
_descriptor.FieldDescriptor(
/home/fanta/.local/miniconda3/envs/mlflow-a3af6a16a95faee04af2c40a1852613ae4973185/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in __new__
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
======================================================================= short test summary info ========================================================================
ERROR test_data.py - TypeError: Descriptors cannot not be created directly.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================== 1 error in 0.30s ===========================================================================
2022/09/03 13:56:53 ERROR mlflow.cli: === Run (ID 'e69d9c038757490e847e2fe3d2f28cc2') failed ===
Pandas profiling library is deprecated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.