iter_files_distros(path=None, repeated_distro='first') will fail if one of the elements in sys.path is not a string. For example, my sys.path contained a PosixPath:
['/usr/local/bin', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages', PosixPath('/model_infrastructure')]
PosixPath objects do not implement an rstrip() method, and so the following happens [permalink]:
if path is None:
path = sys.path
# `path` is now a list containing some strings and in my accidental case, a PosixPath
for folder in path:
if folder.rstrip('/\\').endswith('.egg'):
# oops
I'll defer to someone with far more Python experience than I have to decide whether this is just a silly user error (i.e. entirely my fault), or something that should be handled within iter_files_distros(...).
Maybe it seems absurd that anything other than a string would find its way into sys.path, but for newcomers to Python (i.e. me, since ~3.5) who have grown up with pathlib, it doesn't feel too out of place. In fact, here's a PR.
For search engines and passers-by:
I had encountered this stacktrace in my Django + Dagster + MLflow + Gensim + Docker project:
AttributeError: 'PosixPath' object has no attribute 'rstrip'
File "/usr/local/lib/python3.8/site-packages/dagster/core/execution/plan/utils.py", line 42, in solid_execution_error_boundary
yield
File "/usr/local/lib/python3.8/site-packages/dagster/utils/__init__.py", line 382, in iterate_with_context
next_output = next(iterator)
File "/usr/local/lib/python3.8/site-packages/dagster/core/execution/plan/compute_generator.py", line 65, in _coerce_solid_compute_fn_to_iterator
result = fn(context, **kwargs) if context_arg_provided else fn(**kwargs)
File "/model_infrastructure/solids/gensim.py", line 160, in cyclic_word2vec
experiment = mlflow.get_experiment_by_name(experiment_name)
File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/fluent.py", line 815, in get_experiment_by_name
return MlflowClient().get_experiment_by_name(name)
File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/client.py", line 434, in get_experiment_by_name
return self._tracking_client.get_experiment_by_name(name)
File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 128, in get_experiment_by_name
return self.store.get_experiment_by_name(name)
File "/usr/local/lib/python3.8/site-packages/mlflow/store/tracking/rest_store.py", line 263, in get_experiment_by_name
response_proto = self._call_endpoint(GetExperimentByName, req_body)
File "/usr/local/lib/python3.8/site-packages/mlflow/store/tracking/rest_store.py", line 55, in _call_endpoint
return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)
File "/usr/local/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 163, in call_endpoint
response = http_request(
File "/usr/local/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 47, in http_request
from mlflow.tracking.request_header.registry import resolve_request_headers
File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/request_header/registry.py", line 39, in <module>
_request_header_provider_registry.register_entrypoints()
File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/request_header/registry.py", line 21, in register_entrypoints
for entrypoint in entrypoints.get_group_all("mlflow.request_header_provider"):
File "/usr/local/lib/python3.8/site-packages/entrypoints.py", line 236, in get_group_all
for config, distro in iter_files_distros(path=path):
File "/usr/local/lib/python3.8/site-packages/entrypoints.py", line 128, in iter_files_distros
if folder.rstrip('/\\').endswith('.egg'):
It took me a while to find the cause.
My BASE_DIR in Django's settings.py was a pathlib Path object:
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
For whatever reason, I had to append BASE_DIR to sys.path from within a Dagster solid:
sys.path.append(BASE_DIR)
This resulted in a PosixPath object being added to sys.path. As explained above, sys.path is expected presently to hold only strings, and so iter_files_distros(path=None) will just call rstrip(...) on each element without type consideration.
A solution is ensure that a string is appended to sys.path:
sys.path.append(str(BASE_DIR.absolute()))
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent