Comments (2)
VincentMagn
commented on June 16, 2024
Here is news, I have tried with WSL:
Reproduce
- Git clone code repo
- install all dep (poetry)
dvc list(same result)dvc import
$dvc import https://gitlab.XXX/data.git data/doe_full
Importing 'data/doe_full (https://gitlab.XXX/data.git)' -> 'doe_full'
ERROR: unexpected error - failed to load directory ('data', 'doe_full'):
[Errno 2] No such file or directory: '/home/YYY/modia/.dvc/cache/files/md5/77/ede8bd9897ebb5e3fb93e8a971b1bf.dir'
Environment information
$ dvc doctor
DVC version: 3.42.0 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
Subprojects:
dvc_data = 3.8.0
dvc_objects = 3.0.6
dvc_render = 1.0.1
dvc_task = 0.3.0
scmrepo = 2.0.4
Supports:
http (aiohttp = 3.9.2, aiohttp-retry = 2.8.3),
https (aiohttp = 3.9.2, aiohttp-retry = 2.8.3),
s3 (s3fs = 2023.12.2, boto3 = 1.34.27)
Config:
Global: /home/YYY/.config/dvc
System: /etc/xdg/dvc
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: ext4 on /dev/sdb
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/6edfbf2dbf3fa80a51187823e844f50bAdditional Information (if any):
$ dvc import -v https://gitlab.XXX/data.git data/doe_full
2024-01-29 11:51:58,593 DEBUG: v3.42.0 (pip), CPython 3.10.12 on Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
2024-01-29 11:51:58,593 DEBUG: command: /home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/bin/dvc import -v https://gitlab.XXX/data.git data/doe_full
2024-01-29 11:51:58,925 DEBUG: Removing output 'doe_full' of stage: 'doe_full.dvc'.
2024-01-29 11:51:58,926 DEBUG: Removing '/home/UUU/YYY/doe_full'
Importing 'data/doe_full (https://gitlab.XXX/data.git)' -> 'doe_full'
2024-01-29 11:51:58,930 DEBUG: Computed stage: 'doe_full.dvc' md5: '6454f7176c400f597cb06c2c82c53166'
2024-01-29 11:51:58,930 DEBUG: 'md5' of stage: 'doe_full.dvc' changed.
2024-01-29 11:51:58,931 DEBUG: Creating external repo https://gitlab.XXX/data.git@None
2024-01-29 11:51:58,931 DEBUG: erepo: git clone 'https://gitlab.XXX/data.git' to a temporary dir
2024-01-29 11:52:02,845 DEBUG: failed to load ('data', 'doe_full') from storage local (/home/UUU/YYY/.dvc/cache/files/md5) - [Errno 2] No such file or directory: '/home/UUU/YYY/.dvc/cache/files/md5/77/ede8bd9897ebb5e3fb93e8a971b1bf.dir'
Traceback (most recent call last):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 625, in _load_from_storage
_load_from_object_storage(trie, entry, storage)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 561, in _load_from_object_storage
obj = Tree.load(storage.odb, root_entry.hash_info, hash_name=storage.odb.hash_name)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/hashfile/tree.py", line 192, in load
with obj.fs.open(obj.path, "r") as fobj:
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 332, in open
return self.fs.open(path, mode=mode, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/local.py", line 131, in open
return open(path, mode=mode, encoding=encoding) # noqa: SIM115
FileNotFoundError: [Errno 2] No such file or directory: '/home/UUU/YYY/.dvc/cache/files/md5/77/ede8bd9897ebb5e3fb93e8a971b1bf.dir'
2024-01-29 11:52:02,849 ERROR: unexpected error - failed to load directory ('data', 'doe_full'): [Errno 2] No such file or directory: '/home/UUU/YYY/.dvc/cache/files/md5/77/ede8bd9897ebb5e3fb93e8a971b1bf.dir'
Traceback (most recent call last):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 625, in _load_from_storage
_load_from_object_storage(trie, entry, storage)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 561, in _load_from_object_storage
obj = Tree.load(storage.odb, root_entry.hash_info, hash_name=storage.odb.hash_name)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/hashfile/tree.py", line 192, in load
with obj.fs.open(obj.path, "r") as fobj:
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 332, in open
return self.fs.open(path, mode=mode, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/local.py", line 131, in open
return open(path, mode=mode, encoding=encoding) # noqa: SIM115
FileNotFoundError: [Errno 2] No such file or directory: '/home/UUU/YYY/.dvc/cache/files/md5/77/ede8bd9897ebb5e3fb93e8a971b1bf.dir'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/cli/__init__.py", line 211, in main
ret = cmd.do_run()
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/cli/command.py", line 27, in do_run
return self.run()
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/commands/imp.py", line 15, in run
self.repo.imp(
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/repo/imp.py", line 44, in imp
return self.imp_url(path, out=out, erepo=erepo, frozen=True, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/repo/__init__.py", line 65, in wrapper
return f(repo, *args, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/repo/scm_context.py", line 152, in run
return method(repo, *args, **kw)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/repo/imp_url.py", line 86, in imp_url
stage.run(jobs=jobs, no_download=no_download)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/funcy/decorators.py", line 47, in wrapper
return deco(call, *dargs, **dkwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/stage/decorators.py", line 44, in rwlocked
return call()
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/funcy/decorators.py", line 68, in __call__
return self._func(*self._args, **self._kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/stage/__init__.py", line 614, in run
self._sync_import(dry, force, kwargs.get("jobs", None), no_download)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/funcy/decorators.py", line 47, in wrapper
return deco(call, *dargs, **dkwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/stage/decorators.py", line 44, in rwlocked
return call()
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/funcy/decorators.py", line 68, in __call__
return self._func(*self._args, **self._kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/stage/__init__.py", line 651, in _sync_import
sync_import(self, dry, force, jobs, no_download)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/stage/imports.py", line 67, in sync_import
stage.deps[0].download(
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/dependency/base.py", line 50, in download
fs_download(self.fs, self.fs_path, to.fs_path, jobs=jobs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/fs/__init__.py", line 62, in download
from_infos = [
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/fs/__init__.py", line 62, in <listcomp>
from_infos = [
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 537, in find
yield from self.fs.find(path)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/fsspec/spec.py", line 497, in find
for _, dirs, files in self.walk(path, maxdepth, detail=True, **kwargs):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/fsspec/spec.py", line 420, in walk
listing = self.ls(path, detail=True, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc/fs/dvc.py", line 372, in ls
for info in dvc_fs.ls(dvc_path, detail=True):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 527, in ls
return self.fs.ls(path, detail=detail, **kwargs)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/fs.py", line 164, in ls
for key, info in self.index.ls(root_key, detail=True):
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 778, in ls
self._ensure_loaded(root_key)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 775, in _ensure_loaded
self._load(prefix, entry)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 724, in _load
self.onerror(entry, exc)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 652, in _onerror
raise exc
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 722, in _load
_load_from_storage(self._trie, entry, storage_info)
File "/home/UUU/.cache/pypoetry/virtualenvs/YYY-ugeuY09o-py3.10/lib/python3.10/site-packages/dvc_data/index/index.py", line 640, in _load_from_storage
raise DataIndexDirError(f"failed to load directory {entry.key}") from last_exc
dvc_data.index.index.DataIndexDirError: failed to load directory ('data', 'doe_full')
2024-01-29 11:52:02,918 DEBUG: Version info for developers:
DVC version: 3.42.0 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
Subprojects:
dvc_data = 3.8.0
dvc_objects = 3.0.6
dvc_render = 1.0.1
dvc_task = 0.3.0
scmrepo = 2.0.4
Supports:
http (aiohttp = 3.9.2, aiohttp-retry = 2.8.3),
https (aiohttp = 3.9.2, aiohttp-retry = 2.8.3),
s3 (s3fs = 2023.12.2, boto3 = 1.34.27)
Config:
Global: /home/UUU/.config/dvc
System: /etc/xdg/dvc
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: ext4 on /dev/sdb
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/6edfbf2dbf3fa80a51187823e844f50b
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2024-01-29 11:52:02,920 DEBUG: Analytics is enabled.
2024-01-29 11:52:02,970 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmp37bhog06', '-v']
2024-01-29 11:52:02,976 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmp37bhog06', '-v'] with pid 1928
2024-01-29 11:52:02,979 DEBUG: Removing '/tmp/tmpu37pn54idvc-clone'
2024-01-29 11:52:02,981 DEBUG: Removing '/tmp/tmpcw922rk2dvc-cache'```from dvc.
VincentMagn
commented on June 16, 2024
Found the error, it comes from rights.
from dvc.
Related Issues (20)
- Can I push to remote with different path? HOT 1
- Ability to configure the replication factor for the HDFS file system HOT 3
- unexpected error - b'worktreeConfig' HOT 6
- dvc exp run --run-all: only runs two experiments and then hangs (similar to #8165) HOT 3
- unchanged files showing as changed HOT 8
- `dvc data status` key error HOT 1
- dvc doctor fails in repo HOT 3
- metrics diff atol and rtol
- dvc fetch: hangs forever on "Fetching" HOT 5
- exp run: pygit2 in script does not find active branch and last commit HOT 2
- `dvc repro --pull`: invalidates cache HOT 8
- Dataset delete when using 'dvc add' due to low memory on pc HOT 2
- import: hangs when pulling many files from GCS remote and one fails HOT 2
- Stages with conditional dependency HOT 4
- bug: running exp with `--queue` always take workspace version for python deps HOT 4
- Feature proposal: `dvc artifacts get --show-url` HOT 1
- dvc.scm.CloneError: SCM error HOT 1
- Config setting studio.url is not respected
- pull: breaks if imported dir ends with "/" HOT 1
- dvc queue status doesn't report active workers HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dvc.