dinobektesevic / ci_hsc_example Goto Github PK

During "source ingest_data.bash" an error is thrown (see below). It is true the "fakes" directory does not exist, and the requested parq files are not in the directory structure.

lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:
Traceback (most recent call last):
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/cli/cmd/commands.py", line 125, in butler_import
script.butlerImport(*args, **kwargs)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/script/butlerImport.py", line 67, in butlerImport
butler.import_(
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/direct_butler.py", line 1865, in import_
doImport(stream)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/direct_butler.py", line 1854, in doImport
backend.load(
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/transfers/_yaml.py", line 460, in load
resolvedRefs = self.registry._importDatasets(datasets)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/utils.py", line 61, in inner
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/registry/sql_registry.py", line 1164, in importDatasets
refs = list(storage.import(runRecord, expandedDatasets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/registry/datasets/byDimensions/storage.py", line 669, in import
self._validateImport(tmp_tags, run)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/registry/datasets/byDimensions/_storage.py", line 763, in validateImport
raise ConflictingDefinitionError(
lsst.daf.butler.registry.exceptions.ConflictingDefinitionError: Dataset a29702e5-672d-39f8-b3a7-b5d6ea411ceb was provided with type ID 20 in run 'HSC/external', but was already defined with type 'jointcalSkyWcsCatalog' in run RunRecord(key=18, name='HSC/external').
lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:
Traceback (most recent call last):
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/cli/cmd/commands.py", line 125, in butler_import
script.butlerImport(*args, **kwargs)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/script/butlerImport.py", line 67, in butlerImport
butler.import(
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/direct_butler.py", line 1865, in import
doImport(stream)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/direct_butler.py", line 1854, in doImport
backend.load(
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/transfers/_yaml.py", line 475, in load
datastore.ingest(*fileDatasets, transfer=transfer)
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/datastore/_datastore.py", line 758, in ingest
prepData = self._prepIngest(*datasets, transfer=transfer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/datastores/fileDatastore.py", line 1148, in _prepIngest
dataset.path = self._standardizeIngestPath(dataset.path, transfer=transfer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/colinchandler/dino/ci_hsc_example/lsst_stack/stack/miniconda3-py38_4.9.2-7.0.1/DarwinX86/daf_butler/gfaf02a00f9+c4141ea9c4/python/lsst/daf/butler/datastores/fileDatastore.py", line 976, in _standardizeIngestPath
raise FileNotFoundError(
FileNotFoundError: Resource at file:///Users/colinchandler/dino/ci_hsc_example/testdata_ci_hsc/fakes/fakeSourceCat_0.parq does not exist; note that paths to ingest are assumed to be relative to file:///Users/colinchandler/dino/ci_hsc_example/butler_repo/ unless they are absolute.