livmats / dtool-smb Goto Github PK

I recognized with Sarah that an empty password in ".config/dtool/dtool.json" leads to an error when executing dtool ls smb://isilon
We see the following error:

[fr_se159@login1 dtool]$ dtool ls smb://isilon
Traceback (most recent call last):
File "/home/fr/fr_fr/fr_se159/.local/bin/dtool", line 11, in
load_entry_point('dtool-cli==0.7.0', 'console_scripts', 'dtool')()
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/dtool_info/dataset.py", line 182, in ls
if dtoolcore._is_dataset(uri, CONFIG_PATH):
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/dtoolcore/init.py", line 85, in _is_dataset
return storage_broker.has_admin_metadata()
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/dtool_smb/storagebroker.py", line 436, in has_admin_metadata
return self._path_exists(self.get_admin_metadata_key())
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/dtool_smb/storagebroker.py", line 280, in _path_exists
self.conn.getAttributes(self.service_name, path)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/smb/SMBConnection.py", line 265, in getAttributes
self._getAttributes(service_name, path, cb, eb, timeout)
File "/home/fr/fr_fr/fr_se159/.local/lib/python3.6/site-packages/smb/base.py", line 741, in _getAttributes_SMB2
raise NotReadyError('SMB connection not authenticated')
smb.base.NotReadyError: SMB connection not authenticated

We have tested: <"">, <"None">, < None >, <>
as empty passwords where everything in between <> is meant.

Putting a dataset on a SMB share works well, but subsequently retrieving it raises an error, which turns out to be a bit of a rabbit hole.

Create a test dataset test_dataset:

.
├── data
│   ├── bar
│   ├── baz
│   │   └── baz
│   └── foo
└── README.yml

Copy the dataset to a SMB share:

dtool cp test_dataset smb://<name>

Try to retrieve the dataset:

dtool cp smb://<share>/<hash> ./

dtool will raise NotImplementedError because the SMB storage broker does not have a method
get_item_abspath. It seems at first that this is a trivial method name problem, because there is a
method get_item_path. I think this method should be renamed. Moreover, the method is currently broken because it sets relpath = hitem["relpath"], where hitem is undefined. I think it should look as follows:

    def get_item_abspath(self, identifier):
        """Return absolute path at which item content can be accessed.

        :param identifier: item identifier
        :returns: absolute path from which the item content can be accessed
        """
        manifest = self.get_manifest()
        relpath = manifest["items"][identifier]["relpath"]
        item_path = os.path.join(self._data_path, relpath)
        return item_path

However, even with this fix, the transfer still fails. It seems that dtool calls storagebroker.put_item from its default storage broker (rather than dtool_smb.storagebroker.put_item). And even it it were
to call dtool_smb.storagebrooker, it would result in a call to pysmb.SMBConnection.storeFile,
which is the correct method only for transfer to the SMB share. For transfer from SMB to the local machine, we would have to use pysmb.SMBConnection.retrieveFile. In my opinion, this points to
a problem with the BaseStorageBroker base class: why is there a method put_item, but no get_item?