chstaiger / ibridges-gui Goto Github PK

In the tab irodsUpDownload:

When data is uploaded and one clicks on the main window, the upload dialog vanishes to the back. Pin to front.

found myself looking for an 'up' button to navigate through the collection hierarchy.

gui.irodsTreeView.IrodsModel gives poor performance for more complex iRODS trees. Determine if entire tree is loaded at once or if something completely different is causing the poor performance.

The overlapping functionality from the Icommands and python iRODS client has already been moved to subclasses.
The next question is what to do with the anonymous user.

Anonymous users should only be allowed to use a subset of the functionalities: login, see and download data, (anything else?)
In the current version the functions are reimplemented for the anonymous user.
They should be combined where possible to remove the duplicate code.

Currently the config item ui_tabs is an unordered list and in the code the order of the appearance of the tabs is hardcoded.
Turn the ui_tabs in an ordered list or dictionary that determines the order of appearance of tabs.

Currently there are three types available:

• Python irods client (default)
• Icommands
• Anonymous user
  There is a large amount of functional overlap between the different classes.
  As a result there is a considerable amount of duplicate code.

Right now the Icommands are only used for operations the python client cannot do, or not as efficiently (iRsync).
In this case it makes sense to automatically switch to Icommands if they are available.
It's a design choice, but if we decide to keep working in this way, I suggest to remove 'icommands' option from the mainmenu.

• move ic.session.foo() calls to appropriate methods in the connector classes (or only manager class?)
• reevaluate where session functionality (data, properties, etc.) is best placed (session.py, manager.py, elsewhere?)
• ensure the current session validity is checked consistently

• the path functions have changed --> update developer.md
• Add debug option to ibridges_config.json

We cannot add metadata to objects or values. The class meta is always None. So we cannot get the functions from that class:

    irods_conn.add_metadata(items, 'ELN', self.elab.metadataUrl)
  File "=iBridges-Gui/irodsConnector/manager.py", line 283, in add_metadata
    return self.meta.add(items, key, value, units)
AttributeError: 'NoneType' object has no attribute 'add'

The session.py only reads the last irods_env from the ibridges_conf, but when we want to set a new irods_env_file it gets ignored.

When you want to provide the necessary information to login with a ticket, the GUI crashes when clicking too wildly around.
The whole window does not seem to be connected well any longer to the rest of the GUI. When one wants to close the session the application crashes.

[2022-10-20 09:55:53,661] {irodsBrowser.py:312} INFO - ERROR in Browser
Traceback (most recent call last):
File "/Users/staig001/git-repos/iBridges-Gui/gui/irodsBrowser.py", line 309, in fillInfo
self.__fillACLs(value, path)
File "/Users/staig001/git-repos/iBridges-Gui/gui/irodsBrowser.py", line 162, in __fillACLs
self.owner_label.setText("Owner: "+item.owner_name)
UnboundLocalError: local variable 'item' referenced before assignment

This issue has a broad scope and we will need to decide when it should be done and discuss how it should be done. Sooner is better than later. And needless to say, any new code should certainly follow a convention. I have implicitly proposed PEP 8 as it is the de facto standard for general Python code formatting, and similarly NumPy docstrings.

Consistently apply the PEP 8 conventions across the board in all manually written Python code and filenames/directories following:

https://peps.python.org/pep-0008/

Certain external library naming conventions cannot be changed, namely PyQt, but those that can, the entities in the UI files that intersect with the code (e.g., target widget names), should adhere where possible.

I can propose names of top-level modules, classes, and similar so we can discuss them prior to implementation.

It may also be an idea to pick a convention for the directory structure that is more typical of open-source Python projects, but still allows the various distribution, testing, and other tools to function properly. For instance, something like this may be ideal:

• iBridges (repository name)
  • doc/
  • ibridges/
    • connector/
    • gui/
    • test/
    • utils/
    • ibridges.py
  • install.py
  • LICENSE
  • README.md
  • requirements.txt

As for docstrings, ALWAYS USE DOCSTRINGS. The NumPy convention for docstrings is quite clear and readable:

https://numpydoc.readthedocs.io/en/latest/format.html

Current tasks:

• update internal naming (class/method/function/variable)
• iBridges.py
• iBridgesCli.py
• gui/*py
• gui/ui_files/*
• irodsConnector/*py
• utils/*py

Other tasks:

• update external (file/directory) naming?
• restructure modules?

this seems to be associated with the elab functionality. but always using upper() changes existing data and may not be what the user wants. perhaps should be implemented with an option somewhere, rather than always on.

The pyinstaller turns the code into an executable for Windows and Mac. However, all files and dependencies are dumped in one big folder where users would have to search for the executable file.

We need a batter structured folder or a shortcut on the Desktop or in the menu with which the GUI can be started.

Currently we dump all of the configurations into the irods_environment.json
We need to seperate then such that the irods env file only contains iRODS keywords and create a separate iBridges config file. The CLI already deals with a separate config file and can maybe serve as an example which we can extend.

irods env (requested by GUI):

• irods_default_resource
• irods_home (if users do not have an own home directory, e.g. YODA servers only deal with group home directories)

iBridges config:

• ui_tabs
• force_unknown_free_space
• webdav server
• ui_ienvFilePath
• Elab journal token
• Other application tokens to access software like ElabJournal

IRODS allows users to change the portnumber from 1247. iBridges support this functionality except for the connection check:
https://github.com/chStaiger/iBridges-Gui/blob/1d38b5506536684afd1033d9faebd0db61f81801/utils/utils.py#LL748C1-L768C25

We should update this function to use the 'irods_port' from the environment json.

At the moment we rely on a valid irods_environment.json and in the future another bridges.config file.
Create a mask which loads all available information from a given irods_environment.json and ibridges.config into a maksk where users can edit items and save them to the appropriate files.

The python-irodsclient will use parallel transfer (multiple threads per file) automatically since 0.9.0.

I believe the icommands checking and switching can be removed from this application.

Currently users need to copy paste their token to the ELN tab --> Read token from .ibridges/config.json and remove textfield

Some malformed logging statements were caused by blindly converting from the original print statements. The main issue is from statements such as:

print(a, b, c)

converted to:

logging.info(a, b, c)

causing an error because arguments after the first (the message) are considered to be string formatting arguments to the first.

in the tab UpDownload:
When diffs are calculated or data is transferred, the app crashes when user clicks on "cancel"

Generic type hinting of classes with a subscript is very handy:

    def get_diff_upload(self,
                        source: Union[str, Path],
                        target: Union[iRODSDataObject, iRODSCollection, str],
                        arguments: str = None) -> list[SyncResult]:

but is not supported below Python 3.9. Removing support for Python < 3.9 is not possible as many mainstream Linux distributions don't allow Python > 3.8 out of the box.

What should be done?

When selecting a 'different' row in the browser view, the information panel at the bottom clears the existing information, but does not repopulate with the new row's information (preview, metadata, resources, etc.) unless the data object name is clicked on directly.

Selecting elsewhere on a row makes it 'highlight', and clears the bottom, but does not populate the bottom panel.

if the .irods directory does not already exist locally, an error is thrown

Users can load search results into the iRODS browser tab main table. However, that does not fit entirely since the iRODS path can differ for different resulting collections and folders. Where to store them?

The text field iRODS Path is not valid for search results and currently we store the full path in the name column, adding a different logic to how to deal with the information in the table.

--> Redesign

As far as I can see we use the contextContainer class only for:

1. shortening variable names from Context
2. make sure all frontend class have all parameters from context through inheriting from contextContainer

In PR #98 I removed the contextContainer as an experiment. It is possible and apart from the longer variable names it does not have any disadvantages as far as I can see. Point 2 is covered by context being a singleton.

We could remove the contextContainer and think of another way to shorten the variable names?!

• Design
• Designdoc
• Implementation
• ...

Create a tab where the user can create/update/delete a file or folder to monitor.
When triggered, the file/folder is syncronised to a remote irods instance.

File access axioms:

1. Ignore files from a treatment belonging to the current date
2. Ignore files that are locked

Things to configure per file/folder to monitor:

1. A (local) source file/folder
2. A (remote) destination path
3. A cron notation?
4. Checkbox to delete synchronized files/folder
   • delete a file only if MD5 checksum has been validated
   • delete files in order from old to young
   • delete a file if older than a week

found myself looking for a 'refresh' button when looking at a collection.

perhaps at the right end of the 'location' bar - near the Home button.

The init of the logging is in iBridges.py. But there it cannot be accessed by the CLI --> move it to utils.utils.py

At the moment, the GUI is required to successfully create an iRODS session with the backend as the user must choose an iRODS environment file before connecting. This and other circumstances can cause errors that are not currently handled correctly or at all.

Add checks to give the appropriate (error) messages when the session is likely to fail or when it has failed unexpectedly. Also add convenience introspection methods to facilitate this and to clarify the state of the context, connector, and associated classes.

When the client is closed the iRODS session also needs to be closed with:

ic.session.cleanup()

This has to happen

• under menu/close
• when the Red Cross in the main frame is clicked

When the Red Cross of the main frame is clicked we receive now this error:

[2022-12-05 09:35:35,832] {connection.py:99} ERROR - Unable to send message. Connection to remote host may have closed. Releasing connection from pool.

# TODO The permissions dictionary is version dependent. From iRODS 4.3.0 the keyvalues are different

https://github.com/rijksuniversiteit-groningen/iBridges/blob/f4ca22b06dd8cc6af66c901c40bd6f51375c1104/gui/irodsBrowser.py#L17

ips currently shows python-irodsclient, the default populated value when a python-irodsclient-based application connects to the server.

Currently when the GUI is started the rods-connector and the GUI tries to load the path /zone/home. However, not all iRODS instances allow users to access this path. In that case the irods_home should be set in the environment file.

Wanted:
Check if irods_home is set and populate table and rods-connectors with content of that path
If irods_home is not set then initialise with /zone/home/user

while it should not be used in production... not listing it could be misleading.

seems like it should not be stripped from the listing.

The "force_unknown_free_space": "True" is ignored when it is not stated in the irods_environment file. However, it is an iBridges parameter and should be defined in .ibridges/config.json

The init of the logging is in iBridges.py. But there it cannot be accessed by the CLI --> move it to utils.utils.py

We would like to roll out the GUI to users with linux operating systems. However, we cannot assume that they know how to steer to the correct version in git and deploy the software. Hence we need to compile rpm or deb packages for them.

The two ienvs, the parameter in main menu and the Irodsconnector.ienv differ:

While the ienv in main menu is a joint of the irods_environment.json and the .ibridges/config.json

{'irods_host': 'its.data.uu.nl', 'irods_port': 1247, 'irods_home': '/nluu12p/home', 'irods_user_name': '[email protected]', 'irods_zone_name': 'nluu12p', 'irods_authentication_scheme': 'pam', 'irods_client_server_negotiation': 'request_server_negotiation', 'irods_client_server_policy': 'CS_NEG_REQUIRE', 'irods_encryption_algorithm': 'AES-256-CBC', 'irods_encryption_key_size': 32, 'irods_encryption_num_hash_rounds': 16, 'irods_encryption_salt_size': 8, 'davrods_server': 'https://its.data.uu.nl', 'irods_default_resource': 'irodsResc', 'amber_token': 'XXXXXXXXXXX', 'eln_token': 'uu.elabjournal.com;.......', 'force_unknown_free_space': False, 'last_ienv': 'irods_environment.json', 'ui_ienvFilePath': LocalPath("/Users/staig001/.irods/irods_environment.json"), 'ui_tabs': ['tabELNData', 'tabAmberWorkflow', 'tabUpDownload']}

the ienv of the iRods Connector only contains the parameters for iRODS:

{'irods_host': 'its.data.uu.nl', 'irods_port': 1247, 'irods_home': '/nluu12p/home', 'irods_user_name': '[email protected]', 'irods_zone_name': 'nluu12p', 'irods_authentication_scheme': 'pam', 'irods_client_server_negotiation': 'request_server_negotiation', 'irods_client_server_policy': 'CS_NEG_REQUIRE', 'irods_encryption_algorithm': 'AES-256-CBC', 'irods_encryption_key_size': 32, 'irods_encryption_num_hash_rounds': 16, 'irods_encryption_salt_size': 8, 'davrods_server': 'https://its.data.uu.nl', 'irods_default_resource': 'irodsResc'}

This makes it difficult to pass on parameters for iBridges Tabs since we only pass the iRODS connector object to the single tabs.

We need to get one ienv with all parameters to configure the tabs and iRODS.

utils/utils.py is currently uncompatible with Python 3.6.9.
Initial error:

  File "/home/maarten/Documents/Projects/ibridges/iBridges-Gui/utils/utils.py", line 4
    from __future__ import annotations
    ^
SyntaxError: future feature annotations is not defined

Manually merge updates from rijksuniversiteit-groningen/iBridges into this repository. The two have diverged far too much for any kind of simple merge.

the resource listing does not understand hierarchies and should parse the replicas differently to represent the number of replicas in the system correctly.

The functionality of that button is confused since we do not use the first column any longer to store paths from the search but instead store the status of the object or collection.