galaxyproject / ansible-galaxy-tools Goto Github PK

An Ansible role for automated installation of tools from a Tool Shed into Galaxy.

License: MIT License

Python 100.00%

ansible-galaxy-tools's Introduction

This Ansible role is for automated installation of tools from a Tool Shed into Galaxy.

When run, this role will create an execution environment, install ephemeris and invoke the shed-install command to install desired tools into Galaxy. The list of tools to install is provided in files/tool_list.yaml file.

Usage

To use the role, you need to create a playbook and include the role in it. A sample playbook is available here that will get you up and running in minutes.

Variables

Required variables

Only one of the two variables is requried (if both are set, the API key takes precedence and a bootstrap user is not created):

galaxy_tools_api_key: the Galaxy API key for an admin user on the target Galaxy instance (not required if the bootstrap user is being created)
galaxy_tools_admin_user_password: a password for the Galaxy bootstrap user (required only if galaxy_install_bootstrap_user variable is set)

Optional variables

See defaults/main.yml for the available variables and their defaults.

Control flow variables

The following variables can be set to either yes or no to indicate if the given part of the role should be executed:

galaxy_tools_install_tools: (default: yes) whether or not to run the tools installation script
galaxy_tools_create_bootstrap_user: (default: no) whether or not to create a bootstrap Galaxy admin user
galaxy_tools_delete_bootstrap_user: (default: no) whether or not to delete a bootstrap Galaxy admin user

ansible-galaxy-tools's People

Contributors

Stargazers

Watchers

ansible-galaxy-tools's Issues

Mark tool installation as failed when stderr contains traceback

Apparently this is not happening right now (and is the source for some confusion on http://dev.list.galaxyproject.org/ERROR-PIP-is-not-a-legl-parament-of-an-Ansible-Play-td4669045.html).
I guess we could check for the exit code / or cleanup messages that go to stderr?

tool_panel_section_label isn't working, nor is tool_panel_section_id

Modulize the installation script

The script is useful as a standalone script, so it's probably not necessary to fully convert it to a module, but a simple module that acts as a wrapper around the install script would make this a bit more Ansibley - that would take care of copying it over and running it for you, and cleaning up afterward. It also means you could have correct reporting of the changed status.

That said, it's actually not necessary to copy it over at all. It could run on the host from which you're running Ansible.

Do not report sucess when tool_list.yml does not conform

When tool_panel_section_label and tool_panel_section_id are missing from the tool_list.yml entries, ansible reports successful installation, when actually nothing has happened.
Since this might be something users would try if they want to have the tools outside of any section,
I guess we should either fail or add a default tool_panel_section_id ("no_tool_panel_section_id").
Any opinions on that issue, @bgruening @afgane @nuwang ?

manage_bootstrap_user.py doesn't delete user

Hi there,
I've been using the manage bootstrap user scripts for a while to install tools etc in my Galaxy setups. However, I've noticed that:

manage_bootstrap_user.py delete -u username doesn't work. It leaves the user in the database...

The delete function being called from __main__ :

def delete_bootstrap_user(ini_file, username):
    app = get_bootstrap_app(ini_file)
    user = delete_user(app, username)
    if user is not None:
        exit(0)
    else:
        log.error("Problem deleting user: {0}".format(username))
        exit(1)

There is another function called delete_user which looks as though it will delete the user from the postgres database but doesn't seem to work.

def delete_user(app, username):
    user = app.sa_session.query(app.model.User).filter(
        app.model.User.table.c.username == username).first()
    app.sa_session.delete(user)
    app.sa_session.flush()
    return user

Cheers,

Simon.

tool_list file loading

The tool list file located in the directory files is not loaded in ansible 2.2.

tasks/install_tool_list.yml

name: get tool list
include_vars: "{{ tool_list_file }}"

fatal: [bioinf-test]: FAILED! => {"ansible_facts": {}, "changed": false, "failed": true, "message": "Unable to find 'tool_list.yaml' in expected paths."}

If I place tool_list.yaml in a director vars it works.

Empty error aborts role but installation continues

When i try to install an additional tool the installation returns an error and aborts. The webinterface is listing my installation as running and after a few it has been finished successfully. I guess this is related to #15 and #galaxyproject/galaxy#1628

equests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine(\"''\",))"

playbook.yml

- hosts: galaxyservers
  become: true
  vars:
      galaxy_restart_handler_enabled: false
      galaxy_tools_base_dir: "{{ galaxy_root }}/var/tmp/"
      galaxy_server_dir: "{{ galaxy_root }}/server"
      galaxy_venv_dir: "{{ galaxy_root }}/venv"
      galaxy_config_file: "{{ galaxy_root }}/config/galaxy.yml"
      galaxy_tools_api_key: 11111111
      galaxy_tools_tool_list_files: [ "files/tools.yml" ]
      galaxy_tools_galaxy_instance_url: http://127.0.0.1:8080/
  roles:
    - galaxyproject.galaxy-tools

tools.yml

tools:
- name: ncbi_blast_plus
  owner: devteam
  tool_panel_section_id: BLAST
  tool_panel_section_label: BLAST

Error

TASK [galaxyproject.galaxy-tools : Install Tool Shed tools] 
failed: [172.18.1.32] (item={u'owner': u'devteam', u'tool_panel_section_id': u'BLAST', u'name': u'ncbi_blast_plus', u'tool_panel_section_label': u'BLAST'}) => {
    "changed": false,
    "cmd": [
        "/srv/galaxy/var/tmp//venv/bin/shed-install",
        "-y",
        "name: ncbi_blast_plus\nowner: devteam\ntool_panel_section_id: BLAST\ntool_panel_section_label: BLAST\n",
        "-a",
        "111111111111111",
        "-g",
        "http://127.0.0.1:8080/"
    ],
    "delta": "0:01:01.338944",
    "end": "2019-05-28 09:31:49.159597",
    "failed_when_result": true,
    "invocation": {
        "module_args": {
            "_raw_params": "/srv/galaxy/var/tmp//venv/bin/shed-install -y \"name: ncbi_blast_plus\nowner: devteam\ntool_panel_section_id: BLAST\ntool_panel_section_label: BLAST\n\" -a \"111111111111111\" -g \"http://127.0.0.1:8080/\"",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "item": {
        "name": "ncbi_blast_plus",
        "owner": "devteam",
        "tool_panel_section_id": "BLAST",
        "tool_panel_section_label": "BLAST"
    },
    "msg": "non-zero return code",
    "rc": 1,
    "start": "2019-05-28 09:30:47.820653",
    "stderr": "(1/1) Installing tool ncbi_blast_plus from devteam to section \"BLAST\" at revision e25d3acf6e68 (TRT: 0:00:00.893317)\r\nTraceback (most recent call last):\n  File \"/srv/galaxy/var/tmp//venv/bin/shed-install\", line 10, in <module>\n    sys.exit(script_main())\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 720, in script_main\n    itm.install_tools()\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 627, in install_tools\n    install_repository_revision(tool, self.tsc)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 502, in install_repository_revision\n    response = tsc.install_repository_revision(**tool)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 146, in install_repository_revision\n    return self._post(url=url, payload=payload)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 152, in _post\n    files_attached=files_attached)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxyclient.py\", line 137, in make_post_request\n    timeout=self.timeout)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 116, in post\n    return request('post', url, data=data, json=json, **kwargs)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 60, in request\n    return session.request(method=method, url=url, **kwargs)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 533, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 646, in send\n    r = adapter.send(request, **kwargs)\n  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/adapters.py\", line 498, in send\n    raise ConnectionError(err, request=request)\nrequests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine(\"''\",))",
    "stderr_lines": [
        "(1/1) Installing tool ncbi_blast_plus from devteam to section \"BLAST\" at revision e25d3acf6e68 (TRT: 0:00:00.893317)",
        "Traceback (most recent call last):",
        "  File \"/srv/galaxy/var/tmp//venv/bin/shed-install\", line 10, in <module>",
        "    sys.exit(script_main())",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 720, in script_main",
        "    itm.install_tools()",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 627, in install_tools",
        "    install_repository_revision(tool, self.tsc)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 502, in install_repository_revision",
        "    response = tsc.install_repository_revision(**tool)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 146, in install_repository_revision",
        "    return self._post(url=url, payload=payload)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 152, in _post",
        "    files_attached=files_attached)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxyclient.py\", line 137, in make_post_request",
        "    timeout=self.timeout)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 116, in post",
        "    return request('post', url, data=data, json=json, **kwargs)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 60, in request",
        "    return session.request(method=method, url=url, **kwargs)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 533, in request",
        "    resp = self.send(prep, **send_kwargs)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 646, in send",
        "    r = adapter.send(request, **kwargs)",
        "  File \"/srv/galaxy/var/tmp/venv/local/lib/python2.7/site-packages/requests/adapters.py\", line 498, in send",
        "    raise ConnectionError(err, request=request)",
        "requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine(\"''\",))"
    ],
    "stdout": "",
    "stdout_lines": []
}
...ignoring

ansible-galaxy-tools does not pass the travis test anymore

If you rerun the travis test of this ansible role, (which I cannot do myself), you will see that it does not pass anymore with this error:

TASK [ansible-galaxy-tools : Install Tool Shed tools] **************************
failed: [localhost] (item={u'owner': u'devteam', u'tool_shed_url': u'https://toolshed.g2.bx.psu.edu', u'tool_panel_section_id': u'cshl_library_information', u'name': u'fastqc', u'revisions': [u'8c650f7f76e9', u'd2cf2c0c8a11']}) => {"changed": false, "cmd": ["/tmp/venv/bin/shed-tools", "install", "-y", "name: fastqc\nowner: devteam\nrevisions:\n- 8c650f7f76e9\n- d2cf2c0c8a11\ntool_panel_section_id: cshl_library_information\ntool_shed_url: https://toolshed.g2.bx.psu.edu\n", "-a", "admin", "-g", "http://127.0.0.1:8080"], "delta": "0:00:00.209664", "end": "2020-08-24 13:02:18.132442", "failed_when_result": true, "item": {"name": "fastqc", "owner": "devteam", "revisions": ["8c650f7f76e9", "d2cf2c0c8a11"], "tool_panel_section_id": "cshl_library_information", "tool_shed_url": "https://toolshed.g2.bx.psu.edu"}, "msg": "non-zero return code", "rc": 1, "start": "2020-08-24 13:02:17.922778", "stderr": "Traceback (most recent call last):\n  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>\n    sys.exit(main())\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main\n    install_tool_manager.install_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories\n    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions\n    installed_repositories_list = tool_shed_client.get_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories\n    return self._get()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get\n    status_code=r.status_code)\nbioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}", "stderr_lines": ["Traceback (most recent call last):", "  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>", "    sys.exit(main())", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main", "    install_tool_manager.install_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories", "    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions", "    installed_repositories_list = tool_shed_client.get_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories", "    return self._get()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get", "    status_code=r.status_code)", "bioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item={u'owner': u'iuc', u'tool_panel_section_label': u'BED tools', u'name': u'bedtools', u'revisions': [u'2cd7e321d259']}) => {"changed": false, "cmd": ["/tmp/venv/bin/shed-tools", "install", "-y", "name: bedtools\nowner: iuc\nrevisions:\n- 2cd7e321d259\ntool_panel_section_label: BED tools\n", "-a", "admin", "-g", "http://127.0.0.1:8080"], "delta": "0:00:00.226166", "end": "2020-08-24 13:02:18.473471", "failed_when_result": true, "item": {"name": "bedtools", "owner": "iuc", "revisions": ["2cd7e321d259"], "tool_panel_section_label": "BED tools"}, "msg": "non-zero return code", "rc": 1, "start": "2020-08-24 13:02:18.247305", "stderr": "Traceback (most recent call last):\n  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>\n    sys.exit(main())\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main\n    install_tool_manager.install_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories\n    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions\n    installed_repositories_list = tool_shed_client.get_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories\n    return self._get()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get\n    status_code=r.status_code)\nbioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}", "stderr_lines": ["Traceback (most recent call last):", "  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>", "    sys.exit(main())", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main", "    install_tool_manager.install_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories", "    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions", "    installed_repositories_list = tool_shed_client.get_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories", "    return self._get()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get", "    status_code=r.status_code)", "bioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item={u'owner': u'devteam', u'tool_shed_url': u'https://toolshed.g2.bx.psu.edu', u'name': u'data_manager_fetch_genome_dbkeys_all_fasta', u'revisions': [u'b1bc53e9bbc5']}) => {"changed": false, "cmd": ["/tmp/venv/bin/shed-tools", "install", "-y", "name: data_manager_fetch_genome_dbkeys_all_fasta\nowner: devteam\nrevisions:\n- b1bc53e9bbc5\ntool_shed_url: https://toolshed.g2.bx.psu.edu\n", "-a", "admin", "-g", "http://127.0.0.1:8080"], "delta": "0:00:00.203915", "end": "2020-08-24 13:02:18.789717", "failed_when_result": true, "item": {"name": "data_manager_fetch_genome_dbkeys_all_fasta", "owner": "devteam", "revisions": ["b1bc53e9bbc5"], "tool_shed_url": "https://toolshed.g2.bx.psu.edu"}, "msg": "non-zero return code", "rc": 1, "start": "2020-08-24 13:02:18.585802", "stderr": "Traceback (most recent call last):\n  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>\n    sys.exit(main())\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main\n    install_tool_manager.install_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories\n    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list\n  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions\n    installed_repositories_list = tool_shed_client.get_repositories()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories\n    return self._get()\n  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get\n    status_code=r.status_code)\nbioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}", "stderr_lines": ["Traceback (most recent call last):", "  File \"/tmp/venv/bin/shed-tools\", line 8, in <module>", "    sys.exit(main())", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main", "    install_tool_manager.install_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories", "    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list", "  File \"/tmp/venv/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions", "    installed_repositories_list = tool_shed_client.get_repositories()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories", "    return self._get()", "  File \"/tmp/venv/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get", "    status_code=r.status_code)", "bioblend.ConnectionError: GET: error 403: '{\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}', 0 attempts left: {\"err_msg\": \"Provided API key is not valid.\", \"err_code\": 403001}"], "stdout": "", "stdout_lines": []}
	to retry, use: --limit @/home/travis/build/ARTbio/ansible-galaxy-tools/test_playbook.retry
PLAY RECAP *********************************************************************
localhost                  : ok=5    changed=1    unreachable=0    failed=1

Any thought welcome

Every second time role is run, it fails when trying to create bootstrap user.

When using this role, it works fine the first time. However, if I run the role again it fails when creating the bootstrap user (it says that it already exists.) If I re-run the role again, it works fine. Then on the 4th time it fails again etc..

Error message:

failed: [ip-address] => {"changed": true, "cmd": ["/mnt/galaxy/galaxy-app/.venv/bin/python", "manage_bootstrap_user.py", "-c", "/mnt/galaxy/galaxy-app/config/galaxy.ini", "create", "-e", "[email protected]", "-p", "TODO-GenerateRandomOrDeleteBootstrapUser"], "delta": "0:00:03.708041", "end": "2016-02-08 00:13:20.316735", "rc": 1, "start": "2016-02-08 00:13:16.608694", "warnings": []}
stderr: Traceback (most recent call last):
File "manage_bootstrap_user.py", line 207, in
create_bootstrap_user(args.config, args.username, args.email, args.password)
File "manage_bootstrap_user.py", line 159, in create_bootstrap_user
user = get_or_create_user( app, user_email, password, username )
File "manage_bootstrap_user.py", line 99, in get_or_create_user
return create_user( app, email, password, username )
File "manage_bootstrap_user.py", line 85, in create_user
app.model.security_agent.create_private_user_role( user )
File "/mnt/galaxy/galaxy-app/lib/galaxy/security/init.py", line 768, in create_private_user_role
self.sa_session.flush()
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 150, in do
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 2004, in flush
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 2122, in _flush
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/util/langhelpers.py", line 60, in exit
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 2086, in _flush
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 373, in execute
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 532, in execute
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 174, in save_obj
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 762, in _emit_insert_statements
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 914, in execute
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1146, in _execute_context
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1332, in _handle_dbapi_exception
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/util/compat.py", line 199, in raise_from_cause
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1139, in _execute_context
File "/mnt/galaxy/galaxy-app/eggs/SQLAlchemy-1.0.1-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py", line 442, in do_execute
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "ix_role_name"
DETAIL: Key (name)=([email protected]) already exists.
[SQL: 'INSERT INTO role (create_time, update_time, name, description, type, deleted) VALUES (%(create_time)s, %(update_time)s, %(name)s, %(description)s, %(type)s, %(deleted)s) RETURNING role.id'] [parameters: {'update_time': datetime.datetime(2016, 2, 8, 0, 13, 20, 173581), 'name': u'[email protected]', 'deleted': False, 'create_time': datetime.datetime(2016, 2, 8, 0, 13, 20, 173551), 'type': 'private', 'description': u'Private Role for [email protected]'}]

FATAL: all hosts have already failed -- aborting

Creating new tool section during installation

It should be possible to also create new sections and put tools into multiple sections. @afgane do you know if the API supports this?

This as a library?

Planemo now has a profiles feature - allowing persistent little environments for using commands like serve, shed_serve, test, and shed_test. This would be nice for tool development (consistent histories across serves) but is almost essential for workflow development. It needs to be possible to quickly get back to a Galaxy with a set of shed tools installed for instance.

So I'd like to implement --tool_list <path/to/tools.yml>, --data_list <path/to/data.yml>, and --workflow_list <path/to/workflows.yml> and I'd like to leverage what is here I think. There are three clear paths to do this without me just copying and pasting everything:

Planemo could add a dependency on ansible and run the role directly. The role would be included into planemo_ext I guess as a submodule.
The scripts could be placed onto PyPI as a new project and I could skip ansible but use the scripts as a library.
The scripts could be moved into planemo and then this role could have a dependency on planemo - again Planemo would use the scripts as a library and skip ansible.

Does anyone have any thoughts on using this as library or within planemo specifically?

(1) seems like it would be the most work for me right now and could potentially not gel well with what I am doing with planemo - on the other-hand it might setup a bunch of infrastructure within planemo that allows it to do some really cool things down the road. Seems the riskiest route - high risk, high reward potential.

(3) Would add a significant dependency for users of this project - but it would give these scripts a nice home that would probably lead to increased testing, linting, Python 3 support, etc.... I'm not sure what I would loose using the scripts directly instead of through ansible when using this as a library - any thoughts on this?

(2) seems like it would be the least disruptive - it would improve this project a little, the scripts a little, and would help planemo. We'd have to come up with a name for the scripts though and that is always stressful.

Ping @afgane @mvdbeek @bgruening

Tool labels

Would be nice to have the possibility to add labels to tools as described here: https://galaxyproject.org/admin/tool-panel/ ..

install_tool_shed_tools.py

where is this file now?

install_tool_shed_tools.py does not work with testtoolshed ?

Installation from toolshed works:

(.venv)root@vm0040:~/ansible-artimed/roles/galaxyprojectdotorg.galaxy-tools/files# python install_tool_shed_tools.py -g http://192.54.201.71/ -a d5c6ca76721721742742e8f8051f7e68 --name fetch_fasta_from_ncbi --owner drosofff --section RNAseq --toolshed https://toolshed.g2.bx.psu.edu/
(1/1) Installing tool fetch_fasta_from_ncbi from drosofff to section "RNAseq" at revision befdb392fece (TRT: 0:00:00.635433)
    Tool fetch_fasta_from_ncbi installed successfully (in 0:00:06.031311) at revision befdb392fece
Installed tools (1): [('fetch_fasta_from_ncbi', u'befdb392fece')]
Skipped tools (0): []
Errored tools (0): []
All tools listed in 'None' have been processed.
Total run time: 0:00:06.667989

but the example given in the help section of the tool does not:

(.venv)root@vm0040:~/ansible-artimed/roles/galaxyprojectdotorg.galaxy-tools/files# python install_tool_shed_tools.py -g http://192.54.201.71/ -a d5c6ca76721721742742e8f8051f7e68 --yaml_tool "{'owner': 'kellrott', 'tool_shed_url': 'https://testtoolshed.g2.bx.psu.edu', 'tool_panel_section_id': 'peak_calling', 'name': 'synapse_interface'}"
(1/1) Installing tool synapse_interface from kellrott to section "peak_calling" at revision 3dcbc98040a1 (TRT: 0:00:00.807190)
    * Error installing a tool (after synapse_interface)! Name: 0:00:00.523945,owner: kellrott, revision: 3dcbc98040a1, error: {"err_msg": "Uncaught exception in exposed API method:", "err_code": 0}
Installed tools (0): []
Skipped tools (0): []
Errored tools (1): [('synapse_interface', u'3dcbc98040a1')]
All tools listed in 'None' have been processed.
Total run time: 0:00:01.331707

Also tested without success using a yml file, or parameters supplied in the command line.

dynamic and static include in restart_galaxy.yml handler

Dear experts,
I've a problem with the handlers/restart_galaxy.yml task, triggered by the notify option, which is ignoring the when conditional.
I'm using supervisor, so this is leading to 2 errors (one for docker restart, one for run.sh restart), since all three tasks are triggered.

This is, likely, due to the new ansible default config:
http://docs.ansible.com/ansible/playbooks_roles.html#dynamic-versus-static-includes
In particular I'm referring to this:
"You cannot use notify to trigger a handler name which comes from a dynamic include."

For this reason all the when condition in handlers/restart_galaxy.yml are ignored.

I tested the "static=no" condition in restart_galaxy.yml, for each include and solve the problem.
Moving all when condition from restart_galaxy.yml to the corresponding tasks
restart_galaxy_docker.yml
restart_galaxy_run_sh.yml
restart_galaxy_supervisor.yml
works too.
Both of them are not so elegant solutions...
Finally, using "static=no" in handler/main.yml does not work, since ansible is not able to properly find the restart_galaxy.yml task.

I have it with both Ansible 2.2.2.0 and 2.2.1.0.

Any suggestion?
Thanks in advance,
Marco.

Create Galaxy bootstrap user Error

The script manage_bootstrap_user.py was trying to parse an ini file instead of a yml.

fatal: [galaxy-ubuntu]: FAILED! => {"changed": true, "cmd": ["/home/galaxy/venv/bin/python", "manage_bootstrap_user.py", "-c", "/home/galaxy/config/galaxy.yml", "create", "-e", "[email protected]", "-u", "cloud", "-p", "admin", "-a", "admin"], "delta": "0:00:02.683745", "end": "2018-12-06 20:54:10.927082", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-12-06 20:54:08.243337", "stderr": "Traceback (most recent call last):\n File \"manage_bootstrap_user.py\", line 265, in <module>\n create_bootstrap_user(args.config, args.username, args.email, args.password, args.preset_api_key)\n File \"manage_bootstrap_user.py\", line 207, in create_bootstrap_user\n app = get_bootstrap_app(ini_file)\n File \"manage_bootstrap_user.py\", line 197, in get_bootstrap_app\n config_parser.read(ini_file)\n File \"/usr/lib/python2.7/ConfigParser.py\", line 305, in read\n self._read(fp, filename)\n File \"/usr/lib/python2.7/ConfigParser.py\", line 512, in _read\n raise MissingSectionHeaderError(fpname, lineno, line)\nConfigParser.MissingSectionHeaderError: File contains no section headers.\nfile: /home/galaxy/config/galaxy.yml, line: 1\n'---\\n'", "stderr_lines": ["Traceback (most recent call last):", " File \"manage_bootstrap_user.py\", line 265, in <module>", " create_bootstrap_user(args.config, args.username, args.email, args.password, args.preset_api_key)", " File \"manage_bootstrap_user.py\", line 207, in create_bootstrap_user", " app = get_bootstrap_app(ini_file)", " File \"manage_bootstrap_user.py\", line 197, in get_bootstrap_app", " config_parser.read(ini_file)", " File \"/usr/lib/python2.7/ConfigParser.py\", line 305, in read", " self._read(fp, filename)", " File \"/usr/lib/python2.7/ConfigParser.py\", line 512, in _read", " raise MissingSectionHeaderError(fpname, lineno, line)", "ConfigParser.MissingSectionHeaderError: File contains no section headers.", "file: /home/galaxy/config/galaxy.yml, line: 1", "'---\\n'"], "stdout": "", "stdout_lines": []}

`install_tool_dependencies: False` is not working

  install_tool_dependencies: False
  install_repository_dependencies: False

Is not working reliable. I get repositories added to the database but the installation is not triggered. This option get more and more important as we try to get first flavours up and running with pure conda/docker dependencies.

Installing but hiding tools

I see our self in a situation where we need to install tools to keep workflows working but would like to hide tools from the user. This use-case needs to be supported in the not so far future :)

manage_bootstrap_user.py command line parsing

The command line parser in manage_bootstrap_user.py uses

required="false" and required="true"

which should be

required=False and required=True

Thanks

SSL: CERTIFICATE_VERIFY_FAILED

When using a Galaxy instance that is not secure, is it possible to ignore SSL certificate verification?

e.g. with bioblend the following parameters can be set

gi = bioblend.galaxy.GalaxyInstance(url, api_key)
gi.verify = False

Otherwise I get the following error when trying to install

failed: [galaxy-dev] (item={'name': 'column_maker', 'owner': 'devteam', 'tool_panel_section_id': 'textutil'}) => {"changed": false, "cmd": ["/tmp/venv/bin/shed-tools", "install", "-y", "name: column_maker\nowner: devteam\ntool_panel_section_id: textutil\n", "-a", "7f72a11757132ffbf31de35a59ead848", "-g", "https://metabolomics-dev.galaxy.bham.ac.uk/"], "delta": "0:00:00.274420", "end": "2018-05-02 11:40:32.412191", "failed_when_result": true, "item": {"name": "column_maker", "owner": "devteam", "tool_panel_section_id": "textutil"}, "msg": "non-zero return code", "rc": 1, "start": "2018-05-02 11:40:32.137771", "stderr": "Traceback (most recent call last):\n  File \"/tmp/venv/bin/shed-tools\", line 11, in <module>\n    sys.exit(main())\n  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main\n    install_tool_manager.install_repositories()\n  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories\n    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list\n  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions\n    installed_repositories_list = tool_shed_client.get_repositories()\n  File \"/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories\n    return self._get()\n  File \"/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get\n    status_code=r.status_code)\nbioblend.ConnectionError: HTTPSConnectionPool(host='metabolomics-dev.galaxy.bham.ac.uk', port=443): Max retries exceeded with url: /api/tool_shed_repositories?key=7f72a11757132ffbf31de35a59ead848 (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),)), 0 attempts left: ", "stderr_lines": ["Traceback (most recent call last):", "  File \"/tmp/venv/bin/shed-tools\", line 11, in <module>", "    sys.exit(main())", "  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 744, in main", "    install_tool_manager.install_repositories()", "  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 558, in install_repositories", "    installed_repositories_list = installed_repository_revisions(self.gi)  # installed tools list", "  File \"/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_tools.py\", line 157, in installed_repository_revisions", "    installed_repositories_list = tool_shed_client.get_repositories()", "  File \"/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 36, in get_repositories", "    return self._get()", "  File \"/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 136, in _get", "    status_code=r.status_code)", "bioblend.ConnectionError: HTTPSConnectionPool(host='metabolomics-dev.galaxy.bham.ac.uk', port=443): Max retries exceeded with url: /api/tool_shed_repositories?key=7f72a11757132ffbf31de35a59ead848 (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),)), 0 attempts left: "], "stdout": "", "stdout_lines": []

Restart handler mechanism appears to have problems

Problem:

The restart handler mechanism doesn't appear to work consistently and/or work well with other roles.

Background:

See discussion here: #47

Issues:

There are several attendant issues that also need to be solved:

The build should fail when test tool installation during testing fails. However, it currently does not.
The restart handler name used to be "restart galaxy", which is the same handler name used in the galaxy playbook: https://github.com/galaxyproject/ansible-galaxy/blob/6544db9b17cc4db6cfbaf6940545fb57edc37cdf/handlers/main.yml#L5.
but this can cause the incorrect handler to be called when both galaxy and galaxy-tools roles are used together. We could use the same handler name if we use a "generic topic" as introduced in ansible 2.2 onwards or make the handler name unique, which is what was done in PR: #47. However, the latter seems to have caused a build break for unknown reasons.
Supervisor seems to be having trouble restarting Galaxy as shown in: #47 (comment)
Since a restart command should succeed, whether Galaxy is running or not, there may also be a supervisor config issue in the extras playbook.

Installed ansible version (for tests) is not compatible with the ansible synthax of the repo

The
pip install ansible command install ansible 2.7.4, which is not compatible with the syntax of many code lines of the repo.
As a results, the travis tests are failing.
I see 2 options:

pin ansible to pip install ansible==2.4
update the code syntax to be compatible with ansible 2.7.4

Incompatability with Galaxy 19.05

Hello,

It would seem that this role has been broken by Galaxy 19.05. In the manage_bootstrap_user.py script, the line from galaxy.web import security produces the following error:

Traceback (most recent call last):
  File "manage_bootstrap_user.py", line 21, in <module>
    from galaxy.web import security
ImportError: cannot import name security

And sure enough when you check the lib of galaxy, the security module has been removed (PR: galaxyproject/galaxy#7560). I did not check for other incompatibilities in the script.

Is there any work around or plan to update this role for compliance with 19.05? It otherwise worked well in 19.01.

Thank you!

putative new task/role

Just working on a role/task to add files from file system to a library (using https://github.com/galaxyproject/galaxy/blob/dev/scripts/api/library_upload_dir.py). Was wondering if it is an idea to integrate this here or elsewhere in https://github.com/galaxyproject/

SSLCertVerificationError

when I install tool，there is an error:Invalid server response. HTTPSConnectionPool(host='toolshed.g2.bx.psu.edu', port=443): Max retries exceeded with url: /api/categories (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)'))).

Re-enable multiple tools files once Ansible 2.0 comes out

PR #11 disabled use of multiple tools files (from PR #7) in exchange for interactive tool installation progress reporting. Ansible 2.0 will have support for with_ directive in loops making it possible to add the multi-file support back in. Ansible 2.0 is expected any day now.

Privilege separation support

I installed galaxy with the ansible-galaxy role from the admin training and used it with galaxy_separate_privileges: true
My priviliged user is called gxpriv and the normal user is called galaxy.
Folder <galaxy_install_folder>/server/database has 0755 and gxpriv:galaxy
Folder <galaxy_install_folder>/server/tools has 0755 and gxpriv:galaxy

When i try to install tools with this role, i get a permission denied error on both folders, since the user galaxy is not allowed to write to the defined folders. If i set galaxy_user_name: gxpriv i get the following error during installation:

failed: [192.168.1.1] (item={u'owner': u'devteam', u'tool_panel_section_id': u'BLAST', u'name': u'ncbi_blast_plus'}) => {
    "changed": false,
    "cmd": [
        "/media/tmp//venv/bin/shed-install",
        "-y",
        "name: ncbi_blast_plus\nowner: devteam\ntool_panel_section_id: BLAST\n",
        "-a",
        "*********************",
        "-g",
        "http://127.0.0.1:8080/"
    ],
    "delta": "0:01:01.309185",
    "end": "2019-05-27 06:50:58.395925",
    "failed_when_result": true,
    "invocation": {
        "module_args": {
            "_raw_params": "/media/tmp//venv/bin/shed-install -y \"name: ncbi_blast_plus\nowner: devteam\ntool_panel_section_id: BLAST\n\" -a \"****************\" -g \"http://127.0.0.1:8080/\"",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "item": {
        "name": "ncbi_blast_plus",
        "owner": "devteam",
        "tool_panel_section_id": "BLAST"
    },
    "msg": "non-zero return code",
    "rc": 1,
    "start": "2019-05-27 06:49:57.086740",
    "stderr": "(1/1) Installing tool ncbi_blast_plus from devteam to section \"BLAST\" at revision e25d3acf6e68 (TRT: 0:00:00.668739)\r\nTraceback (most recent call last):\n  File \"/media/tmp//venv/bin/shed-install\", line 10, in <module>\n    sys.exit(script_main())\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 720, in script_main\n    itm.install_tools()\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 627, in install_tools\n    install_repository_revision(tool, self.tsc)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 502, in install_repository_revision\n    response = tsc.install_repository_revision(**tool)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 146, in install_repository_revision\n    return self._post(url=url, payload=payload)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 152, in _post\n    files_attached=files_attached)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxyclient.py\", line 137, in make_post_request\n    timeout=self.timeout)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 116, in post\n    return request('post', url, data=data, json=json, **kwargs)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 60, in request\n    return session.request(method=method, url=url, **kwargs)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 533, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 646, in send\n    r = adapter.send(request, **kwargs)\n  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/adapters.py\", line 498, in send\n    raise ConnectionError(err, request=request)\nrequests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine(\"''\",))",
    "stderr_lines": [
        "(1/1) Installing tool ncbi_blast_plus from devteam to section \"BLAST\" at revision e25d3acf6e68 (TRT: 0:00:00.668739)",
        "Traceback (most recent call last):",
        "  File \"/media/tmp//venv/bin/shed-install\", line 10, in <module>",
        "    sys.exit(script_main())",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 720, in script_main",
        "    itm.install_tools()",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 627, in install_tools",
        "    install_repository_revision(tool, self.tsc)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/ephemeris/shed_install.py\", line 502, in install_repository_revision",
        "    response = tsc.install_repository_revision(**tool)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py\", line 146, in install_repository_revision",
        "    return self._post(url=url, payload=payload)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxy/client.py\", line 152, in _post",
        "    files_attached=files_attached)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/bioblend/galaxyclient.py\", line 137, in make_post_request",
        "    timeout=self.timeout)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 116, in post",
        "    return request('post', url, data=data, json=json, **kwargs)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/api.py\", line 60, in request",
        "    return session.request(method=method, url=url, **kwargs)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 533, in request",
        "    resp = self.send(prep, **send_kwargs)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/sessions.py\", line 646, in send",
        "    r = adapter.send(request, **kwargs)",
        "  File \"/media/tmp/venv/local/lib/python2.7/site-packages/requests/adapters.py\", line 498, in send",
        "    raise ConnectionError(err, request=request)",
        "requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine(\"''\",))"
    ],
    "stdout": "",
    "stdout_lines": []
}

Role finishes before all tools are installed

This is not a big deal for running instances (since the install process will just continue), but when using this in a playbook that trigger a galaxy restart or shutdown you get the remaining tools stuck in "new" or "installing dependencies" state.
This behaviour depends on the list of tools that you install, and happens mostly if at the end of the list you have something with a huge dependency chain (R packages are good candidates).
I'll have a look if we can use bioblend to check on the installation status.

Installation behaviour for updated versions

The problem appeared for hisat version 16 which was an update of version 15 (galaxyproject/tools-iuc#1514). I had 15 installed and tried to install also 16. The log output was:

Skipping installation of revision 21e8ea8e1adb of repository 'hisat2' because it was installed with th
e (possibly updated) revision d5fe9aead222 and its current installation status is 'Installed'.

The ansible script returned no error. I would expect some error or that the tool gets installed..?

urlopen error [Errno 104] Connection reset by peer>", "err_code": 500001

My galaxy flavor installs 26 tools. One of the installs fails:

(11/26) Installing tool rgrnastar from iuc to section "RNAseq" at revision ace9f5a2b40f (TRT: 0:42:06.345389)
        * Error installing a tool (after 0:07:06.264252 seconds)! Name: rgrnastar,owner: iuc, revision: ace9f5a2b40f, error: {"err_msg": "Error attempting to retrieve installation information from tool shed https://toolshed.g2.bx.psu.edu/ for revision ace9f5a2b40f of repository rgrnastar owned by iuc: <urlopen error [Errno 104] Connection reset by peer>", "err_code": 500001}

What is this error?

SQL IntegrityError

Installation using the bootstrap user fails when the role is executed multiple times (when boostrap user creation and deletion is yes).

During the task Create Galaxy bootstrap user with the following exception is raised:

sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "ix_role_name"\n [SQL: 'INSERT INTO role (create_time, update_time, name, description, type, deleted) VALUES (%(create_time)s, %(update_time)s, %(name)s, %(description)s, %(type)s, %(deleted)s) RETURNING role.id'

The problem is that manage_bootstrap_user.py (but I guess the problem is deeper) does not delete the role which is created with the user.