Git Product home page Git Product logo

ceph-iscsi-config's Introduction

ceph-iscsi-config

This project provides the common logic for creating and managing LIO gateways for ceph, together with a startup daemon
called rbd-target-gw which is responsible for restoring the state of LIO following a gateway reboot/outage.

Usage

This package should be installed on each node that is intended to be an iSCSI gateway. The config modules are used by

Installation

Via RPM

Simply install the provided rpm with
rpm -ivh ceph-iscsi-config-<ver>.el7.noarch.rpm

Manually

The following packages are required by ceph-iscsi-config and must be installed before starting the rbd-target-gw service:

python-rados python-rbd python-netifaces python-rtslib python-crypto

To install the python package that provides the application logic, run the provided setup.py script
i.e. > python setup.py install

For the management daemon (rbd-target-gw), simply copy the following files into their equivalent places on each gateway

  • <archive_root>/usr/lib/systemd/system/rbd-target-gw.service --> /lib/systemd/system
  • <archive_root>/usr/bin/rbd-target-gw --> /usr/bin

Once the daemon is in place, reload the configuration with

systemctl daemon-reload
systemctl enable rbd-target-gw 
systemctl start rbd-target-gw

Features

The functionality provided by each module in the python package is summarised below;

Module Description
client logic handling the create/update and remove of a NodeACL from a gateway
config common code handling the creation and update mechanisms for the rados configuration object
gateway definition of the iSCSI gateway (target plus target portal groups)
lun rbd image management (create/resize), combined with mapping to the OS and LIO instance
utils common code called by multiple modules

The rbd-target-gw daemon performs the following tasks;

  1. At start up remove any osd blacklist entry that may apply to the running host
  2. Read the configuration object from Rados
  3. Process the configuration
    3.1 map rbd's to the host
    3.2 add rbd's to LIO
    3.3 Create the iscsi target, TPG's and port IP's
    3.4 Define clients (NodeACL's)
    3.5 add the required rbd images to clients

ceph-iscsi-config's People

Contributors

hxrforcode avatar ktdreyer avatar lenzgr avatar leseb avatar mikechristie avatar pcuzner avatar rjerk avatar smithfarm avatar vshankar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ceph-iscsi-config's Issues

On the issue of log processing

I found that we used loging modules to process logs, and we did some configuration for logs, such as configuring /etc/logrotate.d in spec files.

A small bug when define the disks in the config to LIO

In the define_luns of lun.py, the code "pool_disks = [disk_key for disk_key in srtd_disks if disk_key.startswith(pool)]" should be instead of "pool_disks = [disk_key for disk_key in srtd_disks if disk_key.startswith(pool+'.')]", otherwise it may be failed when starting.

IPv6 support for ceph iSCSI gateway

I tried to configure a iSCSI gateway on a IPv6 only host. It turns out that IPv6 is not supported at all :/. Why is this? LIO (and targetcli where gwcli is based on) both support IPv6

rbd-target-api fails to lock non-existent 'gateway.conf' file

With a clean configuration, the daemon fails with the following error:

Traceback (most recent call last):
File "/bin/rbd-target-api", line 783, in
config = Config(logger)
File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/common.py", line 87, in init
self.config = self.get_config()
File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/common.py", line 146, in _get_ceph_config
self._seed_rbd_config()
File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/common.py", line 202, in _seed_rbd_config
self.lock()
File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/common.py", line 167, in lock
ioctx.lock_exclusive(self.config_name, 'lock', 'config')
File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106)
File "rados.pyx", line 3001, in rados.Ioctx.lock_exclusive (rados.c:31907)
rados.Error: Ioctx.rados_lock_exclusive(rbd): failed to set lock lock on gateway.conf: error code 95

rbd-target-gw socket.error: [errno 97] address family not supported by protocol

when prometheus exporter is enabled, which it is by default iscsi-gw fails to start on systems with ipv6 disabled
'''
app.run(host='::',
port=settings.config.prometheus_port,
debug=False,
threaded=True)
'''

This should be a relatively easy fix, by making prometheus host configurable. If it's required, i'd be more than willing to implement this solution

Problem starting

I am trying to start the rbd-target-gw and I am gettting this error:

Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 33, in exception_handler

File "/usr/local/lib/python2.7/dist-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 75, in clearconfig

File "build/bdist.linux-x86_64/egg/ceph_iscsi_config/lio.py", line 15, in init
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/root.py", line 85, in init
self._set_dbroot_if_needed()
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/root.py", line 167, in _set_dbroot_if_needed
fwrite(dbroot_path, self._preferred_dbroot+"\n")
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/utils.py", line 79, in fwrite
file_fd.write(str(string))
IOError: [Errno 22] Invalid argument

Original exception was:
Traceback (most recent call last):
File "/usr/local/bin/rbd-target-gw", line 4, in
import('pkg_resources').run_script('ceph-iscsi-config==2.3', 'rbd-target-gw')
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 719, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 1511, in run_script
exec(script_code, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 462, in

File "/usr/local/lib/python2.7/dist-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 210, in halt

File "/usr/local/lib/python2.7/dist-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 75, in clearconfig

File "build/bdist.linux-x86_64/egg/ceph_iscsi_config/lio.py", line 15, in init
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/root.py", line 85, in init
self._set_dbroot_if_needed()
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/root.py", line 167, in _set_dbroot_if_needed
fwrite(dbroot_path, self._preferred_dbroot+"\n")
File "/usr/local/lib/python2.7/dist-packages/rtslib_fb/utils.py", line 79, in fwrite
file_fd.write(str(string))
IOError: [Errno 22] Invalid argument

Support for multiple iSCSI Targets

Currently only one iSCSI Target is supported. Multiple iSCSI Targets would allow for strict seperation of iSCSI Targets in different network segments (VLAN, interfaces). Each Target would have it's own Target Portal Groups and Portals. A "per tenant" Target configuration would be possible, exposing only those targets in a specific network to iSCSI clients.

I start the rbd-target-gw.service failed

12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/ceph_iscsi_config-2.3-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 8, in
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "build/bdist.linux-x86_64/egg/netifaces.py", line 7, in
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "build/bdist.linux-x86_64/egg/netifaces.py", line 4, in bootstrap
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 945, in resource_filename
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: self, resource_name
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1633, in get_resource_filename
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: self._extract_resource(manager, self._eager_to_zip(name))
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1661, in _extract_resource
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: self.egg_name, self._parts(zip_path)
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1025, in get_cache_path
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: self.extraction_error()
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 991, in extraction_error
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: raise err
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: pkg_resources.ExtractionError: Can't extract file(s) to egg cache
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: The following error occurred while trying to extract file(s) to the Python egg
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: cache:
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: [Errno 30] Read-only file system: '/root/.python-eggs'
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: The Python egg cache directory is currently set to:
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: /root/.python-eggs
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: Perhaps your account does not have write access to this directory? You can
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: change the cache directory by setting the PYTHON_EGG_CACHE environment
12月 26 17:29:37 ceph-iscsi-gw-1 rbd-target-gw[16174]: variable to point to an accessible directory.
12月 26 17:29:37 ceph-iscsi-gw-1 systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE
12月 26 17:29:37 ceph-iscsi-gw-1 systemd[1]: Unit rbd-target-gw.service entered failed state.
12月 26 17:29:37 ceph-iscsi-gw-1 systemd[1]: rbd-target-gw.service failed.
12月 26 17:29:38 ceph-iscsi-gw-1 systemd[1]: rbd-target-gw.service holdoff time over, scheduling restart.
12月 26 17:29:38 ceph-iscsi-gw-1 systemd[1]: start request repeated too quickly for rbd-target-gw.service
12月 26 17:29:38 ceph-iscsi-gw-1 systemd[1]: Failed to start Setup system to export rbd images through LIO.
-- Subject: Unit rbd-target-gw.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

-- Unit rbd-target-gw.service has failed.

-- The result is failed.
12月 26 17:29:38 ceph-iscsi-gw-1 systemd[1]: Unit rbd-target-gw.service entered failed state.
12月 26 17:29:38 ceph-iscsi-gw-1 systemd[1]: rbd-target-gw.service failed.
12月 26 17:37:37 ceph-iscsi-gw-1 polkitd[551]: Registered Authentication Agent for unix-process:16189:2198857 (system bus name :1.71 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freede
12月 26 17:37:37 ceph-iscsi-gw-1 systemd[1]: start request repeated too quickly for rbd-target-gw.service
12月 26 17:37:37 ceph-iscsi-gw-1 systemd[1]: Failed to start Setup system to export rbd images through LIO.
-- Subject: Unit rbd-target-gw.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

-- Unit rbd-target-gw.service has failed.

Rbd resized - iscsi gw not updated

Hi,
Maybe i am doing something wrong, but after an image is resized, i have to restart rbd-target-gw daemon.
Otherwise, the volume size does get updated on iscsi initiator.
Is this behaviour normal ? I am thinking about all the connected clients, that will encounter connectivity issues during gateway service restart...
Using:
ceph-iscsi-config
Version : 2.0, Release: 4.el7cp
tcmu-runner
Version : 1.2.1, Release: 0.2.20170104.git3d33566.el7cp
On RHEL7
I am thinking to implement this software in production since it is way better than classic rbd mapping and targetd exported, but i really need to know if this gw restart is a must after every rbd resize.
Thank you very much !

rbd-target-gw can't stop

hello. when i try to restart rbd-target-gw with: sudo systemctl restart rbd-target-gw,i found it can't stop the old service , it just start a new process. And the service process can't stop by kill -9 either. How can i stop the service? thanks.

StripExtension in setup.py beaks bdist_rpm

Just as stated in the comment of https://stackoverflow.com/questions/4359231/rename-script-file-in-distutils/11400431#11400431, removing the .py extension breaks bdist_rpm.

Running "python setup.py bdist_rpm" generates following error:

...
Processing files: ceph_iscsi_config-2.6-1.noarch
error: File not found: /root/ceph/iscsi/ceph-iscsi-config/build/bdist.linux-x86_64/rpm/BUILDROOT/ceph_iscsi_config-2.6-1.x86_64/usr/bin/rbd-target-gw.py
RPM build errors:
File not found: /root/ceph/iscsi/ceph-iscsi-config/build/bdist.linux-x86_64/rpm/BUILDROOT/ceph_iscsi_config-2.6-1.x86_64/usr/bin/rbd-target-gw.py
...

drop dependency on python-crypto

The python-crypto library is problematic for a couple reasons, pulling in extra crypo libs
(libtommath & libtomcrypt) and lacking FIPS compliance.

We use it in ceph_iscsi_config/client.py:

from Crypto.PublicKey import RSA
from Crypto.Cipher import PKCS1_OAEP

This is the only location, right? How can we replace this with the cryptography package instead? (python-cryptography is available in RHEL 7)

pool rbd must exsist

pool rbd must exist, or rbd-target-gw will not start. Is there a way to select the pool you would like to use?

standardize versioning

95b2b63 bumps the version to 2.0, but setup.py still has an old version number.

Would you please bump the version in setup.py, and push a tag to GitHub?

(It might be best to just go to 2.1 at this point.)

AttributeError: 'GWClient' object has no attribute 'controls'

add host to hostgroup

curl --insecure --user admin:admin -d action=add -d members=iqn.2018-08.com.umcloud.iscsi-gw:host1  -X PUT http://192.168.2.80:5000/api/hostgroup/test

got error: 'GWClient' object has no attribute 'controls'

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/bin/rbd-target-api", line 97, in decorated
    return f(*args, **kwargs)
  File "/usr/bin/rbd-target-api", line 1656, in _hostgroup
    grp.apply()
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 217, in apply
    self.enforce_policy()
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 318, in enforce_policy
    self.update_client(client_iqn, image_list)
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 345, in update_client
    client.define_client()                          # sets up tpg lun list
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/client.py", line 175, in define_client
    try:
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/client.py", line 150, in update_acl_controls
    self.acl.set_attribute('dataout_timeout', '{}'.format(
AttributeError: 'GWClient' object has no attribute 'controls'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/bin/rbd-target-api", line 97, in decorated
    return f(*args, **kwargs)
  File "/usr/bin/rbd-target-api", line 1588, in hostgroup
    http_method='put', api_vars=api_vars)
  File "/usr/bin/rbd-target-api", line 1809, in call_api
    fail_msg += api.response.json()['message']
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

after add self.controls = {} to GWClient, still got an error:

File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/client.py", line 151, in update_acl_controls
    self.controls['dataout_timeout']))
KeyError: 'dataout_timeout'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/bin/rbd-target-api", line 97, in decorated
    return f(*args, **kwargs)
  File "/usr/bin/rbd-target-api", line 1656, in _hostgroup
    grp.apply()
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 217, in apply
    self.enforce_policy()
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 318, in enforce_policy
    self.update_client(client_iqn, image_list)
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/group.py", line 345, in update_client
    client.define_client()                          # sets up tpg lun list
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/client.py", line 176, in define_client
    self.update_acl_controls()
  File "/usr/lib/python2.7/site-packages/ceph_iscsi_config/client.py", line 151, in update_acl_controls
    self.controls['dataout_timeout']))
KeyError: 'dataout_timeout'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/bin/rbd-target-api", line 97, in decorated
    return f(*args, **kwargs)
  File "/usr/bin/rbd-target-api", line 1588, in hostgroup
    http_method='put', api_vars=api_vars)
  File "/usr/bin/rbd-target-api", line 1809, in call_api
    fail_msg += api.response.json()['message']
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

qfull timeout patch issue

@mikechristie
I want to apply the kernel patches required by below commit. But here's some questions about it. My kernel is 4.14.2

commit 4e23d10
Author: Mike Christie [email protected]
Date: Sun Aug 13 12:46:21 2017 -0500

lun: set qfull time out

Do not let commands sit in the ring buf wait queue so long
that they end up executing after the initiator has failed
the command and retried on another path.

This patch requires these kernel patches

I saw patches which are for kernel 4.16 versions.
https://www.spinics.net/lists/target-devel/msg16162.html
And I also saw some patches which are for kernel 4.15-rc1
http://www.spinics.net/lists/target-devel/msg16275.html

I have no idea how to merge them based on kernel v4.14 tag. Any suggestion?

issue when trying to start rados-target-gw

Hi, When trying to start rados-target-gw I get the below error, googling around has not been fruitful.

I'm running this on CentOS 7.5 with the following kernel:

Linux 4.19.2-1.el7.elrepo.x86_64

Journalctl output:

Nov 18 17:30:29 localhost.localdomain systemd[1]: Starting Setup system to export rbd images through LIO...
-- Subject: Unit rbd-target-gw.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit rbd-target-gw.service has begun starting up.
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: Error in sys.excepthook:
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: Traceback (most recent call last):
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/ceph_iscsi_config-2.6-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 37, in exception_handler
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/ceph_iscsi_config-2.6-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 83, in clearconfig
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: NameError: global name 'config' is not defined
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: Original exception was:
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: Traceback (most recent call last):
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/bin/rbd-target-gw", line 5, in <module>
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: pkg_resources.run_script('ceph-iscsi-config==2.6', 'rbd-target-gw')
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 540, in run_script
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: self.require(requires)[0].run_script(script_name, ns)
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1462, in run_script
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: exec_(script_code, namespace, namespace)
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 41, in exec_
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: exec("""exec code in globs, locs""")
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "<string>", line 1, in <module>
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/ceph_iscsi_config-2.6-py2.7.egg/EGG-INFO/scripts/rbd-target-gw", line 431, in <module>
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "build/bdist.linux-x86_64/egg/ceph_iscsi_config/common.py", line 76, in __init__
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "build/bdist.linux-x86_64/egg/ceph_iscsi_config/common.py", line 37, in __init__
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/rados.py", line 253, in __init__
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: self.conf_read_file(conffile)
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: File "/usr/lib/python2.7/site-packages/rados.py", line 302, in conf_read_file
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: raise make_ex(ret, "error calling conf_read_file")
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: rados.Error: error calling conf_read_file: errno EINVAL
Nov 18 17:30:29 localhost.localdomain rbd-target-gw[2709]: Exception AttributeError: "'CephCluster' object has no attribute 'cluster'" in <bound method CephCluster.__del__ of <ceph_iscsi_config.common.CephCluster object at 0x
Nov 18 17:30:29 localhost.localdomain systemd[1]: rbd-target-gw.service: main process exited, code=exited, status=1/FAILURE
Nov 18 17:30:29 localhost.localdomain systemd[1]: Unit rbd-target-gw.service entered failed state.
Nov 18 17:30:29 localhost.localdomain systemd[1]: rbd-target-gw.service failed.
Nov 18 17:30:30 localhost.localdomain systemd[1]: rbd-target-gw.service holdoff time over, scheduling restart.
Nov 18 17:30:30 localhost.localdomain systemd[1]: start request repeated too quickly for rbd-target-gw.service
Nov 18 17:30:30 localhost.localdomain systemd[1]: Failed to start Setup system to export rbd images through LIO.
-- Subject: Unit rbd-target-gw.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit rbd-target-gw.service has failed.
--
-- The result is failed.
Nov 18 17:30:30 localhost.localdomain systemd[1]: Unit rbd-target-gw.service entered failed state.
Nov 18 17:30:30 localhost.localdomain systemd[1]: rbd-target-gw.service failed.

" KeyError: 'targets' " when adding second gateway on ceph-iscsi on ubuntu - BUG

Hi,
I have installed ceph-iscsi on ubuntu 20 manully,
But when I want to add second gateway it will show me error:


OS: Ubuntu 20 LTS
ceph version : octopus
I install cluster with ceph-ansible but I install ceph-iscsi manually via following link:
https://docs.ceph.com/en/latest/rbd/iscsi-target-cli/
(instead of 'yum install ceph-iscsi' I commit 'apt install ceph-iscsi')

root@dev13:~# gwcli -v
gwcli - 2.7



/iscsi-target...-igw/gateways> create ceph-gateway1 192.168.200.33
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
KeyError: 'targets'

and it will jump out of gwcli!
this is my gwcli log:

root@dev13:~# cat gwcli.log 
2020-11-25 14:24:56,440 DEBUG    [ceph.py:32:__init__()] Adding ceph cluster 'ceph' to the UI
2020-11-25 14:24:57,049 DEBUG    [ceph.py:241:populate()] Fetching ceph osd information
2020-11-25 14:24:57,086 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:24:57,197 DEBUG    [storage.py:105:refresh()] Refreshing disk information from the config object
2020-11-25 14:24:57,197 DEBUG    [storage.py:108:refresh()] - Scanning will use 8 scan threads
2020-11-25 14:24:57,254 DEBUG    [storage.py:135:refresh()] - rbd image scan complete: 0s
2020-11-25 14:24:57,254 DEBUG    [gateway.py:378:refresh()] Refreshing gateway & client information
2020-11-25 14:24:57,254 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:24:57,294 DEBUG    [ceph.py:260:refresh()] Gathering pool stats for cluster 'ceph'
2020-11-25 14:25:02,319 DEBUG    [ceph.py:32:__init__()] Adding ceph cluster 'ceph' to the UI
2020-11-25 14:25:03,076 DEBUG    [ceph.py:241:populate()] Fetching ceph osd information
2020-11-25 14:25:03,168 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:25:03,219 DEBUG    [storage.py:105:refresh()] Refreshing disk information from the config object
2020-11-25 14:25:03,219 DEBUG    [storage.py:108:refresh()] - Scanning will use 8 scan threads
2020-11-25 14:25:03,273 DEBUG    [storage.py:135:refresh()] - rbd image scan complete: 0s
2020-11-25 14:25:03,274 DEBUG    [gateway.py:378:refresh()] Refreshing gateway & client information
2020-11-25 14:25:03,274 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:25:03,314 DEBUG    [ceph.py:260:refresh()] Gathering pool stats for cluster 'ceph'
2020-11-25 14:25:32,950 DEBUG    [ceph.py:32:__init__()] Adding ceph cluster 'ceph' to the UI
2020-11-25 14:25:33,614 DEBUG    [ceph.py:241:populate()] Fetching ceph osd information
2020-11-25 14:25:33,652 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:25:33,714 DEBUG    [storage.py:105:refresh()] Refreshing disk information from the config object
2020-11-25 14:25:33,714 DEBUG    [storage.py:108:refresh()] - Scanning will use 8 scan threads
2020-11-25 14:25:33,811 DEBUG    [storage.py:135:refresh()] - rbd image scan complete: 0s
2020-11-25 14:25:33,811 DEBUG    [gateway.py:378:refresh()] Refreshing gateway & client information
2020-11-25 14:25:33,811 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:25:33,864 DEBUG    [ceph.py:260:refresh()] Gathering pool stats for cluster 'ceph'
2020-11-25 14:26:02,665 DEBUG    [gateway.py:174:ui_command_create()] CMD: /iscsi create iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
2020-11-25 14:26:02,666 DEBUG    [gateway.py:185:ui_command_create()] Create an iscsi target definition in the UI
2020-11-25 14:26:03,191 INFO     [gateway.py:196:ui_command_create()] ok
2020-11-25 14:26:45,455 DEBUG    [gateway.py:793:ui_command_create()] CMD: ../gateways/ create dev13 ['192.168.200.23'] nosync=False skipchecks=false
2020-11-25 14:26:45,467 INFO     [gateway.py:836:ui_command_create()] Adding gateway, sync'ing 0 disk(s) and 0 client(s)
2020-11-25 14:26:45,949 DEBUG    [gateway.py:854:ui_command_create()] Gateway creation successful
2020-11-25 14:26:45,949 DEBUG    [gateway.py:855:ui_command_create()] Adding gw to UI
2020-11-25 14:26:45,968 DEBUG    [gateway.py:934:refresh()] - checking iSCSI/API ports on dev13
2020-11-25 14:26:45,979 INFO     [gateway.py:874:ui_command_create()] ok
2020-11-25 14:28:41,572 DEBUG    [ceph.py:32:__init__()] Adding ceph cluster 'ceph' to the UI
2020-11-25 14:28:42,269 DEBUG    [ceph.py:241:populate()] Fetching ceph osd information
2020-11-25 14:28:42,297 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:28:42,334 DEBUG    [storage.py:105:refresh()] Refreshing disk information from the config object
2020-11-25 14:28:42,334 DEBUG    [storage.py:108:refresh()] - Scanning will use 8 scan threads
2020-11-25 14:28:42,378 DEBUG    [storage.py:135:refresh()] - rbd image scan complete: 0s
2020-11-25 14:28:42,378 DEBUG    [gateway.py:378:refresh()] Refreshing gateway & client information
2020-11-25 14:28:42,449 DEBUG    [gateway.py:934:refresh()] - checking iSCSI/API ports on dev13
2020-11-25 14:28:42,463 DEBUG    [ceph.py:150:update_state()] Querying ceph for state information
2020-11-25 14:28:42,517 DEBUG    [ceph.py:260:refresh()] Gathering pool stats for cluster 'ceph'
2020-11-25 14:28:52,659 DEBUG    [gateway.py:793:ui_command_create()] CMD: ../gateways/ create ceph-gateway1 ['192.168.200.33'] nosync=False skipchecks=false
2020-11-25 14:28:52,672 INFO     [gateway.py:836:ui_command_create()] Adding gateway, sync'ing 0 disk(s) and 0 client(s)
2020-11-25 14:28:53,406 DEBUG    [gateway.py:854:ui_command_create()] Gateway creation successful
2020-11-25 14:28:53,407 DEBUG    [gateway.py:855:ui_command_create()] Adding gw to UI



root@dev13:~# sudo gwcli export copy
{
    "created": "2020/11/25 10:54:39",
    "discovery_auth": {
        "mutual_password": "",
        "mutual_password_encryption_enabled": false,
        "mutual_username": "",
        "password": "",
        "password_encryption_enabled": false,
        "username": ""
    },
    "disks": {},
    "epoch": 2,
    "gateways": {
        "dev13": {
            "active_luns": 0,
            "created": "2020/11/25 10:56:45",
            "updated": "2020/11/25 10:56:45"
        }
    },
    "targets": {
        "iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw": {
            "acl_enabled": true,
            "auth": {
                "mutual_password": "",
                "mutual_password_encryption_enabled": false,
                "mutual_username": "",
                "password": "",
                "password_encryption_enabled": false,
                "username": ""
            },
            "clients": {},
            "controls": {},
            "created": "2020/11/25 10:56:02",
            "disks": {},
            "groups": {},
            "ip_list": [
                "192.168.200.23"
            ],
            "portals": {
                "dev13": {
                    "gateway_ip_list": [
                        "192.168.200.23"
                    ],
                    "inactive_portal_ips": [],
                    "portal_ip_addresses": [
                        "192.168.200.23"
                    ],
                    "tpgs": 1
                }
            },
            "updated": "2020/11/25 10:56:45"
        }
    },
    "updated": "2020/11/25 10:56:45",
    "version": 11
}

What should I do for this problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.