Git Product home page Git Product logo

purestorage-flocker-driver's Introduction

Pure Storage Flocker Driver

The plugin to provide Pure Storage FlashArray Flocker integration.

Description

ClusterHQ/Flocker provides an efficient and easy way to connect persistent store with Docker containers. This project provides a plugin to provision resillient storage from Pure Storage.

Pure Storage Flocker Intergration Block Diagram

Pure Storage Flocker Integration Block Diagram

Installation

  • Install OpeniSCSI and Multipath tools

    • Ubuntu
    sudo apt-get update
    sudo apt-get install -y open-iscsi multipath-tools python-dev
    • Centos
    sudo yum update
    sudo yum -y install iscsi-initiator-utils device-mapper-multipath
  • Install ClusterHQ/Flocker
    https://docs.clusterhq.com/en/latest/

  • Install Pure Storage Plugin

    Easiest way:

    git clone https://github.com/PureStorage-OpenConnect/purestorage-flocker-driver.git
    pip install purestorage-flocker-driver/

    The pip install will probably need to be run as root or with sudo depending on the environment. If flocker is running with a virtual environment you may need to install into the same one. For example on Ubuntu 14.04 or CentOS 7, as root, run:

    source /opt/flocker/bin/activate
    git clone https://github.com/PureStorage-OpenConnect/purestorage-flocker-driver.git
    pip install purestorage-flocker-driver/

    Alternatively you can clone or download the driver code and install manually.

    git clone https://github.com/PureStorage-OpenConnect/purestorage-flocker-driver.git
    cd purestorage-flocker-driver
    python setup.py install

    Note that this will not install the dependencies. You will need to manually install requirements from the requirements.txt file (specific versions are important!).

  • Ensure that scsi_id is available on the path for the flocker-dataset-agent service.

    export PATH=$PATH:/lib/udev

    Alternatively, it may be easier on some systems to simply put a symlink into /bin

    ln /lib/udev/scsi_id /bin/scsi_id

    This is required for part of the volume attachment process, and is not always in the same location depending on the OS.

  • (Fibre Channel Only) Setup zoning for compute nodes and FlashArray. The driver assumes that zoning has been configured already and will not attempt to automatically zone any nodes.

Usage Instructions

To start the driver on a node, a configuration file must exist on the node at /etc/flocker/agent.yml. This should be as follows, replacing ${pure_ip} & ${pure_api} with the ip/hostname and API token of the Pure Storage FlashArray:

control-service: {hostname: '1.2.3.4', port: 5678}
version: 1
dataset:
    backend: purestorage_flasharray_flocker_driver
    pure_ip: ${pure_ip}
    pure_api_token: ${pure_api}
    pure_storage_protocol: ${pure_storage_protocol}  # Optional
    pure_manage_purity_hosts: ${PURE_MANAGE_PURITY_HOSTS} # Optional
    pure_chap_host_user: ${pure_chap_host_user}  # Optional
    pure_chap_host_password: ${pure_chap_host_password}  # Optional
    pure_verify_https: ${pure_verify_https} # Optional
    pure_ssl_cert: ${pure_ssl_cert}  # Optional

Example agent.yml dataset configuration for Pure:

version: 1
control-service:
   hostname: flocker-controller-1.dev.purestorage.com
   port: 4524

dataset:
    backend: purestorage_flasharray_flocker_driver
    pure_ip: 10.231.128.11
    pure_api_token: 661f9687-0b1e-7b0d-e07d-1e776d50f9eb

Required Parameters

pure_ip
This is the management vip or hostname for the FlashArray.
pure_api_token
A valid api token for the FlashArray being managed.

Optional Parameters

pure_manage_purity_hosts
When True the driver will automatically create and modify Purity hosts to ensure WWNS, IQNS, and/or CHAP credentials are set correctly. Defaults to True.
pure_storage_protocol
The type of storage protocol being used. Valid options are ISCSI or FIBRE_CHANNEL. Defaults to ISCSI.
pure_chap_host_user
The iSCSI CHAP host username to use. This is only used when pure_storage_protocol is ISCSI. To enable CHAP you must set both pure_chap_host_user and pure_chap_host_password.
pure_chap_host_password
The iSCSI CHAP host password to use. This is only used when pure_storage_protocol is ISCSI. To enable CHAP you must set both pure_chap_host_user and pure_chap_host_password.
pure_verify_https
Force verification of HTTPS requests. When enabled, if the certificates are not trusted requests will fail. Defaults to False.
pure_ssl_cert
If pure_verify_https is True then you may specify a path to ca bundle/certificate for use in request validation. Otherwise system defaults will be used.

Contribution

Create a fork of the project into your own repository. Make all your necessary changes and create a pull request with a description on what was added or removed and details explaining the changes in lines of code. If approved, project owners will merge it.

Running Tests

Setup the Flocker dev environment

git clone https://github.com/ClusterHQ/flocker.git

virtualenv flocker-test-env
source ./flocker-test-env/bin/activate

cd ./flocker
pip install --requirement dev-requirements.txt
cd ..

Refer to https://docs.clusterhq.com/en/latest/gettinginvolved/contributing.html for the most up to date instructions.

All further steps need to be using the virtualenv!

Install the Pure driver

git clone https://github.com/PureStorage-OpenConnect/purestorage-flocker-driver.git

cd purestorage-flocker-driver/
pip install .

Run the tests (from the root of purestorage-flocker-driver, while )

trial tests.test_purestorage

This will, by default, look for a configuration file in /etc/flocker/agent.yml If you wish to specify a custom location export a AGENT_CONFIG_FILE environment variable with the full path and it will be used instead. These configuration files are expected to have the same backend definition as a normal agent.yml file using a Pure Storage FlashArray backend.

Compatibility

This plugin has been tested and verified with Flocker 1.14.0 on Ubuntu and CentOS based systems.

It will support Pure FlashArray's running Purity version 3.4.0 and newer.

Licensing

Pure Storage will not provide legal guidance on which open source license should be used in projects. We do expect that all projects and contributions will have a valid open source license, or align to the appropriate license for the project/contribution

Copyright [2016] [Pure Storage Inc.]

Some small portions of this driver were forked from the Apache 2.0 licensed repository at https://github.com/dellstorage/storagecenter-flocker-driver, and the appropriate copyrights apply to that portion.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Support

Please file bugs and issues at the Github issues page. For more general discussions you can contact the Flocker team at Google Groups. The code and documentation are released with no warranties or SLAs and are intended to be supported through a community driven process.

purestorage-flocker-driver's People

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

purestorage-flocker-driver's Issues

/etc/flocker/agent.yml file sample

Greetings,

Thanks so much for writing this driver. I just want to make sure that my agent.yml file makes sense. I would appreciate if you'd be able to post a sample agent.yml file which I can model after. Here is mine, and if you could please verify if settings are correct?

File: /etc/flocker/agent.yml

"version": 1
"control-service": 
  "hostname": 192.168.120.156
  "port": 4523
"dataset": 
    "backend": "purestorage_flasharray_flocker_driver"
    "pure_ip": 172.16.128.157 
    "pure_api_token": "50cac451-fefe-b635-9692-5870aada9c49"
    "pure_storage_protocol": "ISCSI"
    "pure_chap_host_user": "server1"
    "pure_chap_host_password": "pureBXPserver1"

Trying to test pure storage connectivity:

[root@server1 flocker]# trial tests.test_purestorage
tests
  test_purestorage ...                                                  [ERROR]

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/twisted/trial/runner.py", line 644, in loadByNames
    things.append(self.findByName(name))
  File "/usr/lib64/python2.7/site-packages/twisted/trial/runner.py", line 454, in findByName
    return reflect.namedAny(name)
  File "/usr/lib64/python2.7/site-packages/twisted/python/reflect.py", line 513, in namedAny
    raise ObjectNotFound('%r does not name an object' % (name,))
twisted.python.reflect.ObjectNotFound: 'tests.test_purestorage' does not name an object

tests.test_purestorage
-------------------------------------------------------------------------------
Ran 1 tests in 0.249s

FAILED (errors=1)
[root@server1 flocker]# 

Thanks again!

Flocker Volume creation command results in pending state

Operating System:

NAME="Red Hat Enterprise Linux Server"
VERSION="7.2 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.2"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.2 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.2:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.2"

Flocker environment:

clusterhq-python-flocker-1.13.0-1.x86_64
clusterhq-flocker-docker-plugin-1.13.0-1.noarch
clusterhq-flocker-node-1.13.0-1.noarch
Flocker /etc/flocker/agent.yml file

"version": 1
"control-service": 
  "hostname": 192.168.120.156 
  "port": 4524
"dataset": 
    "backend": "purestorage_flasharray_flocker_driver"
    "pure_ip": 172.16.128.157 
    "pure_api_token": "50cac451-fefe-b635-9692-5870aada9c49"
    "pure_chap_host_user": "mslave1"
    "pure_chap_host_password": "XXXXX"

Problem:

Volume creation command results in pending state.

**Step-by-step recreation:

Display agent nodes:

[root@mmaster1 new.cert]#  /usr/local/bin/flockerctl --control-service 192.168.120.156 --user user list-nodes
SERVER     ADDRESS       
1b8bd056   192.168.120.162 
4d35f998   192.168.120.163 
06978551   192.168.120.161 
701fdaad   192.168.120.164 
fb2b98cd   192.168.120.165 

[root@mmaster1 new.cert]# 

Create 10GB volume:

[root@mmaster1 new.cert]# flockerctl --control-service=192.168.120.156 create --node  1b8bd056 --size 10Gb 
created dataset in configuration, manually poll state with 'flocker-volumes list' to see it show up.

[root@mmaster1 new.cert]#

List volumes and status:

[root@mmaster1 new.cert]# /usr/local/bin/flockerctl --control-service 192.168.120.156 list
DATASET                                SIZE     METADATA                 STATUS        SERVER                   
83790f44-1afb-4e43-b03b-e8749be46f01   10.00G                            pending ⌛   1b8bd056 (192.168.120.162) 
b5afba76-949c-44ac-876f-08330f6c5bee   5.00G    name=apples,size=small   pending ⌛   1b8bd056 (192.168.120.162) 

[root@mmaster1 new.cert]# 

As you can see, there are two volumes are being in Pending state...

Journal log entries on mslave1 which is flocker node id 1b8bd056:

Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983423.412408, "task_uuid": "eb6e8448-07e1-4124-a008-5317bb75a243", "action_type": "eliot:remote_task", "action_status": "started", "task_level": [7, 2, 1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"exception": "exceptions.KeyError", "task_level": [7, 2, 2], "action_type": "eliot:remote_task", "reason": "\"Attribute or key '83790f44-1afb-4e43-b03b-e8749be46f01' not found in UnicodeManifestationPMap({u'b5afba76-949c-44ac-876f-08330f6c5bee': Manifestation(primary=True, dataset=Dataset(deleted=False, dataset_id=u'b5afba76-949c-44ac-876f-08330f6c5bee', maximum_size=5368709120, metadata=pmap({u'name': u'apples', u'size': u'small'})))})\"", "timestamp": 1471983423.427221, "task_uuid": "eb6e8448-07e1-4124-a008-5317bb75a243", "action_status": "failed"}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "996124a0-0d10-4e77-8e1e-947b7834b84b", "error": true, "timestamp": 1471983423.436451, "message": "Unhandled Error\nTraceback (most recent call last):\n  File \"/opt/flocker/lib/python2.7/site-packages/twisted/protocols/amp.py\", line 1021, in _commandReceived\n    deferred = self.dispatchCommand(box)\n  File \"/opt/flocker/lib/python2.7/site-packages/twisted/protocols/amp.py\", line 1079, in dispatchCommand\n    return maybeDeferred(responder, box)\n  File \"/opt/flocker/lib/python2.7/site-packages/twisted/internet/defer.py\", line 150, in maybeDeferred\n    result = f(*args, **kw)\n  File \"/opt/flocker/lib/python2.7/site-packages/twisted/protocols/amp.py\", line 1166, in doit\n    return maybeDeferred(aCallable, **kw).addCallback(\n--- <exception caught here> ---\n  File \"/opt/flocker/lib/python2.7/site-packages/twisted/internet/defer.py\", line 150, in maybeDeferred\n    result = f(*args, **kw)\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/control/_protocol.py\", line 1157, in cluster_updated_diff\n    self._current_configuration\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/control/_diffing.py\", line 432, in apply\n    proxy = c.apply(proxy)\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/control/_diffing.py\", line 100, in apply\n    self.path, lambda o: o.set(self.key, self.value)\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/control/_diffing.py\", line 390, in transform\n    target = _get_or_add_proxy_child(target, segment)\n  File \"/opt/flocker/lib/python2.7/site-packages/flocker/control/_diffing.py\", line 348, in _get_or_add_proxy_child\n    segment, parent_proxy._original\nexceptions.KeyError: \"Attribute or key '83790f44-1afb-4e43-b03b-e8749be46f01' not found in UnicodeManifestationPMap({u'b5afba76-949c-44ac-876f-08330f6c5bee': Manifestation(primary=True, dataset=Dataset(deleted=False, dataset_id=u'b5afba76-949c-44ac-876f-08330f6c5bee', maximum_size=5368709120, metadata=pmap({u'name': u'apples', u'size': u'small'})))})\"\n", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "503dfc52-f314-4ca4-ab44-1ddc1dde43ce", "error": false, "timestamp": 1471983423.452151, "message": "AgentAMP connection lost (HOST:IPv4Address(TCP, '192.168.120.161', 55783) PEER:IPv4Address(TCP, '192.168.120.156', 4524))", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ClusterStatus object at 0x42daad0>", "fsm_input": "<ClusterStatusInputs=DISCONNECTED_FROM_CONTROL_SERVICE>", "timestamp": 1471983423.453291, "fsm_rich_input": null, "action_status": "started", "task_uuid": "13348bd8-2880-49f7-bff9-6806e8f4d2a0", "action_type": "fsm:transition", "fsm_state": "<ClusterStatusStates=KNOWLEDGEABLE>", "task_level": [1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ConvergenceLoop object at 0x42bc7d0>", "fsm_input": "<ConvergenceLoopInputs=STOP>", "timestamp": 1471983423.454234, "fsm_rich_input": null, "action_status": "started", "task_uuid": "13348bd8-2880-49f7-bff9-6806e8f4d2a0", "action_type": "fsm:transition", "fsm_state": "<ConvergenceLoopStates=SLEEPING>", "task_level": [2, 1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ConvergenceLoopStates=STOPPED>", "task_level": [2, 2], "action_type": "fsm:transition", "timestamp": 1471983423.454929, "fsm_output": ["<ConvergenceLoopOutputs=CLEAR_WAKEUP>"], "task_uuid": "13348bd8-2880-49f7-bff9-6806e8f4d2a0", "action_status": "succeeded"}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ClusterStatusStates=DISCONNECTED>", "task_level": [3], "action_type": "fsm:transition", "timestamp": 1471983423.455499, "fsm_output": ["<ClusterStatusOutputs=STOP>"], "task_uuid": "13348bd8-2880-49f7-bff9-6806e8f4d2a0", "action_status": "succeeded"}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "3f7459a3-07b1-4c8f-a1b9-c700badb5934", "error": false, "timestamp": 1471983423.457014, "message": "<twisted.internet.tcp.Connector instance at 0x413c680> will retry in 3 seconds", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:03 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "a500c269-3b5c-4b40-b3b3-6cc6d164aa4c", "error": false, "timestamp": 1471983423.467202, "message": "Stopping factory <twisted.internet.protocol.ReconnectingClientFactory instance at 0x7fcb1403edd0>", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:06 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "c890d693-81e1-4318-ae14-a27f2adeb8c2", "error": false, "timestamp": 1471983426.653772, "message": "Starting factory <twisted.internet.protocol.ReconnectingClientFactory instance at 0x7fcb1403edd0>", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:06 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "2bb2d11d-de76-4a0f-a7dc-e22548dfa0c7", "error": false, "timestamp": 1471983426.666226, "message": "AgentAMP connection established (HOST:IPv4Address(TCP, '192.168.120.161', 35975) PEER:IPv4Address(TCP, '192.168.120.156', 4524))", "message_type": "twisted:log", "task_level": [1]}
Aug 23 16:17:06 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ClusterStatus object at 0x42daad0>", "fsm_input": "<ClusterStatusInputs=CONNECTED_TO_CONTROL_SERVICE>", "timestamp": 1471983426.667331, "fsm_rich_input": "<_ConnectedToControlService>", "action_status": "started", "task_uuid": "beeb75dd-fc61-4ad8-94ed-580501d17a16", "action_type": "fsm:transition", "fsm_state": "<ClusterStatusStates=DISCONNECTED>", "task_level": [1]}
Aug 23 16:17:06 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ClusterStatusStates=IGNORANT>", "task_level": [2], "action_type": "fsm:transition", "timestamp": 1471983426.667744, "fsm_output": ["<ClusterStatusOutputs=STORE_CLIENT>"], "task_uuid": "beeb75dd-fc61-4ad8-94ed-580501d17a16", "action_status": "succeeded"}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983427.795813, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "eliot:remote_task", "action_status": "started", "task_level": [4, 2, 1]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ClusterStatus object at 0x42daad0>", "fsm_input": "<ClusterStatusInputs=STATUS_UPDATE>", "timestamp": 1471983427.823243, "fsm_rich_input": "<_StatusUpdate>", "action_status": "started", "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "fsm:transition", "fsm_state": "<ClusterStatusStates=IGNORANT>", "task_level": [4, 2, 2, 1]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ConvergenceLoop object at 0x42bc7d0>", "fsm_input": "<ConvergenceLoopInputs=STATUS_UPDATE>", "timestamp": 1471983427.823805, "fsm_rich_input": "<_ClientStatusUpdate>", "action_status": "started", "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "fsm:transition", "fsm_state": "<ConvergenceLoopStates=STOPPED>", "task_level": [4, 2, 2, 2, 1]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983427.824237, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:agent:converge", "action_status": "started", "task_level": [4, 2, 2, 2, 2, 1]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983427.824591, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:agent:discovery", "action_status": "started", "task_level": [4, 2, 2, 2, 2, 2, 1]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"count": 135, "function": "_discover_raw_state", "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "timestamp": 1471983427.824912, "message_type": "flocker:node:agents:blockdevice:list_volumes", "task_level": [4, 2, 2, 2, 2, 2, 2]}
Aug 23 16:17:07 mslave1 flocker-dataset-agent[11611]: {"Info": "Found Purity volume managed by flocker {u'source': None, u'serial': u'7E4D75D0FFC14E7C00011010', u'size': 5368709120, u'name': u'flocker-bf10eb7b-1119-4699-b5afba76-949c-44ac-876f-08330f6c5bee', u'created': u'2016-08-23T18:21:24Z'}", "timestamp": 1471983427.897228, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "task_level": [4, 2, 2, 2, 2, 2, 3]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"Info": "Volume flocker-bf10eb7b-1119-4699-b5afba76-949c-44ac-876f-08330f6c5bee attached_to = mslave2", "timestamp": 1471983428.065972, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "task_level": [4, 2, 2, 2, 2, 2, 4]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.073424, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "message_type": "agent:blockdevice:raw_state", "raw_state": "RawState(compute_instance_id=u'mslave1', devices_with_filesystems=FilepathPSet([]), devices=UuidToFilepathPMap({}), system_mounts=FilepathToFilepathPMap({FilePath('/dev/mapper/vg_gluster-brick3'): FilePath('/bricks/brick3'), FilePath('/dev/mapper/vg_gluster-brick2'): FilePath('/bricks/brick2'), FilePath('/dev/mapper/vg_gluster-brick5'): FilePath('/bricks/brick5'), FilePath('/dev/mapper/vgsys-root'): FilePath('/'), FilePath('/dev/mapper/vgsys-diskdump'): FilePath('/diskdump'), FilePath('/dev/mapper/vg_gluster-brick4'): FilePath('/bricks/brick4'), FilePath('/dev/mapper/vgcore-opt_bsa_bladelogic'): FilePath('/opt/bsa/bladelogic'), FilePath('/dev/sda1'): FilePath('/boot'), FilePath('/dev/mapper/vgsys-var_tmp'): FilePath('/var/tmp'), FilePath('/dev/mapper/vgcore-users'): FilePath('/users'), FilePath('/dev/mapper/vgsys-tmp'): FilePath('/tmp'), FilePath('/dev/mapper/vgsys-var_log'): FilePath('/var/log'), FilePath('/dev/mapper/vgsys-var'): FilePath('/var'), FilePath('/dev/mapper/vg_gluster-brick1'): FilePath('/bricks/brick1'), FilePath('/dev/mapper/vgcore-usr_bltemp'): FilePath('/usr/bltemp')}), volumes=BlockdevicevolumePVector([BlockDeviceVolume(blockdevice_id=u'flocker-bf10eb7b-1119-4699-b5afba76-949c-44ac-876f-08330f6c5bee', dataset_id=UUID('b5afba76-949c-44ac-876f-08330f6c5bee'), attached_to=u'mslave2', size=5368709120)]), _live_instances=None)", "task_level": [4, 2, 2, 2, 2, 2, 5]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "task_level": [4, 2, 2, 2, 2, 2, 6], "action_type": "flocker:agent:discovery", "timestamp": 1471983428.075773, "state": "BlockDeviceDeployerLocalState(node_uuid=UUID('06978551-c7ee-41f6-8412-42f119777d04'), datasets=UuidToDiscovereddatasetPMap({UUID('b5afba76-949c-44ac-876f-08330f6c5bee'): DiscoveredDataset(blockdevice_id=u'flocker-bf10eb7b-1119-4699-b5afba76-949c-44ac-876f-08330f6c5bee', state=<DatasetStates=ATTACHED_ELSEWHERE>, maximum_size=5368709120, dataset_id=UUID('b5afba76-949c-44ac-876f-08330f6c5bee'))}), hostname=u'192.168.120.161')", "action_status": "succeeded"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"local_changes": [{"paths": {"values": [], "$__class__$": "PMap"}, "uuid": {"hex": "06978551-c7ee-41f6-8412-42f119777d04", "$__class__$": "UUID"}, "$__class__$": "NodeState", "hostname": "192.168.120.161", "devices": {"values": [], "$__class__$": "PMap"}, "applications": null, "manifestations": {"values": [], "$__class__$": "PMap"}}, {"datasets": {"values": [], "$__class__$": "PMap"}, "$__class__$": "NonManifestDatasets"}], "connection": "<AgentAMP at 0x473d310>", "task_level": [4, 2, 2, 2, 2, 3, 1], "action_type": "flocker:agent:send_to_control_service", "timestamp": 1471983428.079364, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_status": "started"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.082172, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "message_type": "flocker:agent:converge:actions", "task_level": [4, 2, 2, 2, 2, 4], "calculated_actions": "NoOp(sleep=datetime.timedelta(0, 60))"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.083624, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:change:noop", "action_status": "started", "task_level": [4, 2, 2, 2, 2, 5, 1]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.08429, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:change:noop", "action_status": "succeeded", "task_level": [4, 2, 2, 2, 2, 5, 2]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ConvergenceLoopStates=CONVERGING>", "task_level": [4, 2, 2, 2, 3], "action_type": "fsm:transition", "timestamp": 1471983428.087224, "fsm_output": ["<ConvergenceLoopOutputs=STORE_INFO>", "<ConvergenceLoopOutputs=CONVERGE>"], "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_status": "succeeded"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ClusterStatusStates=KNOWLEDGEABLE>", "task_level": [4, 2, 2, 3], "action_type": "fsm:transition", "timestamp": 1471983428.090916, "fsm_output": ["<ClusterStatusOutputs=UPDATE_STATUS>"], "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_status": "succeeded"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.091679, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "eliot:remote_task", "action_status": "succeeded", "task_level": [4, 2, 3]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.115874, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:agent:send_to_control_service", "action_status": "succeeded", "task_level": [4, 2, 2, 2, 2, 3, 3]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"delay": 52.606892414095064, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "log_level": "INFO", "timestamp": 1471983428.116817, "message": "Delaying until next convergence loop.", "message_type": "flocker:node:_loop:CONVERGE:delay", "task_level": [4, 2, 2, 2, 2, 6]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ConvergenceLoop object at 0x42bc7d0>", "fsm_input": "<ConvergenceLoopInputs=SLEEP>", "timestamp": 1471983428.119845, "fsm_rich_input": "<_Sleep>", "action_status": "started", "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "fsm:transition", "fsm_state": "<ConvergenceLoopStates=CONVERGING>", "task_level": [4, 2, 2, 2, 2, 7, 1]}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ConvergenceLoopStates=SLEEPING>", "task_level": [4, 2, 2, 2, 2, 7, 2], "action_type": "fsm:transition", "timestamp": 1471983428.120741, "fsm_output": ["<ConvergenceLoopOutputs=SCHEDULE_WAKEUP>"], "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_status": "succeeded"}
Aug 23 16:17:08 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983428.122268, "task_uuid": "36d3303d-9691-4374-bbda-78b055c9c791", "action_type": "flocker:agent:converge", "action_status": "succeeded", "task_level": [4, 2, 2, 2, 2, 8]}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983429.145261, "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_type": "eliot:remote_task", "action_status": "started", "task_level": [5, 2, 1]}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ClusterStatus object at 0x42daad0>", "fsm_input": "<ClusterStatusInputs=STATUS_UPDATE>", "timestamp": 1471983429.149796, "fsm_rich_input": "<_StatusUpdate>", "action_status": "started", "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_type": "fsm:transition", "fsm_state": "<ClusterStatusStates=KNOWLEDGEABLE>", "task_level": [5, 2, 2, 1]}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"fsm_identifier": "<flocker.node._loop.ConvergenceLoop object at 0x42bc7d0>", "fsm_input": "<ConvergenceLoopInputs=STATUS_UPDATE>", "timestamp": 1471983429.159345, "fsm_rich_input": "<_ClientStatusUpdate>", "action_status": "started", "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_type": "fsm:transition", "fsm_state": "<ConvergenceLoopStates=SLEEPING>", "task_level": [5, 2, 2, 2, 1]}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ConvergenceLoopStates=SLEEPING>", "task_level": [5, 2, 2, 2, 2], "action_type": "fsm:transition", "timestamp": 1471983429.160953, "fsm_output": ["<ConvergenceLoopOutputs=STORE_INFO>", "<ConvergenceLoopOutputs=UPDATE_MAYBE_WAKEUP>"], "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_status": "succeeded"}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"fsm_next_state": "<ClusterStatusStates=KNOWLEDGEABLE>", "task_level": [5, 2, 2, 3], "action_type": "fsm:transition", "timestamp": 1471983429.161595, "fsm_output": ["<ClusterStatusOutputs=UPDATE_STATUS>"], "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_status": "succeeded"}
Aug 23 16:17:09 mslave1 flocker-dataset-agent[11611]: {"timestamp": 1471983429.162255, "task_uuid": "9f7d164a-de82-4249-8b99-e5e28cd592bc", "action_type": "eliot:remote_task", "action_status": "succeeded", "task_level": [5, 2, 3]}

Display iSCSI sessions on agent node - mslave1:

[root@mslave1 log]# iscsiadm -m session
iscsiadm: No active sessions.
[root@mslave1 log]# 

Display multipath information:

[root@mslave1 log]# multipath -l
[root@mslave1 log]# 

Display disks by UUID:

[root@mslave1 log]# ls -l /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 10 Jul 12 09:06 02e06f8f-1815-4e8f-989e-3379fcc1888c -> ../../dm-1
lrwxrwxrwx 1 root root 11 Jul 29 14:44 409a500f-8b10-410c-bdd2-0bd844333925 -> ../../dm-10
lrwxrwxrwx 1 root root 10 Jul 12 09:06 62bc4c55-ed6b-4537-bf1c-462349b5a521 -> ../../dm-0
lrwxrwxrwx 1 root root 10 Jul 12 09:06 68acd516-3a51-4ac7-bbea-35a76e8b5605 -> ../../dm-4
lrwxrwxrwx 1 root root 10 Jul 12 09:06 6a6648f5-81d3-43b0-af8d-ce08785c4599 -> ../../dm-2
lrwxrwxrwx 1 root root 11 Jul 29 14:45 755b1b7b-a717-4148-b865-59dd4b9d181d -> ../../dm-11
lrwxrwxrwx 1 root root 11 Jul 29 14:45 81b9f973-c80f-4fa8-96d4-3e8e81acaf34 -> ../../dm-12
lrwxrwxrwx 1 root root 10 Jul 12 09:06 8d3e27b2-ba8c-4ae1-b21e-0a685a33e4d9 -> ../../dm-3
lrwxrwxrwx 1 root root 10 Jul 12 09:06 9a3c6ec1-a022-4cd4-b8f6-23382dede5d1 -> ../../dm-5
lrwxrwxrwx 1 root root 11 Jul 29 14:45 aa952633-c0a1-4235-a79e-06a8c333d159 -> ../../dm-13
lrwxrwxrwx 1 root root 10 Jul 12 09:06 ca6de9ff-b82c-4bc8-a73c-2ff96bf505ac -> ../../dm-6
lrwxrwxrwx 1 root root 10 Jul 12 09:06 cf3633b3-7e80-428c-ac80-4a67ed107a60 -> ../../dm-7
lrwxrwxrwx 1 root root 10 Jul 29 13:09 d4bfdd90-443e-4212-88f1-dd865c093ff7 -> ../../sda1
lrwxrwxrwx 1 root root 10 Jul 12 09:06 ea8e6bd0-676b-43a0-a1dd-56115f052c32 -> ../../dm-8
lrwxrwxrwx 1 root root 10 Jul 12 09:06 f371fe12-6eea-4613-b983-546897861c43 -> ../../dm-9
lrwxrwxrwx 1 root root 

Display disks by path:

[root@mslave1 log]# ls -l /dev/disk/by-path
total 0
lrwxrwxrwx 1 root root  9 Jul 12 09:06 pci-0000:00:07.1-ata-2.0 -> ../../sr0
lrwxrwxrwx 1 root root  9 Aug 18 09:28 pci-0000:03:00.0-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Jul 29 13:09 pci-0000:03:00.0-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Jul 29 13:09 pci-0000:03:00.0-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root  9 Aug 18 09:28 pci-0000:03:00.0-scsi-0:0:1:0 -> ../../sdb
lrwxrwxrwx 1 root root 10 Jul 12 09:06 pci-0000:03:00.0-scsi-0:0:1:0-part1 -> ../../sdb1
lrwxrwxrwx 1 root root  9 Aug 18 09:28 pci-0000:03:00.0-scsi-0:0:2:0 -> ../../sdc
lrwxrwxrwx 1 root root  9 Aug 18 09:28 pci-0000:03:00.0-scsi-0:0:3:0 -> ../../sdd
lrwxrwxrwx 1 root root  9 Aug 18 09:28 pci-0000:03:00.0-scsi-0:0:4:0 -> ../../sde
lrwxrwxrwx 1 root root 10 Jul 29 14:43 pci-0000:03:00.0-scsi-0:0:4:0-part1 -> ../../sde1
[root@mslave1 log]# 

Any pointers on how this can be troubleshooted ?

Thanks!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.