naudit / pystorcli2 Goto Github PK

View Code? Open in Web Editor NEW

2.0 1.0 7.0 252 KB

Python library provides wrapper around storcli tool to manage and control LSI MegaRAID controllers.

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

python python3 storcli wrapper wrapper-library

pystorcli2's Introduction

PyStorCLI2

PyStorCLI2 official fork from pystorcli.

How to use PyStorCLI2:

FIXME: create useful readme

pystorcli2's People

Contributors

Stargazers

Watchers

Forkers

chillisystems hertbertt zhangchi1992 ralequi xcnix liveaction ulmitov

pystorcli2's Issues

Drive state property is missing states

Hi,
Please add "JBOD'" state to this method:

    @property
    def state(self):
        """Get/Set drive state

        One of the following states can be set (str):
            online - changes the drive state to online
            offline - changes the drive state to offline
            missing - marks a drive as missing
            good - changes the drive state to unconfigured good
            jbod - sets the drive state to JBOD

        Returns:
            (str):
                dhs - dedicated hotspare to some virtual drive
                ghs - global hotspare
                bad - bad drive
                good - unconfigured good
                online - already in virtual drive with good state
                offline - already in virtual drive with bad state
        """
        args = [
            'show'
        ]

        state = self._response_properties(self._run(args))['State']
        if state == 'DHS':
            return 'dhs'
        elif state == 'UBad':
            return 'bad'
        elif state == 'Onln':
            return 'online'
        elif state == 'Offln':
            return 'offline'
        elif state == 'GHS':
            return 'ghs'
        return 'good'

Set drive state minor issue

Hi,

When setting drive.state= 'good' it fails (i don't have an output to paste right now)
since there is no 'good' state, only 'ugood' or 'unconfigured good'
while drive.state = 'offline''missing''online' works well, it makes a little confusion

Thanks

Installation issue

pip install  pystorcli2
Defaulting to user installation because normal site-packages is not writeable
Collecting pystorcli2
  Downloading PyStorCLI2-0.5.0-py3-none-any.whl (22 kB)
Installing collected packages: pystorcli2
Successfully installed pystorcli2-0.5.0

In [1]: import pystorcli2

In [3]: pystorcli2.Controllers()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [3], line 1
----> 1 pystorcli2.Controllers()

AttributeError: module 'pystorcli2' has no attribute 'Controllers'

pystorcli2 is incompatible with storcli2/percli2

I'm encountering more and more raid cards that are only compatible with storcli2/percli2, which is problematic with the current library. For the most part the commands are roughly the same, but the output is different. What do we think is the best way to handle different output like this? It would be relatively simple if the keys were just different in name, but some are nested under varying levels of keys to get the same answer; ie

storcli64 output for getting virtual drive facts

"Response Data" : {
"VD0 Properties" : {
"Strip Size" : "256 KB",

storcli2 output for getting virtual drive facts

"Response Data" : {
"Virtual Drives" : [
{
"VD Info" : {
"DG/VD" : "0/1",
"TYPE" : "RAID6",
"State" : "Optl",
"Access" : "RW",
"CurrentCache" : "NR,AWB",
"DefaultCache" : "NR,AWB",
"Size" : "392.896 TiB",
"Name" : ""
},
"PDs" : [
{
"EID:Slt" : "355:6",
...
"Alt-EID" : "-"
}
],
"VD Properties" : {
"Strip Size" : "256 KiB",
"Block Size" : 4096,
"Number of Blocks" : 1054674616

Is it best to keep it in the same library? Create a new library? Probably goes along the lines the of the factory pattern conversation and pyarcconf.

Format attributes

Hi,

Currenlty the serial, model and other fields are not being formatted and have white spaces and different letter case.
Please apply space stripping and to upper() format.
One of purposes for this is also to have same formatting like in pySMART

Example:

In [9]: print(d['megaraid,23'].hba_obj.facts)
{'Drive /c0/e252/s3': [{'EID:Slt': '252:3', 'DID': 23, 'State': 'UGood', 'DG': '-', 'Size': '446.625 GB', 'Intf': 'SATA', 'Med': 'SSD', 'SED': 'N', 'PI': 'N', 'SeSz': '512B', 'Model': 'INTEL SSDSC2KB480G8', 'Sp': 'U', 'Type': '-'}], 'Drive /c0/e252/s3 - Detailed Information': {'Drive /c0/e252/s3 State': {'Shield Counter': 0, 'Media Error Count': 0, 'Other Error Count': 0, 'Drive Temperature': ' 24C (75.20 F)', 'Predictive Failure Count': 0, 'S.M.A.R.T alert flagged by drive': 'No'}, 'Drive /c0/e252/s3 Device attributes': {'SN': 'PHYF1100065U480BGN  ', 'Manufacturer Id': 'ATA     ', 'Model Number': 'INTEL SSDSC2KB480G8', 'NAND Vendor': 'NA', 'WWN': '55CD2E415339189F', 'Firmware Revision': 'XCV10132', 'Raw size': '447.130 GB [0x37e436b0 Sectors]', 'Coerced size': '446.625 GB [0x37d40000 Sectors]', 'Non Coerced size': '446.630 GB [0x37d436b0 Sectors]', 'Device Speed': '6.0Gb/s', 'Link Speed': '6.0Gb/s', 'NCQ setting': 'Enabled', 'Write Cache': 'N/A', 'Logical Sector Size': '512B', 'Physical Sector Size': '4 KB', 'Connector Name': 'Port 0 - 3 x1'}, 'Drive /c0/e252/s3 Policies/Settings': {'Enclosure position': '1', 'Connected Port Number': '8(path0) ', 'Sequence Number': 35, 'Commissioned Spare': 'No', 'Emergency Spare': 'No', 'Last Predictive Failure Event Sequence Number': 0, 'Successful diagnostics completion on': 'N/A', 'FDE Type': 'None', 'SED Capable': 'No', 'SED Enabled': 'No', 'Secured': 'No', 'Cryptographic Erase Capable': 'Yes', 'Sanitize Support': 'Not supported', 'Locked': 'No', 'Needs EKM Attention': 'No', 'PI Eligible': 'No', 'Certified': 'No', 'Wide Port Capable': 'No', 'Multipath': 'No', 'Port Information': [{'Port': 0, 'Status': 'Active', 'Linkspeed': '6.0Gb/s', 'SAS address': '0x4433221108000000'}]}, 'Inquiry Data': '40 00 ff 3f 37 c8 10 00 00 00 00 00 3f 00 00 00 00 00 00 00 48 50 46 59 31 31 30 30 36 30 55 35 38 34 42 30 4e 47 20 20 00 00 00 00 00 00 43 58 31 56 31 30 32 33 4e 49 45 54 20 4c 53 53 53 44 32 43 42 4b 38 34 47 30 20 38 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 01 80 00 40 00 2f 00 40 00 00 00 00 06 00 ff 3f 10 00 3f 00 10 fc fb 00 01 fd ff ff ff 0f 00 00 07 00 '}}

In [5]: d['megaraid,23'].hba_obj.serial
Out[5]: 'PHYF1100065U480BGN  '

In [6]: d['megaraid,23'].hba_obj.model
Out[6]: 'intel ssdsc2kb480g8'

( in this code hba_obj is an added attribute to pySMART.Drive which holds the pystorcli.Drive object )

Storcli runner: some commands do not have json format

Hi,
there is an exception when running some commands, for example:

In [4]: ctrl._run(['show', 'events', 'file=/root/raid_events.log'])
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/storcli.py:189, in StorCLI.run(self, args, stdout, stderr, **kwargs)
    188 try:
--> 189     ret_json = json.loads(ret.stdout)
    190     self.check_response_status(cmd, ret_json)

File /usr/lib/python3.9/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
    347 if cls is None:

File /usr/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
    333 """Return the Python representation of ``s`` (a ``str`` instance
    334 containing a JSON document).
    335 
    336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338 end = _w(s, end).end()

File /usr/lib/python3.9/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
    354 except StopIteration as err:
--> 355     raise JSONDecodeError("Expecting value", s, err.value) from None
    356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
Cell In[4], line 1
----> 1 a._run(['show', 'events', f'file=/root/raid_events.log'])

File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/controller.py:228, in Controller._run(self, args, **kwargs)
    226 args = args[:]
    227 args.insert(0, self._name)
--> 228 return self._storcli.run(args, **kwargs)

File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/storcli.py:197, in StorCLI.run(self, args, stdout, stderr, **kwargs)
    194         return ret_json
    195     except json.JSONDecodeError:
    196         # :/
--> 197         err = re.search('(^.*)Storage.*Command.*$',
    198                         ret.stdout, re.MULTILINE | re.DOTALL).group(1)
    199         raise exc.StorCliCmdError(cmd, err)
    200 except subprocess.TimeoutExpired as err:

AttributeError: 'NoneType' object has no attribute 'group'

This command does not have a json format:

root@ubuntu20-cuda:~# storcli64 /c0 show events file=/root/raid_events.log J
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Controller = 0
Status = Success
Description = None

Events = GETEVENTS

Controller Properties :
=====================

------------------------------------
Ctrl Status  Method          Value  
------------------------------------
   0 Success handleSuboption Events 
------------------------------------

Init a failed controller

Hi,

Having this kind of controller:

COMMAND: storcli64 /c0 show J
RC:
0
STDOUT:
{
"Controllers":[
{
	"Command Status" : {
		"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
		"Operating system" : "Linux 5.4.0-99-generic",
		"Controller" : 0,
		"Status" : "Failure",
		"Description" : "None",
		"Detailed Status" : [
			{
				"Ctrl" : 0,
				"Status" : "Failed",
				"ErrCd" : 59,
				"ErrMsg" : "Incomplete foreign configuration"
			}
		]
	},
	"Response Data" : {
		"Product Name" : "AVAGO JBOD",

Trying to init it with pystorcli:

In [2]: import pystorcli as pp

In [3]: c=pp.controller.Controllers()

In [4]: c.ids
Out[4]: [0]

In [5]: x=c.get_ctl(0)
---------------------------------------------------------------------------
StorCliMissingError                       Traceback (most recent call last)
Cell In[5], line 1
----> 1 x=c.get_ctl(0)

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:478, in Controllers.get_ctl(self, ctl_id)
    468 def get_ctl(self, ctl_id: int) -> Optional[Controller]:
    469     """Get controller object by id
    470 
    471     Args:
   (...)
    476         (:obj:Controller): controller object
    477     """
--> 478     for ctl in self:
    479         if ctl.id == ctl_id:
    480             return ctl

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:457, in Controllers._ctls(self)
    454 @ property
    455 def _ctls(self):
    456     for ctl_id in self._ctl_ids:
--> 457         yield Controller(ctl_id=ctl_id, binary=self._binary)

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:71, in Controller.__init__(self, ctl_id, binary)
     68 self._storcli = StorCLI(binary)
     69 self._name = '/c{0}'.format(self._ctl_id)
---> 71 self._exist()

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:85, in Controller._exist(self)
     83     self._run(['show'])
     84 except exc.StorCliCmdError:
---> 85     raise exc.StorCliMissingError(
     86         self.__class__.__name__, self._name) from None

StorCliMissingError: Object 'Controller' doesnt exist: /c0

The exception is saying that object does not exist.
But the real exception should be:

In [9]: c._storcli.run(['/c0', 'show'])
---------------------------------------------------------------------------
StorCliCmdError                           Traceback (most recent call last)
Cell In[9], line 1
----> 1 c._storcli.run(['/c0', 'show'])

File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:184, in StorCLI.run(self, args, stdout, stderr, **kwargs)
    182 try:
    183     ret_json = json.loads(ret.stdout)
--> 184     self.check_response_status(cmd, ret_json)
    185     ret.check_returncode()
    186     if self.cache_enable:

File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:144, in StorCLI.check_response_status(cmd, out)
    142 if cmd_status['Status'] == 'Failure':
    143     if 'Detailed Status' in cmd_status:
--> 144         raise exc.StorCliCmdError(
    145             cmd, "{0}".format(cmd_status['Detailed Status']))
    146     else:
    147         raise exc.StorCliCmdError(cmd, "{0}".format(cmd_status))

StorCliCmdError: Command '/usr/sbin/storcli64 /c0 show J' error: [{'Ctrl': 0, 'Status': 'Failed', 'ErrCd': 59, 'ErrMsg': 'Incomplete foreign configuration'}]

On the other hand, the controller exists and maybe pystorcli should create the object (although controller has status failed).
Since we can make it healthy:

In [10]: c._storcli.run(['/c0/fall', 'del'])
Out[10]: 
{'Controllers': [{'Command Status': {'CLI Version': '007.2309.0000.0000 Sep 16, 2022',
    'Operating system': 'Linux 5.4.0-99-generic',
    'Controller': 0,
    'Status': 'Success',
    'Description': 'Operation on foreign configuration Succeeded'},
   'Response Data': {'Total Foreign PDs': 0}}]}

In [11]: c._storcli.run(['/c0', 'show'])
Out[11]: 
{'Controllers': [{'Command Status': {'CLI Version': '007.2309.0000.0000 Sep 16, 2022',
    'Operating system': 'Linux 5.4.0-99-generic',
    'Controller': 0,
    'Status': 'Success',
    'Description': 'None'},
   'Response Data': {'Product Name': 'AVAGO JBOD',

But if pystorcli will not create an object this would not be possible.
Now the init is passing:

In [12]: x=c.get_ctl(0)

In [13]: x
Out[13]: <pystorcli.controller.Controller at 0x7f3601a33cd0>

Code excepts when storcli returns no objects

On a host without LSI cards:

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:554, in Controllers.ids(self)
    550 @ property
    551 def ids(self):
    552     """(list of str): controllers id
    553     """
--> 554     return self._ctl_ids

File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:533, in Controllers._ctl_ids(self)
    531 @ property
    532 def _ctl_ids(self) -> List[int]:
--> 533     out = self._storcli.run(['show'], allow_error_codes=[
    534         StorcliError.INCOMPLETE_FOREIGN_CONFIGURATION])
    535     response = common.response_data(out)
    537     if "Number of Controllers" in response and response["Number of Controllers"] == 0:

File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:210, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
    208 # output in JSON format
    209 cmd.append('J')
--> 210 cmd_cache_key = ''.join(cmd)
    212 if self.cache_enable:
    213     if cmd_cache_key in self.__response_cache:

TypeError: sequence item 0: expected str instance, bytes found

In [9]: !storcli64 /call show
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status = Failure
Description = No Controller found

Drive state setter issue

COMMAND: storcli64 /c0/e252/s15 set offline J
RC:
0
STDOUT:
{
"Controllers":[
{
	"Command Status" : {
		"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
		"Operating system" : "Linux 5.4.0-99-generic",
		"Controller" : 0,
		"Status" : "Success",
		"Description" : "Set Drive Offline Succeeded."
	}
}
]
}


STDERR:

Start / End / Elapsed	 15:40:25.739667 / 15:40:25.941269 / 0:00:00.201602
2023-03-09 15:40:25 [ERROR] [set_drive_state] has an error
Traceback (most recent call last):
  File "/home/umit/dev/dev-tests/testsuites/functional/ssd/test_raid.py", line 358, in set_drive_state
    d.state = 'offline'
  File "/home/umit/.local/lib/python3.10/site-packages/pystorcli/drive.py", line 585, in state
    return common.response_setter(self._run(args))
  File "/home/umit/.local/lib/python3.10/site-packages/pystorcli/common.py", line 57, in response_setter
    return response_cmd(data)['Detailed Status'][0]['Value']
KeyError: 'Detailed Status'

INVALID_STATUS text is not really correct

Hi,
Regarding this code in error.py
INVALID_STATUS = (
255, '', 'Invalid status - used for polling command completion.')

It is a a bit misleading, not sure if 255 is same for all cases but i got this one:

storcli64 /c0/e252/s7 set good J
{
"Controllers":[
{
	"Command Status" : {
		"CLI Version" : "007.2408.0000.0000 Nov 15, 2022",
		"Operating system" : "Linux 5.15.0-25-generic",
		"Controller" : 0,
		"Status" : "Failure",
		"Description" : "Set Drive Good Failed.",
		"Detailed Status" : [
			{
				"Drive" : "/c0/e252/s7",
				"Status" : "Failure",
				"ErrCd" : 255,
				"ErrMsg" : "Operation not allowed."
			}
		]
	}
}
]
}

Init of controllers fails on HBA card

Hi,

filing another one.
Init of controllers fails on some cards on this line:

return [ctl['Ctl'] for ctl in common.response_data(out)['System Overview']]

The "System overview" appears to be "IT System Overview" sometimes.
Examples:

Output of HBA card:

CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status Code = 0
Status = Success
Description = None

Number of Controllers = 1
Host Name = ubuntu20-cuda
Operating System  = Linux 5.4.0-99-generic
StoreLib IT Version = 07.2400.0200.0100
StoreLib IR3 Version = 16.14-0

IT System Overview :
==================

---------------------------------------------------------------------------
Ctl Model        AdapterType   VendId DevId SubVendId SubDevId PCI Address 
---------------------------------------------------------------------------
  0 HBA 9400-16i   SAS3416(B0) 0x1000  0xAC    0x1000   0x3000 00:3b:00:00 
---------------------------------------------------------------------------

Output of RAID card:

CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status Code = 0
Status = Success
Description = None

Number of Controllers = 1
Host Name = ubuntu20-cuda
Operating System  = Linux 5.4.0-99-generic

System Overview :
===============

----------------------------------------------------------------------
Ctl Model     Ports PDs DGs DNOpt VDs VNOpt BBU sPR DS  EHS ASOs Hlth 
----------------------------------------------------------------------
  0 AVAGOJBOD    16  16   0     0   0     0 N/A On  1&2 Y      2 Opt  
----------------------------------------------------------------------

Drive object - Enhancement request

Hi,

Kindly asking to add a new attribute to Drive object - Drive.smart_obj (equals to None).
This is to be able to update it later with current drive's pysmart drive object (or any other object).

has_foreign_configurations raises on failure

The following exception is being raised if command's rc is non zero:

In [4]: c['/c0'].has_foreign_configurations()
ERROR:root:RC=59 returned from /usr/sbin/storcli64 /c0 /fall show J
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/storcli.py:225, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
    224 if ret.returncode != 0:
--> 225     raise subprocess.CalledProcessError(
    226         ret.returncode, cmd, ret.stdout, ret.stderr)
    227 if self.cache_enable:

CalledProcessError: Command '['/usr/sbin/storcli64', '/c0', '/fall', 'show', 'J']' returned non-zero exit status 59.

During handling of the above exception, another exception occurred:

StorCliRunTimeError                       Traceback (most recent call last)
Cell In[4], line 1
----> 1 c['/c0'].has_foreign_configurations()

File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/controller/__init__.py:435, in Controller.has_foreign_configurations(self, securitykey)
    432     args.append(f'securitykey={securitykey}')
    434 try:
--> 435     fc_data = common.response_data(self._run(args))
    436     fcs = 0
    438     if 'Total foreign Drive Groups' in fc_data:

File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/controller/__init__.py:83, in Controller._run(self, args, allow_error_codes, **kwargs)
     81 args = args[:]
     82 args.insert(0, self._name)
---> 83 return self._storcli.run(args, allow_error_codes=allow_error_codes, **kwargs)

File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/storcli.py:252, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
    250     raise exc.StorCliRunTimeout(err)
    251 except subprocess.SubprocessError as err:
--> 252     raise exc.StorCliRunTimeError(err)

StorCliRunTimeError: Command '/usr/sbin/storcli64 /c0 /fall show J' returned with non-zero exit status 59:

This was the output:

12:28:52 [ERROR] RC=59 returned from /usr/sbin/storcli64 /c0 /fall show J
12:28:52 [ERROR] STDERR:
STDOUT:{
"Controllers":[
{
	"Command Status" : {
		"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
		"Operating system" : "Linux 5.4.0-99-generic",
		"Controller" : 0,
		"Status" : "Failure",
		"Description" : "Incomplete foreign configuration"
	},
	"Response Data" : {
		"Total Foreign PDs" : 16,
		"Total Locked Foreign PDs" : 0
	}
}
]
}

Additional params in methods - create_vd

Hi,
Regarding controller.create_vd,

For raid1 and raid10 must specify PDperArray=2,
otherwise the add vd command will fail with:
"operation not possible for current RAID level, Cannot create configuration with 1 span".

Can you please add a parameter to this method so it can get some additional params to append to the add vd command ?
Probably the additional params will be needed in other methods also.

Migrate into factory design

Requested by @ulmitov

The design pattern - pystorcli uses singleton when creating the StorCLI object. while pysmart just creates it once and then passes to all rest of objects (somewhat a factory design). I think the factory is much better.