ibm / pyds8k Goto Github PK

DS8000 Python client(pyds8k), which establishes terminal connection with IBM DS8000 storage systems

License: Apache License 2.0

Python 100.00%

pyds8k's Introduction

DS8000 Python Client

This repository contains the IBM RESTful API Python client, which establishes terminal connection with IBM DS8000 storage systems. The Python client protocol enables full management and monitoring of these storage arrays by issuing dedicated RESTful APIs.

Python Compatibility

The content in this collection supports Python 3.6 and newer.

Getting started

Clone the repository, and then add it to your PYTHONPATH directory. The Python client is then ready for import and use. The library is also available to install using pip. See the pypi pyds8k project

To install via pip run the following command:

pip install pyds8k

Documentation

Documentation for the pyds8k library can be generated using sphinx. The documentation for the latest release is also available via pyds8k.readthedocs.io

NOTE: To view older versions of the doc, click on the link at the bottom right corner in the readthedocs link and select the desired version.

Usage examples

Usage examples of the Python client are available in the examples.py file.

Displaying the RESTful APIs reference information

Each storage system of DS8000 and major software version has its own set of RESTful APIs. The RESTful APIs are detailed in the RESTful API reference guides that are available on IBM Knowledge Center (KC).

To display the full RESTful API Reference Guide of a specific storage system and a specific software version:

Navigate to a storage system welcome page on KC:
On the welcome page, select a storage system software version. For example, select Version 8.5.3.

The welcome page of the selected software version is displayed.
If needed, select the Table of contents tab.
On the table of contents, click RESTful API.
Refer to Host commands and to all subsequent chapters.

Contributing

We do not accept any contributions at the moment. This may change in the future, so you can fork, clone, and suggest a pull request.

Running tests

Use tox to run the test suite

tox

Use pytest to run tests or a specific test.

pytest --cov-config=.coveragerc --cov pyds8k --disable-warnings -v
pytest --cov-config=.coveragerc --cov pyds8k --disable-warnings -v pyds8k/test/test_resources/test_ds8k/test_host.py

pyds8k's People

Contributors

Stargazers

Watchers

Forkers

27149chen secabhk001 bhaskers-blu-org1 nick-clayton bwesselb theued ghas-results

pyds8k's Issues

CVE-2023-32681 vulnerability

new vulnerability detected by the IBM mend procedure for this repo.
CVE-2023-32681 for requests package.

returned objects bugs and inconsistency

get_volume - flashcopy attribute is all storage flashcopies and not only the volume's.

>>> volume = client.get_volume('0000')
>>> volume.flashcopy
[<FlashCopy: 1310:1311>, <FlashCopy: 503A:503C>, <FlashCopy: 5130:51D1>, <FlashCopy: 51AA:51EB>, <FlashCopy: 51D1:51D6>, <FlashCopy: 55A9:55AA>]

flashcopy - 2 different objects flashcopy/flashcopies which dont have all the data and missing out_of_sync_tracks/backroundcopy respectively.
flashcopy is the object that returns from get_flashcopies_by_volume and flashcopies from get_cs_flashcopies

>>> flashcopies_by_volume = client.get_flashcopies_by_volume("503A")
>>> flashcopies_by_volume
[<FlashCopy: 503A:503C>]
>>> type(flashcopies_by_volume[0])
<class 'pyds8k.resources.ds8k.v1.flashcopy.FlashCopy'> -> flashcopy
>>> flashcopies_by_volume[0].backgroundcopy
'disabled'

>>> cs_flashcopies = client.get_cs_flashcopies("503A:503C")
>>> cs_flashcopies
<FlashCopy: 503A:503C>
>>> type(cs_flashcopies)
<class 'pyds8k.resources.ds8k.v1.cs.flashcopies.FlashCopy'> -> flashcopies
>>> cs_flashcopies.backgroundcopy
Traceback (most recent call last):
File "/usr/lib64/python3.8/code.py", line 90, in runcode
exec(code, self.locals)
File "", line 1, in
File "/opt/app-root/lib64/python3.8/site-packages/pyds8k/resources/ds8k/v1/common/base.py", line 97, in getattr
return super(Base, self).getattr(key)
File "/opt/app-root/lib64/python3.8/site-packages/pyds8k/base.py", line 393, in getattr
raise AttributeError(k)
AttributeError: backgroundcopy

Documented examples are not working

The examples here do not work
https://github.com/IBM/pyds8k/blob/develop/pyds8k/example.py

Obviously the object structure changed:

from pyds8k.client.ds8k.v1.client import Client
restclient = Client(service_address='9.155.xxx.xxx', user='username', password='********', verify=False)
vol = restclient.get_volumes('9A00')
print(vol)
<Storage Volume: 9A00>
print(vol['name'])
Traceback (most recent call last):
File "", line 1, in
TypeError: 'Volume' object is not subscriptable

To print the name, following syntax has to be used

print(vol.name)
csm_test_9A00

The verify = False parameter for the restclient call should also be documented in the examples, otherwise the connection to DS8000 Rest API with default certificates does not work (default is verify = true)

Execution error on Windows Python

On Windows, the Python module does not work:
Traceback (most recent call last):
File "E:/Education/Python/IBM/ds8ktest.py", line 3, in
from pyds8k.client.ds8k.v1.client import Client
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\client\ds8k\v1\client.py", line 20, in
from pyds8k.resources.ds8k.v1.systems import System,
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\resources\ds8k\v1_init_.py", line 17, in
from . import ioports, flashcopy, events, mappings, pprc, eserep,
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\resources\ds8k\v1\ioports.py", line 24, in
from .io_enclosures import IOEnclosure, IOEnclosureManager
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\resources\ds8k\v1\io_enclosures.py", line 27, in
class IOEnclosure(Base):
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\six.py", line 894, in wrapper
return metaclass(cls.name, cls.bases, orig_vars)
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\base.py", line 94, in new
prefix = get_resource_route_prefix_by_class(new_class)
File "C:\Users\ThomasLuther\AppData\Local\Programs\Python\Python310\lib\site-packages\pyds8k\base.py", line 85, in get_resource_route_prefix_by_class
return route.rsplit(".resources.", 2)[1]
IndexError: list index out of range

Its because of the different path separator. I found a possible fix for it in:
https://github.com/IBM/pyds8k/blob/develop/pyds8k/base.py

I changed line 84 from
route = path.replace("/", ".")
to
route = path.replace(os.sep, ".")

Not sure if there are other places that prevent the libs can be used on Windows Python.

long waiting for get_volumes_by_pool request

our creation process takes a long time (about 45 sec, whereas in comparison it only takes about 5 sec via the SVC storage family)
we timed the get_volumes_by_pool command on dscli client vs pyds8k vs postman on pool with 240 vols and then on pool with 4 vols for comparison and fun.

results:
with 244 vols:
dscli client: 17s
pyds8k: 26s
postman: 24s

with 4 vols:
dscli client: 15s
pyds8k: 4.5s
postman: 4s

we can see that pyds8k is not giving us worst performance than the postman (which uses rest as well) so if there is a problem it's with the rest itself

Example usage returns `405 Method Not Allowed`

I'm just getting started trying out this client so this may be entirely user error..

Install pyds8k

 $ python3 -m venv pyds8k
 $ . pyds8k/bin/activate
 (pyds8k) $ pip install pyds8k
>>> OUTPUT REMOVED <<<

Try example

(pyds8k) $ python
Python 3.6.8 (default, Sep 26 2019, 11:57:09) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyds8k.client.ds8k.v1.client import Client
>>> restclient = Client(service_address='REDACTED', user='REDACTED', password='REDACTED')
>>> restclient.get_volumes()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/resources/ds8k/v1/common/mixins.py", line 180, in get_volumes
    return self.all(types.DS8K_VOLUME, rebuild_url=True).list()
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/base.py", line 434, in list
    return self.manager.list(**kwargs)
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/resources/ds8k/v1/volumes.py", line 136, in list
    return self._list(url=url, obj_class=obj_class, body=body, **kwargs)
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/base.py", line 724, in _list
    _, body = self.client.get(self.url, **kwargs)
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/httpclient.py", line 230, in get
    return self.request(url, 'GET', **kwargs)
  File "/home/vmorris/pyds8k/lib64/python3.6/site-packages/pyds8k/httpclient.py", line 210, in request
    raise exceptions.raise_error(resp, body, self.service_type)
pyds8k.exceptions.MethodNotAllowed: HTTP 405 Method Not Allowed. Method Not Allowed