Python library for working with HPE Container Platform (Prototype)

Home Page: https://hpe-container-platform-community.github.io/hpecp-python-library/index.html

License: MIT License

Python 96.12% Dockerfile 0.68% Shell 3.20%

hpe-container-platform kubernetes python python-cli python-library

hpecp-python-library's Introduction

hpe-container-platform-community

github pages landing page: https://hpe-container-platform-community.github.io/

hpecp-python-library's People

Contributors

Stargazers

Watchers

Forkers

pramaku snowch-forks prakashmirji rahul1804

hpecp-python-library's Issues

Write blob post about the library

Implement client.role.list()

Implement client.role.list() (see GatewayController.list() for an example)

GET /api/v1/role HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
...

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 516
Content-Type: application/json
Date: Mon, 18 May 2020 17:44:01 GMT
Server: BlueData EPIC 5.0

{
  "_links": {
    "self": {
      "href": "/api/v1/role"
    }
  },
  "_embedded": {
    "roles": [
      {
        "_links": {
          "self": {
            "href": "/api/v1/role/1"
          }
        },
        "label": {
          "name": "Site Admin",
          "description": "Role for Site Admin"
        }
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/role/4"
          }
        },
        "label": {
          "name": "K8S Admin",
          "description": "Role for Kubernetes Admin"
        }
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/role/3"
          }
        },
        "label": {
          "name": "Member",
          "description": "Role for Tenant Member"
        }
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/role/2"
          }
        },
        "label": {
          "name": "Admin",
          "description": "Role for Tenant Admin"
        }
      }
    ]
  }
}

Coverage seems to use stderr on 3.x causing tests to fail

For now the stderr checks have been disabled on 3.x

======================================================================
FAIL: test_get_output_with_invalid_catalog_id (library.catalog_test.TestCLIGet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.5.9/x64/lib/python3.5/site-packages/mock/mock.py", line 1330, in patched
    return func(*args, **keywargs)
  File "/home/runner/work/hpecp-python-library/hpecp-python-library/tests/library/catalog_test.py", line 703, in test_get_output_with_invalid_catalog_id
    self.assertEqual(error, expected_stderr)
AssertionError: '/home/runner/work/hpecp-python-library/h[211 chars]ist.' != "'/api/v1/catalog/101' does not exist."
- /home/runner/work/hpecp-python-library/hpecp-python-library/hpecp/client.py:147: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmpo6x_o8sm'>
-   config.readfp(codecs.open(config_file, "r", "utf8"))
  '/api/v1/catalog/101' does not exist.

======================================================================
FAIL: test_get_output_with_unknown_exception (library.catalog_test.TestCLIGet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.5.9/x64/lib/python3.5/site-packages/mock/mock.py", line 1330, in patched
    return func(*args, **keywargs)
  File "/home/runner/work/hpecp-python-library/hpecp-python-library/tests/library/catalog_test.py", line 731, in test_get_output_with_unknown_exception
    self.assertEqual(stderr, expected_stderr)
AssertionError: '/home/runner/work/hpecp-python-library/hp[227 chars]EBUG' != 'Unknown error. To debug run with env var LOG_LEVEL=DEBUG'
- /home/runner/work/hpecp-python-library/hpecp-python-library/hpecp/client.py:147: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmpc5sx8bhc'>
-   config.readfp(codecs.open(config_file, "r", "utf8"))
  Unknown error. To debug run with env var LOG_LEVEL=DEBUG

======================================================================
FAIL: test_get_failed_login (library.cli_test.TestCLIHttpClient)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.5.9/x64/lib/python3.5/site-packages/mock/mock.py", line 1330, in patched
    return func(*args, **keywargs)
  File "/home/runner/work/hpecp-python-library/hpecp-python-library/tests/library/cli_test.py", line 244, in test_get_failed_login
    "Could not connect to controller - set LOG_LEVEL=DEBUG to see more detail.\n",
AssertionError: '/home/runner/work/hpecp-python-library/hp[246 chars]l.\n' != 'Could not connect to controller - set LOG[30 chars]l.\n'
- /home/runner/work/hpecp-python-library/hpecp-python-library/hpecp/client.py:147: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmpu6g84uk3'>
-   config.readfp(codecs.open(config_file, "r", "utf8"))
  Could not connect to controller - set LOG_LEVEL=DEBUG to see more detail.

Provide --json parameter as an alternative option to --json-file

Implement bash autocomplete for User functionality

See AutoComplete.bash() in bin/cli.py for examples of license, etc.

Implement client.user.delete()

Implement client.user.delete() (see GatewayController.delete() for example)

DELETE /api/v1/user/16 HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 0
Accept-Encoding: gzip, deflate
...

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Mon, 18 May 2020 17:40:55 GMT
Server: BlueData EPIC 5.0

Provide password parameter on user.create()

It should be possible to provide a password to user.create() when is_internal==True:

{“password”:“********“,”is_external”:false,“label”:{“name”:“test”,“description”:“test”}}

Color terminal output for CLI

Maybe using: https://pypi.org/project/blessings/

Stderr should be read, Stdout white

Wait-for-state/status should display progress bar.

Gateway tabulate column order is broken

Refactor common code to base class or mixin

There's a lot of repeated code in gateway, k8scluster, k8sworker, etc for list() wait_for_x(), etc. Investigate moving this to a base class or a mixin

Implement client.user.create()

Implement client.user.create() (see GatewayController.create_with_ssh() for example)

internal user:

POST /api/v1/user HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 68
Accept-Encoding: gzip, deflate
...

{"is_external": true, "label": {"name": "chris", "description": ""}}
HTTP/1.1 201 Created
Access-Control-Allow-Origin: *
Content-Length: 13
Content-Type: text/plain
Date: Mon, 18 May 2020 17:21:47 GMT
Location: /api/v1/user/17
Server: BlueData EPIC 5.0

201 Created

external user:

POST /api/v1/user HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 68
Accept-Encoding: gzip, deflate
...

{"is_external": true, "label": {"name": "chris", "description": ""}}
HTTP/1.1 201 Created
Access-Control-Allow-Origin: *
Content-Length: 13
Content-Type: text/plain
Date: Mon, 18 May 2020 17:21:47 GMT
Location: /api/v1/user/17
Server: BlueData EPIC 5.0

201 Created

Add docs for `User` functionality

The live docs can be found here: https://hpe-container-platform-community.github.io/hpecp-python-library/hpecp.html#

To view docs in gitpod - see here: https://github.com/hpe-container-platform-community/hpecp-python-library/blob/master/DEVELOPING.md#building-docs

This doc definition will need updating: https://raw.githubusercontent.com/hpe-container-platform-community/hpecp-python-library/master/docs/source/hpecp.rst

A new doc definition file will need to be created: hpecp.user.rst with the user documentation code. See here for an example: https://raw.githubusercontent.com/hpe-container-platform-community/hpecp-python-library/master/docs/source/hpecp.gateway.rst

Implement client.license.upload() methods

Implement:

client.license.upload_with_ssh_key()
client.license.upload_with_ssh_pass()

See:

Implement client.catalog.refresh()

POST /api/v1/catalog HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 21
Accept-Encoding: gzip, deflate
...

{"action": "refresh"}
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Mon, 18 May 2020 18:32:25 GMT
Server: BlueData EPIC 5.0

Implement client.role.get()

Implement client.role.get() (see GatewayController.get() for an example)

GET /api/v1/role/1 HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
...

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 147
Content-Type: application/json
Date: Mon, 18 May 2020 17:53:49 GMT
Server: BlueData EPIC 5.0

{
  "_links": {
    "self": {
      "href": "/api/v1/role/1"
    },
    "all_roles": {
      "href": "/api/v1/role"
    }
  },
  "label": {
    "name": "Site Admin",
    "description": "Role for Site Admin"
  }
}

Provide objects for Gateway list() and get()

See:

Implement `user.list()` in CLI

This would be done in the file: https://github.com/hpe-container-platform-community/hpecp-python-library/blob/master/bin/cli.py#L710

Implement /api/v2/k8skubeconfig/

Requires tenant session (see #191)

GET /api/v2/k8skubeconfig/ HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/13bb569a-ed47-4591-96e6-7a322afb8463
Authorization: /api/v2/session/13bb569a-ed47-4591-96e6-7a322afb8463

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Disposition: attachment; filename="config"
Content-Length: 2481
Content-Type: text/yaml
Date: Wed, 22 Jul 2020 11:10:24 GMT
Server: HPE Ezmeral Container Platform 5.1

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxxx==
    server: https://ip-10-1-0-68.us-west-2.compute.internal:9500
  name: demo-hpecp-c1
contexts:
- context:
    cluster: demo-hpecp-c1
    user: demo-hpecp-admin
  name: demo-hpecp-c1-admin
- context:
    cluster: demo-hpecp-c1
    user: demo-hpecp-admin
    namespace: t1
  name: demo-hpecp-c1-t1-admin
current-context: demo-hpecp-c1-t1-admin
kind: Config
preferences: {}
users:
- name: demo-hpecp-admin
  user:
    exec:
      command: kubectl
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - hpecp
      - authenticate
      - ip-10-1-0-68.us-west-2.compute.internal:8080
      - --hpecp-user=admin
      - --hpecp-token=/api/v2/session/13bb569a-ed47-4591-96e6-7a322afb8463
      - --hpecp-token-expiry=1595498029
      - --force-reauth=false
      - --insecure=true
      - --insecure-skip-tls-verify=true

Implement client.tenant.revoke_user_from_role()

Implement client.tenant.revoke_user_from_role(tenant_id, role_id, client_id)

PUT /api/v1/tenant/2?user HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 76
Accept-Encoding: gzip, deflate
...

{
  "operation": "revoke",
  "role": "/api/v1/role/3",
  "user": "/api/v1/user/17"
}
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Mon, 18 May 2020 18:12:09 GMT
Server: BlueData EPIC 5.0

support dir/file autocomplete with stdout redirect

hpecp k8scluster admin-kube-config --id /api/v2/k8scluster/1 > generated/admin_kube_config.conf

set_storage is broken

Details coming soon...

Implement client.tenant.assign_user_to_role()

Implement client.tenant.assign_user_to_role(tenant_id, role_id, user_id)

PUT /api/v1/tenant/2?user HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 76
Accept-Encoding: gzip, deflate
...

{
  "operation": "assign",
  "role": "/api/v1/role/3",
  "user": "/api/v1/user/17"
}
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Mon, 18 May 2020 17:44:12 GMT
Server: BlueData EPIC 5.0

create a kubectl plugin out of your cli tool

here you can find more information:
https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/

Log output should be encoded as json

Use json.dumps on

investigate providing a local dev environment, e.g. codeserver

https://hub.docker.com/r/codercom/code-server

docker run -it -p 127.0.0.1:8080:8080 \
  -v "$PWD:/home/coder/project" \
  -u "$(id -u):$(id -g)" \
  codercom/code-server:latest

Refactor role.py and user.py to use base_resource.py

See gateway.py for an example

Implement query on k8sworker

Example:

hpecp k8sworker list --query status:['storage_pending']

See: https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#cli-usage-output-filter

Note: k8s_worker.py will need updating and also cli.py.

Implement /api/v2/k8scluster/1/change_task

POST /api/v2/k8scluster/1/change_task HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 82
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/13bb569a-ed47-4591-96e6-7a322afb8463
Content-Type: application/json
Authorization: /api/v2/session/13bb569a-ed47-4591-96e6-7a322afb8463

{"change_spec": {"addons": ["picasso"]}, "operation": "reconfigure", "reason": ""}HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Wed, 22 Jul 2020 10:12:27 GMT
Server: HPE Ezmeral Container Platform 5.1

Manually implement autocomplete script

Python fire auto completion is broken. In the meantime, manually implement the autocomplete script.

See example: https://cement.readthedocs.io/en/stable-2.4.x/examples/bash_auto_completion/#example-bash-rc

Implement /api/v1/config/

GET /api/v1/config/ HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c
Authorization: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 4466
Content-Type: application/json
Date: Tue, 21 Jul 2020 14:11:29 GMT
Server: HPE Ezmeral Container Platform 5.1

{
  "result": "Success",
  "objects": {
    "timezone": "UTC",
    "total_tenant_storage": 3072,
    "auth_types": [
      "LDAP",
      "Active Directory"
    ],
    "cpu_allocation_ratio": 1,
    "ram_allocation_ratio": 1,
    "proxy_port_map_ranges": [
      {
        "start_port": 10000,
        "end_port": 50000
      }
    ],
    "external_auth": "Active Directory",
    "gateway_ssl_cert_info": [],
    "tenant_storage_root": {
      "bdfs_root": {
        "path_from_endpoint": "/hcp"
      },
      "endpoint": {
        "cluster_name": "hcp.mapr.cluster",
        "ticket": "hcp-service-ticket",
        "type": "mapr",
        "secure": true,
        "cldb": [
          "ip-10-1-0-210.us-west-2.compute.internal"
        ]
      },
      "label": {
        "name": "TenantStorage",
        "description": "Protected DataTap for a tenant-specific sandboxed storage space."
      }
    },
    "tmp_defaultfs_root": {
      "bdfs_root": {
        "path_from_endpoint": "/hcp"
      },
      "endpoint": {
        "cluster_name": "hcp.mapr.cluster",
        "ticket": "hcp-service-ticket",
        "type": "mapr",
        "secure": true,
        "cldb": [
          "ip-10-1-0-210.us-west-2.compute.internal"
        ]
      },
      "label": {
        "name": "TenantStorage",
        "description": "Protected DataTap for a tenant-specific sandboxed storage space."
      }
    },
    "external_auth_params": {
      "type": "Active Directory",
      "ldap_protocol": "ldaps",
      "user_attr": "sAMAccountName",
      "bind_type": "search_bind",
      "tls_enabled": false,
      "saml_available": true,
      "host": "10.1.0.132",
      "port": 636,
      "base_dn": "CN=Users,DC=samdom,DC=example,DC=com",
      "bind_dn": "cn=Administrator,CN=Users,DC=samdom,DC=example,DC=com",
      "tls_verify_peer": false
    },
    "tenant_types": [
      {
        "tenant_type": "k8s",
        "quota_support": {
          "cores": true,
          "gpus": true,
          "memory_gb": true,
          "disk_gb": true,
          "tenant_storage_gb": true,
          "persistent_storage_size_gb": true
        },
        "label": {
          "name": "K8S",
          "description": "Tenant type for managing vclusters in Kubernetes."
        },
        "fields": [
          {
            "id": "specified_namespace_name",
            "name": "Specified Namespace Name",
            "tooltip": "A name for the namespace that will be created or adopted. If this is omitted in the case of creating a namespace, a unique name will be autogenerated.",
            "type": "string",
            "required": false,
            "message": "Optionally enter a specified namespace name",
            "editable": false
          },
          {
            "id": "adopt_existing_namespace",
            "name": "Adopt Existing Namespace",
            "tooltip": "Whether to attach the tenant to an existing namespace. If true, a namespace name must also be specified. If false, a namespace will be created.",
            "type": "boolean",
            "required": false,
            "message": "Choose whether to adopt an existing namespace",
            "default": false,
            "editable": false
          },
          {
            "id": "is_namespace_owner",
            "name": "Is Namespace Owner",
            "tooltip": "Whether the namespace (and all its contents) should be deleted on tenant deletion.",
            "type": "boolean",
            "required": false,
            "message": "Choose whether the tenant will own the namespace",
            "default": true
          },
          {
            "id": "map_services_to_gateway",
            "name": "Map Services To Gateway",
            "tooltip": "Whether NodePort services in this tenant's namespace will automatically be plumbed to a gateway port.",
            "type": "boolean",
            "required": false,
            "message": "Choose whether to auto-map NodePort services to gateway",
            "default": true
          },
          {
            "id": "enable_service_mesh",
            "name": "Enable Istio Service Mesh",
            "tooltip": "Whether Istio Envoy sidecar proxy injection will be enabled for this namespace.",
            "type": "boolean",
            "required": false,
            "message": "Choose whether to enable service mesh on this namespace.",
            "default": false
          }
        ]
      },
      {
        "tenant_type": "docker",
        "quota_support": {
          "cores": true,
          "gpus": true,
          "memory_gb": true,
          "disk_gb": true,
          "tenant_storage_gb": true,
          "persistent_storage_size_gb": true
        },
        "label": {
          "name": "On-Prem",
          "description": "Tenant type for managing clusters locally."
        },
        "fields": [
          {
            "id": "qos_multiplier",
            "name": "QOS Multiplier",
            "tooltip": "Input an integer (between 1 and 32) to prioritize the cluster performance in this tenant relative to other tenants. Higher integer value results in allocation of proportionately larger \"CPU\" shares.",
            "type": "integer",
            "required": true,
            "message": "Please enter a valid QOS Multiplier",
            "default": 1,
            "min": 1,
            "max": 32,
            "editable": true
          }
        ]
      }
    ],
    "system_quota": [
      {
        "tenant_type": "docker",
        "cores": 0,
        "memory": 0,
        "swap": 0,
        "disk": 0,
        "nodes": 0,
        "gpus": 0
      },
      {
        "tenant_type": "k8s",
        "cores": 24,
        "memory_gb": 183,
        "gpus": 0,
        "persistent_storage_size_gb": 3072,
        "ephemeral_storage_size_gb": 3072
      }
    ],
    "upload_image_settings_fields": [
      {
        "tenant_type": "docker"
      },
      {
        "tenant_type": "k8s"
      }
    ],
    "bds_network_publicinterface": "ens5",
    "bds_install_workerreboot": "false",
    "bdshared_install_agent": "false",
    "bds_global_bundleflavor": "minimal",
    "bds_global_version": "5.1",
    "bds_global_buildnumber": "2055",
    "bds_ha_enabled": "No",
    "bds_global_user": "centos",
    "bdshared_global_bddomain": "demo.bdlocal",
    "bdshared_global_bdprefix": "bluedata-",
    "bds_storage_cstorevg": "VolBDSCStore",
    "bdshared_ha_clusterip": "",
    "cluster_dns_name": ""
  }
}

autocomplete does not support --ssh-key-file folder/ expansion

Implement text output for: `hpecp k8sworker list`

Similar to:

EXISTING_GATEWAY_IDS=$(hpecp gateway list --columns "['id']" --output text)

Check and recreate session if expires

remove excluded files from flake8 docstring tests

https://github.com/hpe-container-platform-community/hpecp-python-library/blob/master/.travis.yml#L39

Implement client.catalog.list()

Implement client.catalog.list() (see GatewayController.list() for an example)

GET /api/v1/catalog/ HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
...

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 19842
Content-Type: application/json
Date: Mon, 18 May 2020 18:35:23 GMT
Server: BlueData EPIC 5.0

{
  "_links": {
    "self": {
      "href": "/api/v1/catalog/"
    },
    "feedlog": {
      "href": "/api/v1/catalog/feedlog"
    },
    "feed": [
      {
        "href": "http://127.0.0.1:8080/api/v1/feed/local",
        "name": "Feed generated from local bundles."
      },
      {
        "href": "https://s3.amazonaws.com/bluedata-catalog/bundles/catalog/external/docker/EPIC-5.0/feeds/feed.json",
        "name": "BlueData EPIC-5.0 catalog feed for docker"
      }
    ]
  },
  "catalog_api_version": 6,
  "feeds_refresh_period_seconds": 86400,
  "feeds_read_counter": 5,
  "catalog_write_counter": 5,
  "_embedded": {
    "independent_catalog_entries": [
      {
        "_links": {
          "self": {
            "href": "/api/v1/catalog/29"
          },
          "feed": [
            {
              "href": "https://s3.amazonaws.com/bluedata-catalog/bundles/catalog/external/docker/EPIC-5.0/feeds/feed.json",
              "name": "BlueData EPIC-5.0 catalog feed for docker"
            }
          ]
        },
        "distro_id": "bluedata/spark240juphub7xssl",
        "label": {
          "name": "Spark240",
          "description": "Spark240 multirole with Jupyter Notebook, Jupyterhub with SSL and gateway node"
        },
        "version": "2.8",
        "timestamp": 0,
        "isdebug": false,
        "osclass": [
          "centos"
        ],
        "logo": {
          "checksum": "1471eb59356066ed4a06130566764ea6",
          "url": "http://10.1.0.53/catalog/logos/bluedata-spark240juphub7xssl-2.8"
        },
        "documentation": {
          "checksum": "52f53f1b2845463b9e370d17fb80bea6",
          "mimetype": "text/markdown",
          "file": "/opt/bluedata/catalog/documentation/bluedata-spark240juphub7xssl-2.8"
        },
        "state": "initialized",
        "state_info": ""
      },
      ... items removed for brevity
    ]
  }
}

Implement CLI: `hpecp k8sworker set-storage`

Implement CLI

    data = {"op_spec": {"persistent_disks": ["/dev/nvme2n1"], "ephemeral_disks": ["/dev/nvme1n1"]}, "op": "storage"}
    client.k8s_worker.set_storage(worker_id=worker_id, data=data)

Implement client.catalog.install()

Implement client.catalog.install()

POST /api/v1/catalog/1 HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 21
Accept-Encoding: gzip, deflate
...

{"action": "install"}
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Content-Length: 0
Content-Type: text/plain
Date: Mon, 18 May 2020 18:51:09 GMT
Server: BlueData EPIC 5.0

autocomplete should not repeat parameters

hpecp lock create --reason --reason --reason

CLI error - ImportError: No module named _struct

The following error is encountered running on Centos 7.7:

[centos@ip-10-1-0-176 2]$ pip2 install --quiet --upgrade git+https://github.com/hpe-container-platform-community/hpecp-client@master
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python2 /usr/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpVSVQuM
       cwd: /tmp/pip-req-build-xVMcvp
  Complete output (10 lines):
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 17, in <module>
      import json
    File "/usr/lib64/python2.7/json/__init__.py", line 108, in <module>
      from .decoder import JSONDecoder
    File "/usr/lib64/python2.7/json/decoder.py", line 5, in <module>
      import struct
    File "/usr/lib64/python2.7/struct.py", line 1, in <module>
      from _struct import *
  ImportError: No module named _struct
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python2 /usr/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpVSVQuM Check the logs for full command output.
[centos@ip-10-1-0-176 2]$

autocomplete --columns not working

implement /api/v1/install (includes reconfig, e.g. cpu allocation ratio)

GET /api/v1/install/ HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c
Authorization: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 509
Content-Type: application/json
Date: Tue, 21 Jul 2020 14:11:29 GMT
Server: HPE Ezmeral Container Platform 5.1

{"install_state":"installed","original_controllerip":"10.1.0.210","config_data":[],"sysinfo":[],"bds_ha_enabled":"No","primary_ip":"10.1.0.210","primary_hostname":"ip-10-1-0-210.us-west-2.compute.internal","bds_global_version":"5.1","bds_global_buildnumber":"2055","bds_global_sku":"Enterprise-Docker","bds_global_isskudocker":"true","float_range":"172.18.0.0/16","intgateway_ip":"172.18.0.1","routable_network":false,"proxy_present":true,"tenant_network_isolation":true,"bdshared_storage_localfstype":"mapr"}

remove flake8 per-file-ignores

https://github.com/hpe-container-platform-community/hpecp-python-library/blob/master/setup.cfg

Add wait for gateway status

Implement /api/v1/stats

GET /api/v1/stats/ HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c
Authorization: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 886
Content-Type: application/json
Date: Tue, 21 Jul 2020 14:11:29 GMT
Server: HPE Ezmeral Container Platform 5.1

{"result":"Success","objects":{"num_workers_by_purpose":[{"purpose":"primary","total":1,"installed":1},{"purpose":"proxy","total":1,"installed":1},{"purpose":"k8s","total":3,"configured":2,"ready":1}],"num_tenants":3,"num_all_tenants_by_tenant_type":[{"tenant_type":"site_admin","num_tenants":1},{"tenant_type":"k8s_admin","num_tenants":1},{"tenant_type":"docker","num_tenants":1},{"tenant_type":"k8s","num_tenants":0}],"num_k8s_clusters":1,"num_users":3,"num_sessions":151,"num_compute_nodes":1,"num_proxy_nodes":1,"num_active_jobs":0,"num_filesystems":1,"num_active_clusters":0,"num_active_clusters_current_user":0,"num_active_clusters_by_category":[],"num_stopped_clusters":0,"num_stopped_clusters_current_user":0,"num_stopped_clusters_by_category":[],"num_all_vnodes":0,"num_all_vnodes_by_tenant_type":[{"tenant_type":"docker","num_vnodes":0},{"tenant_type":"k8s","num_vnodes":0}]}}

Fix tests were disabled for Python 2.7

I had to disable some tests because they weren't working on Python 2.7 and I couldn't figure out why!

Improve code coverage

Implement k8s supported versions lookup

GET http://localhost:8080/api/v2/k8smanifest

Sample Output:

{"_version":"1.0","supported_versions":["1.14.10","1.15.7","1.16.4","1.17.0","1.18.0"],"version_info":{"1.14.10":{"_version":"1.0","min_upgrade_version":"1.13.0","relnote_url":"https://v1-14.docs.kubernetes.io/docs/setup/release/notes/","hpecsi":"1.14"},"1.15.7":{"_version":"1.0","min_upgrade_version":"1.14.0","relnote_url":"https://v1-15.docs.kubernetes.io/docs/setup/release/notes/","hpecsi":"1.15"},"1.16.4":{"_version":"1.0","min_upgrade_version":"1.15.0","relnote_url":"https://v1-16.docs.kubernetes.io/docs/setup/release/notes/","hpecsi":"1.16"},"1.17.0":{"_version":"1.0","min_upgrade_version":"1.16.0","relnote_url":"https://v1-17.docs.kubernetes.io/docs/setup/release/notes/","hpecsi":"1.17"},"1.18.0":{"_version":"1.0","min_upgrade_version":"1.17.0","relnote_url":"https://kubernetes.io/docs/setup/release/notes/","hpecsi":"1.18"}}}

You can get ‘supported_versions’

Implement create k8s tenant - reuse tenant.py?

POST /api/v1/tenant HTTP/1.1
Host: 127.0.0.1:8080
Content-Length: 595
Accept-Encoding: gzip, deflate
REMOTE_ADDR: 86.138.132.112
Accept: */*
User-Agent: python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-1127.13.1.el7.x86_64
Connection: keep-alive
X-BDS-SESSION: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c
Authorization: /api/v2/session/5a1cec10-f62e-44dc-9692-e3f52a2dcc7c

{"tenant_type_info": {"map_services_to_gateway": true, "specified_namespace_name": "tenant1", "is_namespace_owner": true, "adopt_existing_namespace": false}, "tenant_type": "k8s", "features": {"ml_project": false}, "is_namespace_owner": true, "external_user_groups": [], "quota": {"memory": "", "persistent": "", "gpus": "", "cores": "", "disk": "", "tenant_storage": ""}, "label": {"name": "tenant1", "description": "tenant1"}, "k8s_cluster": "/api/v2/k8scluster/2", "specified_namespace_name": "tenant1", "map_services_to_gateway": true, "adopt_existing_namespace": false, "krb_enc_types": []}HTTP/1.1 201 Created
Access-Control-Allow-Origin: *
Content-Length: 13
Content-Type: text/plain
Date: Tue, 21 Jul 2020 14:12:02 GMT
Location: /api/v1/tenant/5
Server: HPE Ezmeral Container Platform 5.1

201 Created

Request pretty printed:

{
  "tenant_type_info": {
    "map_services_to_gateway": true,
    "specified_namespace_name": "tenant1",
    "is_namespace_owner": true,
    "adopt_existing_namespace": false
  },
  "tenant_type": "k8s",
  "features": {
    "ml_project": false
  },
  "is_namespace_owner": true,
  "external_user_groups": [],
  "quota": {
    "memory": "",
    "persistent": "",
    "gpus": "",
    "cores": "",
    "disk": "",
    "tenant_storage": ""
  },
  "label": {
    "name": "tenant1",
    "description": "tenant1"
  },
  "k8s_cluster": "/api/v2/k8scluster/2",
  "specified_namespace_name": "tenant1",
  "map_services_to_gateway": true,
  "adopt_existing_namespace": false,
  "krb_enc_types": []
}

implement client.user.list()

Implement: client.user.list() (see GatewayController.list() for example)

GET /api/v1/user HTTP/1.1
Host: 127.0.0.1:8080
Accept-Encoding: gzip, deflate
...

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 954
Content-Type: application/json
Date: Mon, 18 May 2020 17:21:48 GMT
Server: BlueData EPIC 5.0

{
  "_links": {
    "self": {
      "href": "/api/v1/user"
    }
  },
  "_embedded": {
    "users": [
      {
        "_links": {
          "self": {
            "href": "/api/v1/user/16"
          }
        },
        "label": {
          "name": "csnow",
          "description": "chris"
        },
        "is_group_added_user": false,
        "is_external": false,
        "is_service_account": false,
        "default_tenant": "",
        "is_siteadmin": false
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/user/5"
          }
        },
        "label": {
          "name": "admin",
          "description": "BlueData Administrator"
        },
        "is_group_added_user": false,
        "is_external": false,
        "is_service_account": false,
        "default_tenant": "/api/v1/tenant/1",
        "is_siteadmin": true
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/user/17"
          }
        },
        "label": {
          "name": "chris",
          "description": ""
        },
        "is_group_added_user": false,
        "is_external": true,
        "is_service_account": false,
        "default_tenant": "",
        "is_siteadmin": false
      },
      {
        "_links": {
          "self": {
            "href": "/api/v1/user/13"
          }
        },
        "label": {
          "name": "demo.user",
          "description": "BlueData Anonymous User"
        },
        "is_group_added_user": false,
        "is_external": false,
        "is_service_account": false,
        "default_tenant": "",
        "is_siteadmin": false
      }
    ]
  }
}

Provide additional parameters for login

{
  "name": "admin",
  "password": "admin",
  "domain": "string",
  "saml_response": "string",
  "tenant_key": "string",
  "tenant": "/api/v1/tenant/2",
  "tenant_name": "Demo Tenant"
}

hpe-container-platform-community / hpecp-python-library Goto Github PK