exoscale / python-exoscale Goto Github PK

View Code? Open in Web Editor NEW

14.0 22.0 7.0 3.74 MB

Python bindings for the Exoscale APIs

Home Page: https://exoscale.github.io/python-exoscale/

License: ISC License

Makefile 3.98% Python 96.02%

python

python-exoscale's Introduction

python-exoscale: Python bindings for Exoscale API

This library allows developers to use the Exoscale cloud platform API with extensive Python bindings. API documentation and usage examples can be found at this address: https://exoscale.github.io/python-exoscale

Development

First create a new virtual environment and run python -m pip install -e .[dev].

You can then run pytest with the following command:

pytest -x -s -vvv

python-exoscale's People

Contributors

Stargazers

Watchers

Forkers

cruncher blatoy astuanax rommellayco aiven mlec1 thomas-chauvet

python-exoscale's Issues

ssh-key option not working when creating a nodepool

Hello
I am trying to create a instance_pool and passing my ssh key to all the instances.
I've created a ssh key on my host machine and also added the key in my exoscale account
I've written:

exo = exoscale.Exoscale(config_file=str(Path.home())+"/.config/exoscale/exoscale.toml")    
key = exo.compute.get_ssh_key("My SSH key") 
zone = exo.compute.get_zone("some_zone")
print(key)
# returns: SSHKey(name='My SSH key', fingerprint='correct fingerprint')
instance_pool = exo.compute.create_instance_pool(    
    zone = zone,    
    name = "some_name",    
    size = 10,    
    instance_type=exo.compute.get_instance_type("medium"),    
    instance_template=exo.compute.get_instance_template(    
        zone,    
        "54d8e676-d46a-4559-87e7-e0feed79e770"),    
    instance_ssh_key=key,     
    description="some_description",    
    instance_user_data="#cloud-config")

The error i receive is:

{
   "errors":[
      {
         "in":[
            "ssh-key"
         ],
         "path":"ssh-key",
         "message":"should be a Map",
         "description":"SSH key"
      }
   ],
   "message":"Invalid value in ssh-key (description: `SSH key`) - should be a Map\n"
}

which tells me, that your python-exoscale library is not having any problem with the datatype is specified but the underlying openapi-v2.exoscale.com - api
So i guess there must be some problem when calling the api in your python-exoscale lib?

Without specifying an ssh-key the instance-pool generation is working fine.

Some help or fixes would be very nice.

Greets HD

Feature request dbaas

Hello

I wanted to ask if it is planned to extend the API with DBAAS so one is able to spawn and destroy databases ?

Use better versioning in the future

Dear Exoscale Team,

It would be great if you could stick to proper versioning practices to ensure clarity and consistency. Specifically, use major updates for significant changes and improvements, and use minor updates for smaller, incremental adjustments.

I noticed that in the v0.8.0 update, there were some substantial changes included in what is typically considered a minor update, but totally deprecates the old client. I think it is a breaking change, and it should have been a major version update. It is especially important for other project which depends on these dependencies and have automated updates with Renovate or Dependabot for example. It plays also a role in prioritizing and managing our development efforts effectively.

I know the version 0.8.0 got released a few months ago already, but as I was working on the documentation, I noticed that a version 0.9.0 is planned. I wanted to raise this issue in case the version 0.9.0 will contain other breaking changes.

I hope you will understand my point here.

Crash on Windows

The Python library crashes on Windows because __init__.py relies on the HOME environment variable to be set in like 21. Instead, pathlib should be used:

from pathlib import Path
_DEFAULT_CONFIG_FILE = os.path.join(str(Path.home()), ".exoscale", "config.toml")

Getting instances by zone returns nothing

Hi everybody,

i have just started using the python wrapper for the exoscale api but already came across an issue.
I am not able to retrieve all instances by zone, as the generator that is returned is empty.

zones = exo.compute.list_zones()

for zone in zones:
    instances = exo.compute.list_instances(zone=zone)
    for x in instances:
        print(x)

In my case instances is just an empty generator.
Any idea what is happening here and where the problem is?

exoscale version: 0.8.0
python version: 3.10

Tests depending on time equality are flaky

Fedora package building fails because of flaky test:

________________ TestCompute.test_create_network_load_balancer _________________
self = <tests.test_compute.TestCompute object at 0x7f95a435f7d0>
exo = <[AttributeError("'ExoscaleMock' object has no attribute 'api_key'") raised in repr()] ExoscaleMock object at 0x7f95a3398ef0>
zone = {'id': '870666c8-f21f-4f1e-ae6f-bdd09d4a9a29', 'name': 'ch-gva-2'}
nlb = <function nlb.<locals>._nlb at 0x7f95a37f45e0>
    def test_create_network_load_balancer(self, exo, zone, nlb):
        zone = zone()
        nlb_name = _random_str()
        nlb_description = _random_str()
        nlb_creation_date = datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
        operation_id = _random_uuid()
    
        expected = nlb(
            zone=zone["name"],
            name=nlb_name,
            description=nlb_description,
            created=nlb_creation_date,
        )
    
        def _assert_request(request, context):
            body = json.loads(request.body)
            assert body["name"] == nlb_name
            assert body["description"] == nlb_description
    
            context.status_code = 200
            context.headers["Content-Type"] = "application/json"
            return {
                "id": operation_id,
                "state": "success",
                "reference": {"id": expected["id"]},
            }
    
        exo.mock_post(zone["name"], "load-balancer", _assert_request)
        exo.mock_get_operation(zone["name"], operation_id, expected["id"])
        exo.mock_get_v2(
            zone["name"], "load-balancer/" + expected["id"], expected
        )
    
        actual = exo.compute.create_network_load_balancer(
            zone=Zone._from_cs(zone),
            name=nlb_name,
            description=nlb_description,
        )
        assert actual.zone.name == zone["name"]
        assert actual.name == expected["name"]
        assert actual.description == expected["description"]
>       assert (
            actual.creation_date.strftime("%Y-%m-%dT%H:%M:%SZ")
            == nlb_creation_date
        )
E       AssertionError: assert '2023-10-01T16:34:47Z' == '2023-10-01T16:34:46Z'
E         - 2023-10-01T16:34:46Z
E         ?                   ^
E         + 2023-10-01T16:34:47Z
E         ?                   ^
tests/test_compute.py:864: AssertionError

I suggest to compare times using subtraction and comparing it to small timedelta.

Error while listing files

I'm using the latest release of this package (0.3.0) and when I try to list the file in my bucket, I get an exception.

My code

path = Path.home() / ".aws/exoscale"
exo = Exoscale(config_file=str(path))

bucket = exo.storage.get_bucket(bucket_name)

for item in bucket.list_files(prefix="stuff/"):
    print(item)

The exception

Traceback (most recent call last):
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/httpsession.py", line 254, in send
    urllib_response = conn.urlopen(
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connection.py", line 309, in connect
    conn = self._new_conn()
  File "/path/to/virtualenv/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x1104ac370>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/virtualenv/lib/python3.8/site-packages/exoscale/api/storage.py", line 570, in list_files
    for page in pages:
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/paginate.py", line 332, in _make_request
    return self._method(**current_kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/client.py", line 337, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/client.py", line 642, in _make_api_call
    http, parsed_response = self._make_request(
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/client.py", line 662, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/endpoint.py", line 136, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/endpoint.py", line 253, in _needs_retry
    responses = self._event_emitter.emit(
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 250, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 277, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 316, in __call__
    checker_response = checker(attempt_number, response,
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 222, in __call__
    return self._check_caught_exception(
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/endpoint.py", line 200, in _do_get_response
    http_response = self._send(request)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/endpoint.py", line 269, in _send
    return self.http_session.send(request)
  File "/path/to/virtualenv/lib/python3.8/site-packages/botocore/httpsession.py", line 283, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://s3.ch-dk-2.amazonaws.com/prometheus-bl?list-type=2&prefix=stuff%2F&max-keys=100&encoding-type=url"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "script.py", line 21, in <module>
    for item in bucket.list_files(prefix="stuff/"):
  File "/path/to/virtualenv/lib/python3.8/site-packages/exoscale/api/storage.py", line 574, in list_files
    raise APIException(e)
exoscale.api.APIException: Could not connect to the endpoint URL: "https://s3.ch-dk-2.amazonaws.com/prometheus-bl?list-type=2&prefix=stuff%2F&max-keys=100&encoding-type=url"

The observation

The url is strange.

It's a mixture of a Amazon URL but with the Exoscale zone: https://s3.ch-dk-2.amazonaws.com/prometheus-bl

What am I doing wrong here?

Handle multiple path parameters

Hello,

I noticed that the v2 part of the library does not handle multiple path parameters.

For instance when using:

c = Client(api_key, api_secret, url="https://api-ch-gva-2.exoscale.com/v2")
c.update_dbaas_pg_connection_pool(
            connection_pool_name=connection_pool_name,
            service_name=dbaas_service_name,
            database_name=connection_pool_db,
            mode=connection_pool_mode,
            size=connection_pool_number,
        )

It will raise a KeyError because only the first parameter (service_name) will be given and not the two parameters needed for this route (service_name and connection_pool_name).

I've opened a PR to fix this: #57

Creating compute instance with security group generates error messags

Hi everyone,

I am getting multiple error messages when I try to create an instance with a security group. I created the security group beforehand on the web interface.

from exoscale.api.v2 import Client

client_key = '...'
client_secret = '...'
zone = 'de-fra-1'

client = Client(client_key, client_secret, zone=zone)

name = 'testname'
standard_micro_instance_id = '71004023-bb72-4a97-b1e9-bc66dfce9470'
instance_type = client.get_instance_type(id=standard_micro_instance_id)

ubuntu_22_04_image_id = '67756dc3-a1de-4abd-abcf-843ef9223bb6'
template = client.get_template(id=ubuntu_22_04_image_id)

security_group_id = '59378c27-c232-4079-9e18-dd68b2c75011'
security_group = client.get_security_group(id=security_group_id)

ssh_key = {'name': 'TESTKEYPAIR'}

server = client.create_instance(name=name, ssh_key=ssh_key, instance_type=instance_type, template=template, public_ip_assignment='inet4', disk_size=10, security_groups=[security_group])

print(security_group)
print(server)

Output:

Security group: {'id': '59378c27-c232-4079-9e18-dd68b2c75011', 'name': 'test', 'description': '', 'visibility': 'private', 'rules': [{'id': '845f79b6-1e5c-4613-a59d-f3a36cbb16cc', 'protocol': 'tcp', 'flow-direction': 'ingress', 'description': 'SSH', 'network': '0.0.0.0/0', 'start-port': 22, 'end-port': 22}]}

Server: {'errors': [{'in': [2, 'exoscale.entity.instance/security-groups', 0, 'exoscale.entity.security-group/visibility'], 'path': 'security-groups[0].visibility', 'message': 'should be one of :private, :public'}, {'in': [2, 'exoscale.entity.instance/security-groups', 0, 'exoscale.entity.security-group/rules', 0], 'path': 'security-groups[0].rules[0]', 'message': "null OR missing keys 'flow-direction', 'protocol'"}, {'in': [2, 'exoscale.entity.instance/security-groups'], 'path': 'security-groups', 'message': 'should be nil'}], 'message': "Invalid value in security-groups[0].visibility - should be one of :private, :public\nInvalid value in security-groups[0].rules[0] - null OR missing keys 'flow-direction', 'protocol'\nInvalid value in security-groups - should be nil\n"}

Any idea what the problem is?

user_data parameter in create_instance returns "bad request"

I am unable to create a server and provide user_data in cloud-init format.
Using API v2, version: 0.8, and the function create_instance

Cloud init script and code to test:

script = """#!/bin/bash
echo "Test"
"""

ubuntu = c.create_instance(
public_ip_assignment="inet4",
security_groups=[{"id": SECURITY_GROUP_ID}],
name="demo2",
instance_type={"id": INSTANCE_TYPE_ID},
template={"id": TEMPLATE_ID},
ssh_keys=[{"name": SSH_KEY_NAME}],
disk_size=50,
user_data=script)

{'message': 'bad request'}

When removing the user_data parameter the function works fine, and the result is a new server.
Adding the user_data back and it breaks.

Tested with several formats listed here: https://cloudbase-init.readthedocs.io/en/latest/userdata.html#bash

script = """#!/usr/bin/env python

print("test")
"""
Same problem.

Cloud config:
script = """#cloud-config
runcmd:

'cd'
['echo', '1']
"""

Same issue.

It looks like that to me

Add a storage.Bucket/BucketFile "url" attribute

It'd be convenient to expose a url attribute to the storage.Bucket and storage.BucketFile objects, returning the usable HTTP URL of a SOS bucket or a file stored in it.

Example:

bucket = exoscale.storage.create_bucket("my-bucket", zone="ch-gva-2")
print(bucket.url) # → "https://sos-ch-gva-2.exo.io/my-bucket/"

a_file = bucket.put_file("/path/to/a_file.txt")
print(a_file.url) # → "https://sos-ch-gva-2.exo.io/my-bucket/a_file.txt"