Missing the ability to see a list of available regions in which to work. It would be nice to see all the places in which you could do some work.

I'm unable to get pyrax to pull a list of our existing cloud servers. Below is the code that I'm using.

import pyrax
import os
creds_file = os.path.expanduser("~/.rackspace_cloud_credentials")
pyrax.set_credential_file(creds_file)
cs = pyrax.cloudservers
cs.servers.list()
[]

Thanks

The variable default_region is defined as global. On line 192 of 'init.py' the variable is assigned as self.default_region = region. This causes the set_credentials method to fail with 'global name "self" is not defined.'

-sam chapler

Setting region to 'LON' in ~/.pyrax.cfg will set pyrax.identity.auth_endpoint correctly. However, I believe this should be something that can be parsed from the file in pyrax.set_credential_file(). The auth endpoint seems like a natural fit with username and API key.

Is there a better way that doesn't require a user-wide setting like ~/.pyrax.cfg? The alternatives that I see don't feel as convenient as pyrax.set_credential_file().

When using a callback with wait_until on a new server object, the adminPass attribute is destroyed.

In pyrax/utils.py, lines 282/283:

            # For servers:
            obj = obj.manager.get(obj.id)

This appears to destroy adminPass because it re-initializes the object from the API instead of refreshing it. Given that wait_until is used to detect changes to the object, should a non-destructive refresh be used here?

In a quick test, changing line 283 to the following refreshes the server from the API successfully, and the object passed to the callback function includes the adminPass attribute as it should:

            obj.get()

Some quick test code that shows the refreshed server status (to prove a successful refresh), and whether or not the adminPass attribute exists in the callback:

#!/usr/bin/env python
import os
import pyrax

# credentials file in INI format:
#    [rackspace_cloud]
#    username = cloudusername
#    api_key = 0x0x0x0x0x0x0x0x0x0x
raxcreds = os.path.expanduser("~/.rackspace_cloud_credentials")

# set the credentials file in pyrax
pyrax.set_credential_file(raxcreds)

def server_built(server):
    print "Status:", server.status
    if hasattr(server, 'adminPass'):
        print "root password:", server.adminPass
    else:
        print "root password was not passed to this function"

server = pyrax.cloudservers.servers.create("testbuild", "a10eacf7-ac15-4225-b533-5744f1fe47c1", 2)

pyrax.utils.wait_until(server, "status", ['ACTIVE', 'ERROR'], callback=server_built, interval=15, attempts=0)

Of note, I'm not sure how this affects the try/except test being used around this get, and there may be a reason for the current method that I'm not aware of--please enlighten me if this behavior is intentional.

The new sync_folder_to_container looks awesome, but I'm getting a type error when I attempt to call it.

My script: https://github.com/Linuturk/www.onitato.com/blob/master/cf_pyrax.py

Output after the various credentials are supplied at the top of the file.

Syncing all objects to www.onitato.com container from folder /var/lib/jenkins/workspace/www.onitato.com/output/.
Traceback (most recent call last):
  File "cf_pyrax.py", line 25, in <module>
    upload_key, total_bytes = cf_ord.sync_folder_to_container(folder, container, delete=True, include_hidden=True, ignore_timestamps=True)
  File "/usr/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 516, in sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/usr/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/usr/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 542, in _sync_folder_to_container
    obj = cont.get_object(fullname)
  File "/usr/lib/python2.6/site-packages/pyrax/cf_wrapper/container.py", line 85, in get_object
    name = name.decode(pyrax.encoding)
TypeError: decode() argument 1 must be string, not None

The current design of the utils.wait_until() method only returns the updated object if the object's attribute reaches the desired value; if it fails, None is returned.

This should be changed so that the updated object is returned in either case, so that the code can examine it for any information that might be useful. Two options exist for indicating success: return a tuple of (object, success), or simply return the object and have the calling program examine the attribute to determine success.

There's an issue in pyrax where, upon setting the condition of a cloud load balancer node to "DRAINING", any future attempts to change the condition via pyrax are ignored.

The issue stems from the Node class implementation, where it checks to ensure the existing status of a new node is "ENABLED" or "DISABLED"

cloudloadbalancers.py: line 917
if condition.upper() not in ("ENABLED", "DISABLED"):
raise exc.InvalidNodeCondition("Nodes must be created in either 'ENABLED' or 'DISABLED' "
"condition; '%s' is not valid." % condition)

This is the expected behavior when creating a new node, but it causes problems when trying to update the status of an existing node. I believe it's because the same constructor code is called when iterating through the existing load balancer nodes:

cloudloadbalancers.py: line 71
def _add_details(self, info):
"""Override the base behavior to add Nodes, VirtualIPs, etc."""
for (key, val) in info.iteritems():
if key == "nodes":
val = [Node(parent=self, **nd) for nd in val]

Steps to reproduce:

Python 2.7 (r27:82508, Jul 3 2010, 20:17:05)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import pyrax
pyrax.set_credentials("myser", "myapikey")
clb = pyrax.cloud_loadbalancers
lb = clb.get(92155)
lb.nodes[0].condition = "DRAINING"
lb.nodes[0].update()
lb.nodes[0]
Node type=PRIMARY, condition=DRAINING, id=206517, address=10.181.24.101, port=80 weight=1
lb = clb.get(92155)
Traceback (most recent call last):
File "", line 1, in
File "pyrax/client.py", line 132, in get
return self._manager.get(item)
File "pyrax/manager.py", line 83, in get
return self._get(uri)
File "pyrax/manager.py", line 134, in _get
return self.resource_class(self, body[self.response_key], loaded=True)
File "pyrax/cloudloadbalancers.py", line 58, in init
super(CloudLoadBalancer, self).init(_args, *_kwargs)
File "pyrax/resource.py", line 43, in init
self._add_details(info)
File "pyrax/cloudloadbalancers.py", line 75, in _add_details
val = [Node(parent=self, **nd) for nd in val]
File "pyrax/cloudloadbalancers.py", line 919, in init
"condition; '%s' is not valid." % condition)
pyrax.exceptions.InvalidNodeCondition: Nodes must be created in either 'ENABLED' or 'DISABLED' condition; 'DRAINING' is not valid.

There are probably a half-dozen ways to address this, and I'd just submit a patch if I knew which way you want to proceed. You could subclass Node override it's behavior when iterating over existing nodes, change the logic, etc...

Calling pyrax.set_setting("region", "ORD") should then change the active region to ORD, but the _safe_region() function in pyrax/init.py only checks for default_region.

Swift enables you to set a time on an object for its expiration; in other words, set a time after which it will automatically be deleted.

This is able to be set via pyrax, but once set, there is no way currently to see which files are going to be deleted, and when. It is possible to hack the headers of HEAD requests on an object to get this information, but python-swiftclient doesn't expose it directly.

In the previous API I used, you could search for servers based on private_ip which you could get from the node. In pyrax you can still get the private_ip from the node but I do not see any way to search for the original server using that.

In [18]: fdns.dns.findall
Out[18]: bound method CloudDNSClient.findall of pyrax.clouddns.CloudDNSClient object at 0x3737290

In [19]: fdns.dns.findall()
Out[19]: [CloudDNSDomain accountId=740110, emailAddress=[email protected], id=3688916, name=rs.tjnii.com]

In [20]: fdns.dns.find
Out[20]: bound method CloudDNSClient.find of pyrax.clouddns.CloudDNSClient object at 0x3737290

In [21]: fdns.dns.find(id="3688916")
]Snip Trace]
NotFound: No CloudDNSDomain matching: {'id': '3688916'}. (HTTP 404)

In [22]: fdns.dns.find(name="rs.tjnii.com")
[Snip Trace]
NotFound: Not found (HTTP 404)

find(name=name) used to work in the 1.3.x version I was using previously.

I just read through the pull requests for #64 and #66, #67. I am not a Pythonista, so I am loathe to attempt this. (I do wish I was though!)

The X-DELETE-AFTER header is extremely useful to me on the file storage side for automagically deleting database backups after a certain amount of time. If we were using the CDN they would be great for cache expiry. Give this, I am likely missing the code that sets them or the doc that explains it.

The other side of this is that I am not sure this header is valid any longer. Here is the doc:
http://docs.rackspace.com/files/api/v1/cf-devguide/content/Expiring_Objects-e1e3228.html). It does appear that swift supports x-delete-after as well.

When attaching a CBS volume to a Cloud Server, if successful, the return code is "True". In the case of detaching the CBS volume, the return code is "None". Per email w/Ed, the thought is to change attach to return "None" unless there is an exception.

I get the following error trying to authenticate in pyrax after upgrading to python-novaclient 2.13.0::

Python 2.7.3 (default, Apr 20 2012, 22:39:59)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyrax
pyrax.set_credentials("redacted","redacted")
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pyrax/init.py", line 196, in set_credentials
connect_to_services(region=region)
File "/usr/local/lib/python2.7/dist-packages/pyrax/init.py", line 296, in connect_to_services
cloudservers = connect_to_cloudservers(region=region)
File "/usr/local/lib/python2.7/dist-packages/pyrax/init.py", line 175, in _wrapped
return fnc(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyrax/init.py", line 343, in connect_to_cloudservers
http_log_debug=_http_debug)
File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/client.py", line 148, in init
cacert=cacert)
File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 53, in init
raise exceptions.AuthSystemNotFound(auth_system)
novaclient.exceptions.AuthSystemNotFound: AuthSystemNotFound: 'rackspace'

I see the following commit in 2.13.0 - which if I had to guess sounds like the culprit:

openstack/python-novaclient@d195c6a.

Not sure if this is ultimately a bug in novaclient or a feature.

The CloudDNS "check for timeout" code, checks the status of a request too fast, which causes API rate limiting to be imposed and the pyrax call to fail - even though the requested operation eventually succeeds.

I believe the key lies in clouddns.py, within the _async_call() function, which contains this code:

while (ret_body["status"] == "RUNNING") and not timed_out:
            _resp, ret_body = self.api.method_get(massagedURL)
            if self._timeout:
                timed_out = ((time.time() - start) > self._timeout)

That "while loop" will fetch the status URL as fast as possible.

I've found that the status rate-limit on my account is 5/sec. When this code is executed on a server with sufficiently fast network connectivity to the API endpoints, this rate-limit is encountered every time.

May I suggest that a brief time delay be added into the "while loop"? Perhaps 0.5 seconds? Perhaps this delay could also be exposed as a configurable option?

Note the varying return value from type(instance.volume)

[EDIT]Greater than and less than characters removed as it caused misformatting.[/EDIT]

instance=cdb.find(name="TestDB8")
type(instance)
class 'pyrax.clouddatabases.CloudDatabaseInstance'
type(instance.volume)
class 'pyrax.clouddatabases.CloudDatabaseVolume'
instance.get()
type(instance)
class 'pyrax.clouddatabases.CloudDatabaseInstance'
type(instance.volume)
type 'dict'

Is this expected?
(I haven't gotten a chance to dig into the code for this yet)

testing, please ignore...

Calling the above mentioned function returns an invalid url, mainly the problem seems to be in cf_wrapper/client.py at line 210 as the returned address is clouddrive.comv1 instead of clouddrive.com/v1 because of the stripping.
for example:
https://storage101.ord1.clouddrive.comv1/MossoCloudFS_aa58fa55-4959-475d-98c3-d86ae7d56c8a/pollitapp/userpolls/51812741109eab3e6477db21t1367420789388-0-0?temp_url_sig=3ca6f4a2c86b2192c87041f9b4978a507fc6d054&temp_url_expires=1367423129

it also seems that manually editing the url to .com/v1 ends in a 401 not authorized

I couldn't find one. Is there a method to set account metadata?

Hi,

I'm trying to use django-cumulus with pyrax and novaclient without success.

The error raised is:

IdentityClassNotDefined at ...
No identity class has been defined for the current environment.

In the issue #79, @mlew comments he need to set the identity_type in code. I did it and it worked, but I don't want to change cumulus or my code to include it.

In fact, I read the pyrax code and note that the set method for settings changes the identity_class when the identity_type is changed (pyrax/init.py:166) but for the default environment, this settings param would never be set and I will never fall in the default rackspace identity_type, consequently, the rax_identity.RaxIdentity.

I try to set it through config files without success but I must admit, I didn't try enough. I want to use OS env variables as they are easier to set in my uWSGI deployment.

I can't set the identity_class through OS env variables and if I set identity_type, it will work as a default, it will never pass through the set method and this way, will never update the identity_class.

In fact, there is a default_identity_type/region/encoding but they aren't used until read_config is called, IF I set a config file (something I want to avoid).

Well, this is what I could dig up first time :)

I don't feel secure to fix it myself until work a little more with the lib, this is the first time with pyrax. Anyway, anything I could help just tell me!

I'm experiencing the same issues as:
https://github.com/rackspace/python-cloudfiles/issues/34

Here are 2 tracebacks. The first failed on making a container public. The 2nd failed on creating a new container.

Traceback (most recent call last):
File "./make_new_containers.py", line 82, in
main()
File "./make_new_containers.py", line 47, in main
cont.make_public(ttl=86400)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/container.py", line 183, in make_public
return self.client.make_container_public(self, ttl)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 630, in make_container_public
return self._cdn_set_access(container, ttl, True)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 650, in _cdn_set_access
response = self.connection.cdn_request(mthd, [ct.name], hdrs=hdrs)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 800, in cdn_request
response = retry_request()
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 794, in retry_request
return self.cdn_connection.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.7/socket.py", line 430, in readline
data = recv(1)
File "/usr/lib/python2.7/ssl.py", line 241, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 160, in read
return self._sslobj.read(len)
ssl.SSLError: The read operation timed out

Traceback (most recent call last):
File "./make_new_containers.py", line 79, in
main()
File "./make_new_containers.py", line 44, in main
cont = cf.create_container(cont_name)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 32, in _wrapped
return fnc(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 263, in create_container
return self.get_container(name)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 32, in _wrapped
return fnc(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 556, in get_container
total_bytes=hdrs.get("x-container-bytes-used"))
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/container.py", line 19, in init
self._fetch_cdn_data()
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/container.py", line 25, in _fetch_cdn_data
response = self.client.connection.cdn_request("HEAD", [self.name])
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 800, in cdn_request
response = retry_request()
File "/usr/local/lib/python2.7/dist-packages/pyrax/cf_wrapper/client.py", line 794, in retry_request
return self.cdn_connection.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.7/socket.py", line 430, in readline
data = recv(1)
File "/usr/lib/python2.7/ssl.py", line 241, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 160, in read
return self._sslobj.read(len)
ssl.SSLError: The read operation timed out

This works fine on Linux systems where os.sep is a forward slash, but when run on a windows system, os.sep is , which causes it to return temp_url's like the following:

https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_11111111-2222-3333-4444-555555555555\spam\eggs?temp_url_sig=12345&temp_url_expires=12345

Notice the \ where the account ID, container name and object name are concatenated together.

A recommended solution would be altering line 209 of ./pyrax/cf_wrapper/client.py from:

pth = os.path.join(conn_url[v1pos:], cname, oname)

to something more like:

pth = '/'.join([conn_url[v1pos:], cname, oname])

Not using os.path.join here would actually be preferable as forward slashes are what is appropriate for URL's, and the URL path go's into the generation of the hmac for the temp url.

The design of the config file is that if there isn't a section labeled 'default' or 'settings', the first defined section becomes 'default'. However, the change to create a defined empty 'default' section in the Settings class in 3705cda broke this behavior, since now there is always a section with that name. Thanks to Anne Gentle for helping find this one.

Hello,I have a threaded application, using queues, to read and write files from cloudfiles and I get weird time gaps at the end as the tasks run out. These gaps are anywhere from a couple of minutes to an hour. I am doing other processing that might be exacerbating the times a bit but I did a base test that only accesses Rackspace and found much the same results. Below is a simplified version of our code:

pyrax.set_credentials(USERNAME, API_KEY)
cfconn = pyrax.connect_to_cloudfiles(region=REGION, public=USE_PUBLIC)
container = cfconn.create_container(HISTORY_CONTAINER)

class RackspaceReader(threading.Thread):

    def __init__(self, max_tries, processing_queue):
        threading.Thread.__init__(self)
        self.max_tries = max_tries
        self.processing_queue = processing_queue

    def read(self, work):
        my_id    = work["item_id"]

        contents = None
        while tries < self.max_tries:
            try:
                logger.info("Thread %s: Checking existing logs from rackspace." % self.name)
                remote_file = container.get_object("%s\\%s.json" % (my_id, variable))
                logger.info("Thread %s: Received existing logs sized %d bytes from rackspace." % (self.name, remote_file.total_bytes))
                contents = remote_file.get()
                logger.info("Thread %s: Read contents from files" % (self.name))
            except:
                tries += 1
                logger.error("Thread %s: Error reading logs from rackspace. Retrying attempt %d of %d" % (self.name, tries, self.max_tries))

        return contents


    def run(self):
        while True:
            try:
                work = self.processing_queue.get(True, 1)
                self.read(work)
                self.processing_queue.task_done()
            except Queue.Empty:
                continue


if __name__ == "__main__":

    processing_queue = Queue.Queue()

    processing_pool = []
    for i in range(opts['reader_batches']):
        processing_thread = RackspaceReader(max_tries, processing_queue)
        processing_pool.append(processing_thread)
        processing_thread.setDaemon(True)
        processing_thread.start()

    for item in items:
        processing_queue.put({"item_id":     item_id})

    # As tasks become less, there is a larger and larger time gap for those still reading.
    processing_queue.join()

The main pyrax_doc.md doc was updated to explain the requirement to specify identity_type beginning with version 1.4.0, but this is only mentioned in the section on configuration. It would be helpful to repeat this information in the section on authentication, since it is in this step that users will generally encounter the IdentityClassNotDefined exception.

Attempting to use add_ptr_records():

File "/home/tom/Python/API_Challenge/fluffyWrappers.py", line 488, in addServerRecords
retVal["PTR"] = self.dns.add_ptr_records(server, [request, request6])
File "/usr/local/lib/python2.7/dist-packages/pyrax/clouddns.py", line 1204, in add_ptr_records
return self._manager.add_ptr_records(device, records)
File "/usr/local/lib/python2.7/dist-packages/pyrax/clouddns.py", line 846, in add_ptr_records
device_type = self._resolve_device_type(device)
File "/usr/local/lib/python2.7/dist-packages/pyrax/clouddns.py", line 813, in _resolve_device_type
from tests.unit import fakes
ImportError: No module named tests.unit

Looking at the code:
def _resolve_device_type(self, device):
...
from tests.unit import fakes

$ find /usr/local/lib/python2.7/dist-packages/pyrax -iname "test"
[No files]

Hello,
I created a script to copy some files from a container to another container and it's taking 7 minutes to copy each file.
I checked the Activity Monitor to see what was going on and found out that the network traffic would stay around 500KB/sec (download) when trying to copy each file. Each file is about 300KB big and there are nearly 800000 files in the source container.

Since I'm quite new to python (and pyrax), either I'm doing something wrong or there's an issue when dealing with big containers.

#!/usr/bin/env python

from __future__ import print_function
import csv, pyrax
import pyrax.exceptions as exc
from urlparse import urlsplit

pyrax.set_setting("identity_type", "rackspace")
print ()

try:
    pyrax.set_credentials("username", "API Key")
except exc.AuthenticationFailed:
    print ('Authentication error. Please check login details.')
    quit()

if pyrax.identity.authenticated is True:
    print ('Authenticated successfully. ', end='')

cf = pyrax.cloudfiles
source_cont = 'SOURCE'
destination_cont = 'DESTINATION'
projects_file = 'files.csv'

try:
    cf.get_container(source_cont)
    print ('Source container %s exists.\n' % source_cont)
except exc.NoSuchContainer:
    print ('Source container %s not found.\n' % source_cont)
    quit()
    
with open(projects_file, 'rU') as csvfile:
    projects = csv.reader(csvfile, delimiter=',')
    
    for row in projects:
        asset1_path = urlsplit(row[0])[2][1:]
        asset2_path = urlsplit(row[1])[2][1:]
        
        print ('Copying ' + source_cont + '/' + asset1_path + ' to /' + destination_cont + '... ')
            
        try:
            cf.copy_object(source_cont, asset1_path, destination_cont)
            print ('DONE\n')
                            
        except exc.NoSuchObject:
              print ('FAILED.\n')
        
        except exc.FileNotFound:
            print ('FAILED.\n')

        print ('Copying ' + source_cont + '/' + asset2_path + ' to /' + destination_cont + '... ')
                
        try:
            cf.copy_object(source_cont, asset2_path, destination_cont)
            print ('DONE\n')
            
        except exc.NoSuchObject:
            print ('FAILED.\n')
        
        except exc.FileNotFound:
            print ('FAILED.\n')

I've been attempting to define ORD as the region for my Cloud Files connection, and I've been unsuccessful in this connection.

pyrax.set_default_region("ORD") before connecting still had the eventual folder upload go to DFW instead of ORD. There is documentation pertaining to defining a region when creating and managing servers, but no such mention of how to define the region for Cloud Files.

https://github.com/rackspace/pyrax/blob/master/docs/pyrax_doc.md#working-with-multiple-regions

Was just working on a simple script to import a bunch of zone files into cloud dns with pyrax and got this ugly error:

Traceback (most recent call last):
  File "./add-zonefiles.py", line 37, in <module>
    dns.import_domain(clean_zonedata)
  File "/usr/local/lib/python2.7/site-packages/pyrax/clouddns.py", line 1058, in import_domain
    return self._manager.import_domain(domain_data)
  File "/usr/local/lib/python2.7/site-packages/pyrax/clouddns.py", line 573, in import_domain
    error_class=exc.DomainCreationFailed)
  File "/usr/local/lib/python2.7/site-packages/pyrax/clouddns.py", line 413, in _async_call
    ret = _resp, ret_body["response"]
KeyError: 'response'

I don't want to reveal my customer's data, but here are the keys that were in that response object, which I retrieved by debugging just before the line that caused this error:

In [6]: ret_body.keys()
Out[6]: [u'status', u'requestUrl', u'request', u'jobId', u'verb', u'callbackUrl']

The import operation I was doing did work, its just that pyrax didn't handle the response from the API right.

prettytable is required in the pyrax/utils.py but is not specified.

The import of json could fail causing it to try and import simplejson in pyrax/client.py.

Keyring would be nice to have, but does not seem to be an actual requirement (all imports seem to be optional).

I would like to see prettytable and possibly simplejson added to the setup.py requirements. Keyring should probably be removed unless it is actually a requirement as well.

A recent change to the Cloud DB API in the flavor name and what they present via the API for flavor has broken the pyrax ClouddatabasesFlavor object.

print inst.flavor.name
Traceback (most recent call last):
File "", line 1, in
File "/Library/Python/2.7/site-packages/pyrax/resource.py", line 84, in getattr
"'%s'." % (self.class, key))
AttributeError: '<class 'pyrax.clouddatabases.CloudDatabaseFlavor'>' object has no attribute 'name'.

print inst.flavor.ram
Traceback (most recent call last):
File "", line 1, in
File "/Library/Python/2.7/site-packages/pyrax/resource.py", line 84, in getattr
"'%s'." % (self.class, key))
AttributeError: '<class 'pyrax.clouddatabases.CloudDatabaseFlavor'>' object has no attribute 'ram'.

print inst.flavor

Trying to connect to LON region I received time expiration from the auth server in the following format:

2013-01-27T05:02:05.000Z

which seems to not work well with the API_DATE_PATTERN regex (rax_identity.py), because of the "Z" in the end instead of a "-+XX:XX"

I'm testing to make sure pyrax works with SYD.

Figured I'd try to use 'smoketest.py' but it fails when starting to build the servers since it's not checking for the minram & mindisk requirements of each image:

Thoughts on getting that in there? I'd love to say it works perfectly with SYD and to use pyrax for more broad testing.

Snippet:

$ python smoketest.py --regions SYD --env default 

[...snip...]

Creating server...
Cleaning up...
Traceback (most recent call last):
  File "smoketest.py", line 340, in <module>
    smoke_tester.run_tests()
  File "smoketest.py", line 69, in run_tests
    self.cs_create_server()
  File "smoketest.py", line 152, in cs_create_server
    img.id, flavor.id)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/v1_1/servers.py", line 600, in create
    **boot_kwargs)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/v1_1/base.py", line 163, in _boot
    return_raw=return_raw, **kwargs)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/base.py", line 145, in _create
    _resp, body = self.api.client.post(url, body=body)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/client.py", line 233, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/client.py", line 217, in _cs_request
    **kwargs)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/client.py", line 199, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/opt/boxen/homebrew/lib/python2.7/site-packages/novaclient/client.py", line 193, in request
    raise exceptions.from_response(resp, body, url, method)
novaclient.exceptions.BadRequest: Instance type's memory is too small for requested image. (HTTP 400) (Request-ID: req-a33b5b6a-9b92-4b05-8fd2-b97cd55a090f)

In trying to write a script to deploy an image from a deleted server, I discovered that servers.list() does not include those images, but I can view them from the web interface.

On the morning of Saturday, March 23, 2013, the 'rackspace' organization on GitHub will be reorganized. All repos will be moved to the new 'rackerlabs' organization, except for those that are designed to be used by Rackspace customers and which are fully supported.

Please update any links to this repo to reflect the new location within GitHub. For example, if the link to your repo is 'https://github.com/rackspace/foo', you need to change it to 'https://github.com/rackerlabs/foo'.

The documentation at https://github.com/rackspace/pyrax/blob/master/docs/pyrax_doc.md#configfile says we can use pyrax.set_setting("identity_type", "rackspace") immediately after import pyrax inlieu of a pyrax configuration file or an environment variable. When I try this, I get a TypeError.

(bonner)aluminum13:bonner$ python
Python 2.7.3 (default, Oct 22 2012, 06:12:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import pyrax
pyrax.set_setting("identity_type", "rackspace")
Traceback (most recent call last):
File "", line 1, in
File "/Users/vwatkins/src/bonner/lib/python2.7/site-packages/pyrax/init.py", line 298, in set_setting
return settings.get(key, val, env=env)
TypeError: get() got multiple values for keyword argument 'env'

This Abbreviated Code:

for server in cs.servers.list():
for volume in cbs.list():
if ((server.id == volume.display_name) and (volume.volume_type == "SATA")):
print "SERVER: ", server
print "SERVER ID: ", server.id
print "VOLUME ID: ", volume.id
print "VOLUME NAME: ", volume.name
print "VOLUME DISPLAY NAME: ", volume.display_name
if (volume.status == "available"):
print "--> Attempting mount of volume.."
volume.attach_to_instance(server, mountpoint=sata_mnt_pt)
else:
pass

Resulted in 404 errors due to the calls being made to the wrong datacenter:

Traceback (most recent call last):
File "./mounttrycs.py", line 25, in
volume.attach_to_instance(server, mountpoint="/dev/xvdb")
File "/Library/Python/2.6/site-packages/pyrax/cloudblockstorage.py", line 149, in attach_to_instance
self.id, mountpoint)
File "/Library/Python/2.6/site-packages/novaclient/v1_1/volumes.py", line 118, in create_server_volume
body, "volumeAttachment")
File "/Library/Python/2.6/site-packages/novaclient/base.py", line 145, in _create
_resp, body = self.api.client.post(url, body=body)
File "/Library/Python/2.6/site-packages/novaclient/client.py", line 230, in post
return self._cs_request(url, 'POST', *_kwargs)
File "/Library/Python/2.6/site-packages/novaclient/client.py", line 214, in _cs_request
*_kwargs)
File "/Library/Python/2.6/site-packages/novaclient/client.py", line 196, in _time_request
resp, body = self.request(url, method, **kwargs)
File "/Library/Python/2.6/site-packages/novaclient/client.py", line 190, in request
raise exceptions.from_response(resp, body, url, method)
novaclient.exceptions.NotFound: The resource could not be found. (HTTP 404) (Request-ID: req-110c3022-b2df-4cfb-b581-61f226d6d58d)

All work is being done in ORD and we were able to confirm the DC was different by inserting the following after "Attempting mount.." in the above code:
print "PYRAX VOLUMES", pyrax.cloudservers.volumes.api.client.management_url

Workaround thanks to Ed Leafe is to insert the following prior to the actual call to attach_to_instance:
volume._nova_volumes = cs.volumes

Subject says it all.

When you query cloud servers using pyrax, it authenticates with the given credentials, and then passes some of that info on to novaclient, however novaclient is firing off another auth request to identity.

We need to determine the cause and cut that un-needed second request to identity.

Authenticating with pyrax.set_credentials() seems to have broken in pyrax >=1.4.0. When I try to run code that worked with 1.3.9 I get the following exception:

pyrax.exceptions.IdentityClassNotDefined: No identity class has been defined for the current environment.

In several places the cloud_servers documentation module references the cs.get() method, where cs is an instance of pyrax.cloudservers. cloudservers has no 'get' method. It should be cloudservers.servers

Example:

Creating an Image of a Server
If you have a Server object and want to create an image of that server, you can call its create_image() method, passing in the name of the image to create, along with any optional metadata for the image.

cs = pyrax.cloudservers
server = cs.get(id_of_server)
server.create_image("my_image_name")

Steps to reproduce:

Python 2.7 (r27:82508, Jul 3 2010, 20:17:05)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import pyrax
pyrax.set_credentials("myuser", "myapikey")
cs = pyrax.cloudservers
dir(cs)
['class', 'delattr', 'dict', 'doc', 'format', 'getattribute', 'hash', 'init', 'module', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', 'aggregates', 'authenticate', 'certs', 'client', 'cloudpipe', 'dns_domains', 'dns_entries', 'exceptions', 'fixed_ips', 'flavor_access', 'flavors', 'floating_ip_pools', 'floating_ips', 'fping', 'get_timings', 'hosts', 'hypervisors', 'images', 'keypairs', 'limits', 'networks', 'project_id', 'quota_classes', 'quotas', 'reset_timings', 'security_group_rules', 'security_groups', 'servers', 'services', 'set_management_url', 'usage', 'virtual_interfaces', 'volume_snapshots', 'volume_types', 'volumes']
dir(cs.servers)
['class', 'delattr', 'dict', 'doc', 'format', 'getattribute', 'hash', 'init', 'module', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', '_action', '_boot', '_create', '_delete', '_get', '_hooks_map', '_list', '_update', 'actions', 'add_fixed_ip', 'add_floating_ip', 'add_hook', 'add_security_group', 'api', 'backup', 'change_password', 'completion_cache', 'confirm_resize', 'create', 'create_image', 'delete', 'delete_meta', 'diagnostics', 'find', 'findall', 'get', 'get_console_output', 'get_vnc_console', 'list', 'live_migrate', 'lock', 'migrate', 'pause', 'reboot', 'rebuild', 'remove_fixed_ip', 'remove_floating_ip', 'remove_security_group', 'rescue', 'reset_state', 'resize', 'resource_class', 'resume', 'revert_resize', 'run_hooks', 'set_meta', 'start', 'stop', 'suspend', 'unlock', 'unpause', 'unrescue', 'update', 'write_to_completion_cache']

Since BaseClient was lifted directly from novaclient, it contains all of novaclient's auth and re-auth infrastructure. This means that once a token expires, each client will have to re-auth, which is inefficient. More seriously, the pending changes to the Identity class to make it Keystone-compatible won't work if the individual clients don't all use the Identity class.

Cannot authenticate with the last version of python-novacllent:

Traceback (most recent call last):
File "./bin/python", line 52, in
exec(compile(file__f.read(), __file, "exec"))
File "test.py", line 3, in
pyrax.set_credential_file('/home/sdouche/.rackspace_cloud_credentials')
File "/home/sdouche/src/sact/internal/sact.vmcloud/eggs/pyrax-1.1.6b-py2.7.egg/pyrax/init.py", line 207, in set_credential_file
connect_to_services()
File "/home/sdouche/src/sact/internal/sact.vmcloud/eggs/pyrax-1.1.6b-py2.7.egg/pyrax/init.py", line 257, in connect_to_services
cloudservers = connect_to_cloudservers()
File "/home/sdouche/src/sact/internal/sact.vmcloud/eggs/pyrax-1.1.6b-py2.7.egg/pyrax/init.py", line 174, in _wrapped
return fnc(_args, *_kwargs)
File "/home/sdouche/src/sact/internal/sact.vmcloud/eggs/pyrax-1.1.6b-py2.7.egg/pyrax/init.py", line 300, in connect_to_cloudservers
region_name=region, service_type="compute")
TypeError: init() got an unexpected keyword argument 'bypass_url'

It works with python-novaclient 2.10.0. Please pin this version in the setup.py file.

Looks like calling a set credentials in the working branch without a config file set will cause the identity_class returned to be None. Haven't been able to get the cause, but the nosetests are also failing on this.

Creating a ~/.pyrax.cfg with only a [settings] header (no values) resolves the issue.

TypeError                                 Traceback (most recent call last)
<ipython-input-1-16392d617b0a> in <module>()
     16 import os
     17
---> 18 pyrax.set_credentials('xxxx', 'xxxx')
     19
     20 def main():

/Users/xxxx/Dropbox/Programming/PycharmProjects/.virtualenvs/pyrax/lib/python2.7/site-packages/pyrax-1.3.9-py2.7.egg/pyrax/__init__.pyc in _wrapped(*args, **kwargs)
    288         if identity is None:
    289             cls = settings.get("identity_class")
--> 290             identity = cls()
    291         return fnc(*args, **kwargs)
    292     return _wrapped

TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_cloud_db (tests.integrated.test_module.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/tests/integrated/test_module.py", line 17, in setUp
    pyrax.set_credential_file(creds_file)
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/pyrax/__init__.py", line 290, in _wrapped
    identity = cls()
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_cloud_loadbalancers (tests.integrated.test_module.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/tests/integrated/test_module.py", line 17, in setUp
    pyrax.set_credential_file(creds_file)
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/pyrax/__init__.py", line 290, in _wrapped
    identity = cls()
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_cloudfiles_base_container (tests.integrated.test_module.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/tests/integrated/test_module.py", line 17, in setUp
    pyrax.set_credential_file(creds_file)
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/pyrax/__init__.py", line 290, in _wrapped
    identity = cls()
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_cloudservers_images (tests.integrated.test_module.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/tests/integrated/test_module.py", line 17, in setUp
    pyrax.set_credential_file(creds_file)
  File "/Users/xxxx/Dropbox/Programming/PycharmProjects/pyrax/pyrax/__init__.py", line 290, in _wrapped
    identity = cls()
TypeError: 'NoneType' object is not callable

Trying to upload a file to a container that contains 17,717 objects, like so:

self.container.upload_file(fileobj,
                           cloud_id,
                           content_type)

where fileobj is a file-like object, cloud_id is 518950d4fbf9f22d6624ea4d-518950d8fbf9f22dcc1f65ed-200x102-Wkx8y1535GnW, and content_type is image/jpeg

I get this exception:

Traceback (most recent call last):
  [...]
  File "/Users/joeshaw/src/gemstone-server/virtualenv/lib/python2.6/site-packages/pyrax/cf_wrapper/container.py", line 149, in upload_file
    content_type=content_type, etag=etag, return_none=return_none)
  File "/Users/joeshaw/src/gemstone-server/virtualenv/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 44, in _wrapped
    return fnc(*args, **kwargs)
  File "/Users/joeshaw/src/gemstone-server/virtualenv/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 566, in upload_file
    return self.get_object(container, obj_name)
  File "/Users/joeshaw/src/gemstone-server/virtualenv/lib/python2.6/site-packages/pyrax/cf_wrapper/client.py", line 416, in get_object
    obj = cont.get_object(self._resolve_name(obj_name))
  File "/Users/joeshaw/src/gemstone-server/virtualenv/lib/python2.6/site-packages/pyrax/cf_wrapper/container.py", line 115, in get_object
    raise exc.NoSuchObject("No object with the name '%s' exists" % name)
NoSuchObject: No object with the name '518950d4fbf9f22d6624ea4d-518950d8fbf9f22dcc1f65ed-200x102-Wkx8y1535GnW' exists

I've tried uploading this against a smaller container and don't have any issues. If I call pyrax's container.get_object() on it, I get a NoSuchObject exception. It works if I use python-cloudfiles.

Is there a limit in the amount of data being fetched back inside get_object()?

Typically there will be a single record for a single host, so the search_records that returns a list makes it cumbersome to be certain you are doing the right thing with the results. It would be much more simple to have something like find_record or get_record that returns a single record object to perform actions on:

dns = pyrax.cloud_dns
dom = dns.find(name='example.com')
rec = dom.get_record(name='host1.example.com')
new_rec = rec.update(data="1.2.3.4")

If there are no records by that name, a NotFound could be thrown, and if there are more than one a NotUnique could be thrown.

Hi, is there a way to use the internalURL?

Thanks!

Just happened today. Here's the stacktrace:

Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/rax/management/commands/raxsync.py", line 37, in handle
    self.connection.sync_folder_to_container(tmp_dir, self.container)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 516, in sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 534, in _sync_folder_to_container
    include_hidden=include_hidden, ignore_timestamps=ignore_timestamps)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 555, in _sync_folder_to_container
    cont.upload_file(pth, obj_name=fullname, etag=local_etag)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/container.py", line 125, in upload_file
    content_type=content_type, etag=etag)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 39, in _wrapped
    return fnc(*args, **kwargs)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 440, in upload_file
    return self.get_object(container, obj_name)
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/client.py", line 319, in get_object
    obj = cont.get_object(self._resolve_name(obj_name))
  File "/var/www/.sites/plumradio_20130209182358/env/local/lib/python2.7/site-packages/pyrax/cf_wrapper/container.py", line 93, in get_object
    raise exc.NoSuchObject("No object with the name '%s' exists" % name)
pyrax.exceptions.NoSuchObject: No object with the name 'static/tiny_mce/plugins/style/langs/et_dlg.js' exists

I ran it again, and it seemed to get past that point, and then threw the error on a different file. I ran it a third time and it completed successfully. (around 2300 files uploaded to my container)

While working on a project, I encountered a pretty standard issue of my authorization expiring, throwing an exception. This exception is easily caught, and pyrax.authenticate() brings things back up to speed.

The issue though is that this is a multi-threaded script, and I would need an easy way to have the threads "pause" themselves when this happens, rather than running through the queue of tasks and failing at each one.

The simplest way to achieve this would be to have the pyrax.exceptions.AuthorizationFailure exception set pyrax.identity.authenticated to False. Then this value could be checked just before an API operation in a thread, if false, wait until true...

Calling pyrax.authenticate() sets this value to true, so the inverse should likely also apply.