smartfile / client-python Goto Github PK

View Code? Open in Web Editor NEW

20.0 20.0 7.0 211 KB

SmartFile API Client (Python).

Home Page: http://app.smartfile.com/api/

License: MIT License

Makefile 1.10% Python 98.90%

client-python's People

Contributors

Stargazers

Watchers

Forkers

mattdeboard pombredanne grunichev chaps mikespub-archive adl-software

client-python's Issues

download method: optional path to write file

When calling the download method ,
a file object is created with the same path as the requested file to download ,
paths like /Documents/somefile won't exist usually under the local filesystems where the
library is being used .

The default path to write the file contents could be the local CWD plus the name of the file being downloaded , and the method could accept a keyword argument in order to explicitly tell the desired path where to write the file contents.

Fails to install via pip on python 3

It can be installed via pip on python 2, just not python 3:

Lukes-iMac:~ luke$ mkvirtualenv -p /usr/local/bin/python3 testsmartfile
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5'
New python executable in testsmartfile/bin/python3.5
Also creating executable in testsmartfile/bin/python
Installing setuptools, pip, wheel...done.
(testsmartfile)Lukes-iMac:~ luke$ pip install smartfile
Collecting smartfile
  Using cached smartfile-2.1-6.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/private/var/folders/bq/489z07n11kg8gcptzrcrq8k00000gn/T/pip-build-wdtffuw6/smartfile/setup.py", line 16, in <module>
        with file(get_path('smartfile/__init__.py')) as f:
    NameError: name 'file' is not defined

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/bq/489z07n11kg8gcptzrcrq8k00000gn/T/pip-build-wdtffuw6/smartfile

Worth noting it still works when installing manually with setup.py (though that seems to be a higher version anyway).

Large plain/text files returned as gzip and library doesn't handle this case

python --version
Python 3.6.1

smartfile==2.19

Example from documentation:

>>> import shutil
>>> from smartfile import BasicClient
>>> api = BasicClient()
>>> f = api.get('/path/data/', 'small.txt')
>>> with open('small.txt', 'wb') as o:
>>>     # f.getheaders().get('Content-Encoding') 
>>>     shutil.copyfileobj(f, o)

works fine with small files, but when file is large (I tested 100Mb text file)
f.getheaders().get('Content-Encoding') returns gzip and instead of expected source text file gzip content is downloaded and saved to file.

It can be handled with:

        if f.getheaders().get('Content-Encoding') == 'gzip':
            o.write(gzip.decompress(f.read()))
        else:
            shutil.copyfileobj(f, o)

but ideally I think library should handle this case.
Or, at least, docs have to be updated and mention this possible issue.

<class Client> has no function 'upload()'

When the library is installed via pip, it doesn't contain the definition of the upload() function in the Client class.
This results in an error when upload() is called on a BasicClient object.
It appears the source code I got in my venv/Lib/site-packages/smartfile/init.py is different from what is published on github.

README should be changed.

Currently, the readme is of the form "restructured text". To keep things consistent across repositories we should update this to be a part of the markdown family instead.

Remove NETRC Usage

It appears that this client allows for us to pass in an NETRC file. NETRC is deprecated. Please remove this.

i.e.
https://github.com/smartfile/client-python/blame/04ff2ce406d98f3a299ab956c476ba27feae8956/smartfile/__init__.py#L135

MemoryError

When I try to upload a file with

api.post('/path/data', file=file('filename', 'rb'))

I get a MemoryError. It is because my server has only 512 mb ram and my upload size does not fit into memory usually. I think it is possible to do a streaming upload but I don't have enough knowledge on requests library.

Moving files

Hello,
According to the examples I should be able to go
api.move('file.txt', '/screenShot')
to move file.txt to screenShot.

However, I am getting the error:
Response 409: Error: b'{"detail": "Destination exists:/screenShot"}'
The api.move method used to work for me. Now it has stopped. Does anyone know how to fix this?

Thanks

Attempting to download .json file parses file as JSON.

If you attempt to download a .json file, the library instead parses the JSON content of the file. This is because parsing is triggered by the Content-Type header, which is set to application/json according to the file type being downloaded.

Downloading should disable the automatic parsing of JSON, or the library should return a response and allow the caller to parse JSON if that is desired.

Submit the latest update to PyPI

When we get a chance, we need to add the changes made in #7 to PyPI. This way people can just use pip install smartfile to import the latest SmartFile client SDK changes.

Upload Memory Error

When trying to upload a file that has more bytes than a string can hold, I get an OverflowError.

>>> import smartfile

>>> 
>>> 
>>> 
>>> api = smartfile.BasicClient('**********', '*****************')
>>> file = open('Downloads/Win10_1511_2_English_x64.iso', 'rb')
>>> api.upload('test.io', file)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/smartfile/__init__.py", line 145, in upload
    return self.post('/path/data/', file=arg)
  File "/usr/lib/python2.7/site-packages/smartfile/__init__.py", line 130, in post
    return self._request('post', endpoint, id=id, data=kwargs)
  File "/usr/lib/python2.7/site-packages/smartfile/__init__.py", line 109, in _request
    return self._do_request(request, url, **kwargs)
  File "/usr/lib/python2.7/site-packages/smartfile/__init__.py", line 206, in _do_request
    return super(BasicClient, self)._do_request(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/smartfile/__init__.py", line 52, in _do_request
    response = request(url, stream=True, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 111, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 403, in send
    timeout=timeout
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python2.7/httplib.py", line 1057, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1097, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 897, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 873, in send
    self.sock.sendall(data)
  File "/usr/lib64/python2.7/ssl.py", line 721, in sendall
    v = self.send(data[count:])
  File "/usr/lib64/python2.7/ssl.py", line 687, in send
    v = self._sslobj.write(data)
OverflowError: string longer than 2147483647 bytes
string longer than 2147483647 bytes

I believe moving towards chunked uploading would be beneficial to looking past this problem.

Parameters for Requests

Based on the documentation, it's not initially clear to me how to add the appropriate query string to get all the children of a given directory. Any help?

a code example for a simple workflow

I wonder if you could add an end to end code example that implements a simple workflow for temporary accounts

create user name x with random password y
create folder name x
give user x write access to folder x
create rule that files are deleted after successful download
create rule that user will be deactivated after x days