mendeley / mendeley-python-sdk Goto Github PK

Hi, I notice that nearly every requirement is locked down/pinned. This is extremely restrictive on projects that depend on this package. It is common for developers to pin the oldest compatible version of something, then to allow up to the next breaking release. So, perhaps for example:

# requirements.txt

...
requests>=2.5.1,<3.0  # declare that you require at least 2.5.1, but allow up to the next breaking release.
...

When searching for a title that contains some parenthesis, I do not get any response, example:

session.catalog.search('A lysimeter study of nitrate leaching, optimum fertilisation rate and growth responses of corn (Zea mays L.) following soil amendment with water-saving super-absorbent polymer')

gives no result even though the entry definitely exist with the exact same title:
https://www.mendeley.com/search/?query=A+lysimeter+study+of+nitrate+leaching%2C+optimum+fertilisation+rate+and+growth+responses+of+corn+%28Zea+mays+L.%29+following+soil+amendment+with+water-saving+super-absorbent+polymer&dgcid=md_homepage

However, from the url of the link I can see some chars were converted to their ascii equivalent, so this search will work:

session.catalog.search('A lysimeter study of nitrate leaching%2C optimum fertilisation rate and growth responses of corn %28Zea mays L.%29 following soil amendment with water-saving super-absorbent polymer')

I see the code is using urlencode for this but I guess it doesn't work that well?

This is a backend issue on Mendeley's servers, I know.... Please let me know if you already have an option for performing "All-And" catalog searches, but as far as I can see when playing with the API is that "Any-Or" is the only one that currently seems to work. Can you implement in the "search/catalog" API an option to search with intersections instead of just unions (within a single query string AND ALSO across catalog document attributes). For example, a search for "particle swarm optimization" searches all documents for "particle", "swarm", or "optimization" instead of all-and. Also, if I want to filter to a specific field or publication by setting the "source" parameter to "water resources planning and management", for example, then currently I get ANY document that contains a match in "source" for "water", "resources", "planning", or "management" instead of actually filtering the results. I would think it would be a trivial task to provide an option for performing "All-And" searches instead of "Any-Or", especially since the main site already performs those types of searches.

When using resource documents.iter(view='all') is there a way to get the corresponding metadata here?

mendeley.models.catalog.CatalogAllDocument.file_attached returns False for all documents.

Similarly, mendeley.models.catalog.CatalogAllDocument.files.iter() yields null generator that will force Mendeley API to throw a 404 not found.

`
import client

def searchMendeley(query, session=None, close ='yes'):
'''
Searches mendeley catalog with given query term on existing session.
Session must be authenticated with startSession() prior.
Returns an iterable catalog search object and the session object search, session
'''
if session == None:
session = client.startSession()

search = session.catalog.search(query=query, view='all').iter()

if (close == 'yes') and (session != None):
    client.closeSession(session)

return search, session

def getFiles(doc_obj):
'''
Takes a single mendeley.catalog.doc object obtained in catalog.search.
Returns an iterable files object that can be searched for files

Returns list of file_names
'''

# Create a files iterator
files = doc_obj.files

def checkFiles(file_iter):
    '''
    Iterate through mendeley.resources.files.Files object and report information.
    ** Having activity scoped here allows try statement to pick up on API errors.
    '''
    for count, file in enumerate(file_iter):
        print(count)
        return count

try:
    checkFiles(files)
except:
    print('File Search Failed')

return files

def buildSearchDict(search_obj, num_docs=10):
'''
Takes a mendeley.catalog.search object and returns a dictionary of
documents and file objects.
'''
search_dict = {}
for count, doc in enumerate(search_obj):
print('checking document', count, ':', doc.file_attached)
if count >= num_docs:
break
if not doc.file_attached:
search_dict[type(doc)] = doc.file_attached
return search_dict
`

In mendeley\resources\catalog.py, there are a few oddities I found:

1. advanced_search() is implemented without a general "query" parameter, so you can't perform general queries in addition filtering by title, author, source, etc.
2. search() does not have the ability to constrain by min_year and max_year... I got around this by adding a "query" parameter to advanced_search and calling that with: advanced_search(query="something", min_year=1970, max_year=1975)
3. a backend issue, the REST service does not take both query and title, source, etc. which would be useful when filtering query results (see my other issue)

TypeError: authenticate() takes exactly 2 arguments (1 given)

from mendeley import Mendeley

# These values should match the ones supplied when registering your application.
mendeley = Mendeley(client_id, client_secret=client_secret)

auth = mendeley.start_implicit_grant_flow()
session = auth.authenticate()

I noticed that the Annotations.list() method supports modified_since and deleted_since (and limit indirectly via the page_size parameter), but not group_id or document_id.

Was this intentional, an oversight, or just a "haven't gotten to it yet"? I am considering forking and adding, but I would have to familiarize myself with how to do the tests and such, I don't often develop in Python so this would be a time and effort commitment for me, which is fine but if it's a 5 minute fix on your end, that's probably better. :-)

Thanks!

Please add Support for Folders Method, is the missing feature. This addition will be very helpful.
Thanks.

mendeley has hard version requirements across the board (== instead of >=), many/all of which are now grossly out of date (requests being the most notable example). With the new pip dependency manager, we can no longer effectively replace these requirements: https://mail.python.org/archives/list/[email protected]/thread/DIWIYIMGAOHDWQXXUZW44YRSW7UYQ4CA/

As a result, our build is now broken, and I'm assuming we're not alone.

Please update your dependencies ASAP.

I'm working on a django app using the mendeley api. I've done the following:

$ pip install mendeley

In my code, simplified:

from mendeley import Mendeley

I get the error, "Cannot import name 'Mendeley'" I have tried this upper-case, lower-case, tried just importing mendeley without the "from" (which works, but I can't figure out how to refer to anything beyond that. I'm happy to help or fix the docs when I figure out what's wrong, but I'm stuck before I've done anything.

#I'm trying to use start_authorization_code_flow() to authorise (I need to have access to my documents)
The code is the following:

from mendeley import Mendeley
from requests import post, get, session

secret = '<secret>'
app_id = '<id'>
redirect_uri = 'mendeley.com'

sdk = Mendeley(app_id, secret, redirect_uri)
auth = sdk.start_authorization_code_flow()
credentials = {'username': '<username>', 'password': '<password>'}

s = session()
login_url = auth.get_login_url()
login = s.post(login_url, credentials)

Then s.post(login_url, credentials) returns a timeout error. Same if I just copy the login url and try to manually log in with browser.
However, if I'm trying to use start_client_credentials_flow() method, everything is well.

Traceback follows

<ipython-input-73-86dcfcea52de> in <module>()
----> 1 login = s.post(login_url, credentials)
/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in post(self, url, data, json, **kwargs)
    502         """
    503 
--> 504         return self.request('POST', url, data=data, json=json, **kwargs)
    505 
    506     def put(self, url, data=None, **kwargs):

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    459         }
    460         send_kwargs.update(settings)
--> 461         resp = self.send(prep, **send_kwargs)
    462 
    463         return resp

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    597 
    598         # Resolve redirects if allowed.
--> 599         history = [resp for resp in gen] if allow_redirects else []
    600 
    601         # Shuffle things around if there's history.

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in <listcomp>(.0)
    597 
    598         # Resolve redirects if allowed.
--> 599         history = [resp for resp in gen] if allow_redirects else []
    600 
    601         # Shuffle things around if there's history.

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in resolve_redirects(self, resp, req, stream, timeout, verify, cert, proxies)
    190                 cert=cert,
    191                 proxies=proxies,
--> 192                 allow_redirects=False,
    193             )
    194 

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    571 
    572         # Send the request
--> 573         r = adapter.send(request, **kwargs)
    574 
    575         # Total elapsed time of the request (approximately)

/home/belyaevadg/anaconda3/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    413 
    414         except (ProtocolError, socket.error) as err:
--> 415             raise ConnectionError(err, request=request)
    416 
    417         except MaxRetryError as e:

ConnectionError: ('Connection aborted.', TimeoutError(110, 'Connection timed out'))

Any help would be appreciated!

I believe there is a bug in your auth code that raises a value error when encountering a 504 as content type if not set on the 504 response.

mendeley/auth.py in handle_text_response at line 12

def handle_text_response(rsp):
    if rsp.headers['content-type'] == 'text/plain':
        rsp._content = bytes(json.dumps({'error': 'invalid_client', 'error_description': rsp.text}), rsp.encoding)
        rsp.headers['content-type'] = 'application/json'
    return rsp

rsp.headers['content-type'] == 'text/plain': Raises a value error. Can you please fix this so the app using the API can choose how to handle the 504 response.

We are using version 0.3.2.

The /metadata call requires an Accept header of 'application/vnd.mendeley-document-lookup.1+json' but the resources/catalog.py#lookup() is creating an Accept header set to obj_type.content_type, which is always 'application/vnd.mendeley-document.1+json'

Working on a biblio data viz from Mendeley database...
Is there a way to retrieve author profiles of a document ?
For now, I have only been able to retrieve authors (list of persons) but not their profiles. That would be a valuable source of informations to get their institute, their country.

Here is my snippet of code:

# "name": "Ocean Acidification (OA-ICC)"
id = "8ffa1079-1e16-30d0-80f3-e0808fa1e017"

group = session.groups.get(id)
d=group.documents.get("00ba8a9e-4536-3355-a3e1-ba1cf1e4afa3") 
for person in d.authors:
        print person.last_name, person.first_name

gives:

Thomsen J
Casties I
Pansch C
Körtzinger A
Melzner F
10.1594/PANGAEA.829723 None

Notice the last one that is indicated as a person whereas it is a DOI. Perhaps a lack of checking when the input has been done. Get the profiles of the authors would help to filter such a case.

Since the node package repo is not public (for what reason?), I am logging this here.

When doing a catalog search:

const api = mendeley({
  authFlow: mendeley.Auth.clientCredentialsFlow({
    clientId: 'clientId',
    clientSecret: 'clientSecret',
    redirectUri:'redirectUri'
  })
});
await api.search.catalog({ title: 'Nitrogen loss from fertilizers applied to maize, wheat and rice in the North China Plain' })

This will fail:

TypeError: Cannot read property 'trim' of undefined
    at extractLinkHeaders ([...]/node_modules/@mendeley/api/lib/request.js:184:25)
    at Request.onDone ([...]/node_modules/@mendeley/api/lib/request.js:125:37)

Even though when I check the reponse data, it contains 10 results with the first result the one I'm looking for. But it's trying to extract this link headers and fails

Some other titles do not cause this issue, but many do, and it's quite the bugger (also please have the node sdk as public so people can log issues?)

When I install mendeley every time then get an warning:
mendeley 0.3.2 has requirement requests==2.5.1, but you'll have requests 2.23.0 which is incompatible.
But Current Release of requests is v2.23.0

Hi,
Firstly, thank you very much for this. Currently I guess this supports only a few of the resources offered in the newly available API. Thanks!

Hello,

I would like to automatically add annotations and highlights in my documents.
Does the API provides a wat to add them? It seems that the annotations can only be listed and updated, but not created.

Thank you for your help!

The website link here

goes to:
https://dev.mendeley.com/methods/

which shows

Is there any way to include a plugin as vibreoffice to interact with pdf as Vi mode, using keys to insert notes, and navigate up and down?

Using this code (adapted from https://github.com/Mendeley/mendeley-api-python-catalog-example/blob/master/mendeley-catalog.py) :

from mendeley import Mendeley

mendeley = Mendeley("999", "XXXXXXXX")

session = mendeley.start_client_credentials_flow().authenticate()

I have the following errors :

---------------------------------------------------------------------------
MissingTokenError                         Traceback (most recent call last)
<ipython-input-15-05641665fe4b> in <module>()
      3 mendeley = Mendeley("XXX", "XXXXX")
      4 
----> 5 session = mendeley.start_client_credentials_flow().authenticate()

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/mendeley/auth.py in authenticate(self)
     29         oauth.compliance_hook['access_token_response'] = [handle_text_response]
     30 
---> 31         token = oauth.fetch_token(self.token_url, auth=self.auth, scope=['all'])
     32         return MendeleySession(self.mendeley,
     33                                token,

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/requests_oauthlib/oauth2_session.py in fetch_token(self, token_url, code, authorization_response, body, auth, username, password, method, timeout, headers, verify, **kwargs)
    197             r = hook(r)
    198 
--> 199         self._client.parse_request_body_response(r.text, scope=self.scope)
    200         self.token = self._client.token
    201         log.debug('Obtained token %s.', self.token)

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/oauthlib/oauth2/rfc6749/clients/base.py in parse_request_body_response(self, body, scope, **kwargs)
    407         .. _`Section 7.1`: http://tools.ietf.org/html/rfc6749#section-7.1
    408         """
--> 409         self.token = parse_token_response(body, scope=scope)
    410         self._populate_attributes(self.token)
    411         return self.token

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/oauthlib/oauth2/rfc6749/parameters.py in parse_token_response(body, scope)
    374 
    375     params = OAuth2Token(params, old_scope=scope)
--> 376     validate_token_parameters(params)
    377     return params
    378 

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/oauthlib/oauth2/rfc6749/parameters.py in validate_token_parameters(params)
    384 
    385     if not 'access_token' in params:
--> 386         raise MissingTokenError(description="Missing access token parameter.")
    387 
    388     if not 'token_type' in params:

MissingTokenError: (missing_token) Missing access token parameter.

Now if I use this code (from http://mendeley-python.readthedocs.org/en/latest/usage.html#authorization-code-flow) :

from mendeley import Mendeley

# These values should match the ones supplied when registering your application.
mendeley = Mendeley("XXXXX", client_secret="XXXX", redirect_uri="https://localhost")

auth = mendeley.start_authorization_code_flow()

# The user needs to visit this URL, and log in to Mendeley.
login_url = auth.get_login_url()

# After logging in, the user will be redirected to a URL, auth_response.
session = auth.authenticate(auth)

I have the following error :

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-17-3d0b0b562bf9> in <module>()
     10 
     11 # After logging in, the user will be redirected to a URL, auth_response.
---> 12 session = auth.authenticate(auth)

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/mendeley/auth.py in authenticate(self, redirect_url)
     66                                        authorization_response=redirect_url,
     67                                        auth=self.auth,
---> 68                                        scope=['all'])
     69 
     70         return MendeleySession(self.mendeley,

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/requests_oauthlib/oauth2_session.py in fetch_token(self, token_url, code, authorization_response, body, auth, username, password, method, timeout, headers, verify, **kwargs)
    154         if not code and authorization_response:
    155             self._client.parse_request_uri_response(authorization_response,
--> 156                     state=self._state)
    157             code = self._client.code
    158         elif not code and isinstance(self._client, WebApplicationClient):

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/oauthlib/oauth2/rfc6749/clients/web_application.py in parse_request_uri_response(self, uri, state)
    172             oauthlib.oauth2.rfc6749.errors.MismatchingStateError
    173         """
--> 174         response = parse_authorization_code_response(uri, state=state)
    175         self._populate_attributes(response)
    176         return response

/home/hadim/local/conda/envs/st/lib/python3.4/site-packages/oauthlib/oauth2/rfc6749/parameters.py in parse_authorization_code_response(uri, state)
    218         raise InsecureTransportError()
    219 
--> 220     query = urlparse.urlparse(uri).query
    221     params = dict(urlparse.parse_qsl(query))
    222 

/home/hadim/local/conda/envs/st/lib/python3.4/urllib/parse.py in urlparse(url, scheme, allow_fragments)
    290     Note that we don't break the components up in smaller bits
    291     (e.g. netloc is a single string) and we don't expand % escapes."""
--> 292     url, scheme, _coerce_result = _coerce_args(url, scheme)
    293     splitresult = urlsplit(url, scheme, allow_fragments)
    294     scheme, netloc, url, query, fragment = splitresult

/home/hadim/local/conda/envs/st/lib/python3.4/urllib/parse.py in _coerce_args(*args)
    110     if str_input:
    111         return args + (_noop,)
--> 112     return _decode_args(args) + (_encode_result,)
    113 
    114 # Result objects are more helpful than simple tuples

/home/hadim/local/conda/envs/st/lib/python3.4/urllib/parse.py in _decode_args(args, encoding, errors)
     94 def _decode_args(args, encoding=_implicit_encoding,
     95                        errors=_implicit_errors):
---> 96     return tuple(x.decode(encoding, errors) if x else '' for x in args)
     97 
     98 def _coerce_args(*args):

/home/hadim/local/conda/envs/st/lib/python3.4/urllib/parse.py in <genexpr>(.0)
     94 def _decode_args(args, encoding=_implicit_encoding,
     95                        errors=_implicit_errors):
---> 96     return tuple(x.decode(encoding, errors) if x else '' for x in args)
     97 
     98 def _coerce_args(*args):

AttributeError: 'MendeleyAuthorizationCodeAuthenticator' object has no attribute 'decode'

Any help would be great !

$ conda info
Current conda install:

             platform : linux-64
        conda version : 3.11.0
  conda-build version : 1.12.1
       python version : 3.4.3.final.0
     requests version : 2.6.2
     root environment : /home/hadim/local/conda  (writable)
  default environment : /home/hadim/local/conda/envs/st
     envs directories : /home/hadim/local/conda/envs
        package cache : /home/hadim/local/conda/pkgs
         channel URLs : https://repo.continuum.io/pkgs/free/linux-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/linux-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : None
    is foreign system : False

$ pip freeze | grep mendeley
mendeley==0.3.2

Because there are many non-English users, some actually Chinese users, want to add an automatic translation plug-in, you can use the mouse to select the document to get the translated content.

Hello,

Currently when trying to use the file.download method on the file model I'm getting the following:

MendeleyApiException: The Mendeley API returned an error (status: 400, message: 
InvalidArgumentOnly one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specifiedAuthorizationBearer MSwxNDIwODY0MTgzMzk5LDI2NTAxNjkxLDEwMjgsYWxsLCxHcDBlenRJU3dpMFR2WFllN2hSc3FTZXQ2UkE3C648160ABEAE54FFe/e5FSQ8t91QOHG2FIzJhO1C9uC1ZvXoqZuBsG9EmBycyGsgIgZFVJqmVduF+SD)

All other queries work fine.

Please let me know what I may be doing wrong.

Thanks!

When I use your mendeley-api-python-example and the access token gets expired I expected the access token gets refreshed automatically, but it won´t.

So here in are the steps to reproduce:

1. install and run the mendeley-api-python-example
2. wait until the token gets expired (1h)
3. refresh website

expected result

the access_token will be refreshed with the refresh_token

actual result

TokenExpiredError

expected fix: using MendeleyAuthorizationCodeTokenRefresher

I expected to fix this bug by using MendeleyAuthorizationCodeTokenRefresher in mendeley-example.py#L101
So I change the method to:

def get_session_from_cookies():
    authenticator = MendeleyAuthorizationCodeAuthenticator(mendeley, session['state'])
    refresher = MendeleyAuthorizationCodeTokenRefresher(authenticator)
    return MendeleySession(mendeley, session['token'], refresher=refresher)

actual result

TokenExpiredError
and log: No new refresh token given. Re-using old.

possible fix in MendeleyAuthorizationCodeAuthenticator

the problem is oauth.refresh_token try to find a refresh token his object, but because the OAuth2Session object is recreated this token must be passed explicit:

--- auth.py (revision )
+++ auth.py (revision )
@@ -104,7 +104,7 @@
         self.redirect_uri = authenticator.mendeley.redirect_uri

     def refresh(self, session):
-        oauth = OAuth2Session(client=self.client, redirect_uri=self.redirect_uri, scope=['all'])
+        oauth = OAuth2Session(client=self.client, redirect_uri=self.redirect_uri, scope=['all'], token=session.token)
         oauth.compliance_hook['access_token_response'] = [handle_text_response]

         session.token = oauth.refresh_token(self.token_url, auth=self.auth)

This should be fixed the bug
regards,
david