Any suggestion for error 404? about scholarly HOT 5 CLOSED

This request #26 allows the use of proxy for the request.

from scholarly.

This request #26 allows the use of proxy for the request.

Hi! Thanks for your advise!

Besides proxy, I recommend if we could add a list of headers rather than a fixed one. Scholarly will be more useful if this project become robust for intensive collection work!

from scholarly.

The issue is that the ReCAPTCHA flagging system is a black box, so it's unclear what triggers it. I've tried to have the code as accommodating as possible (including the long query delay in _get_page()), but it still triggers the ReCAPTCHA verification. So, it's an ongoing issue for this project.

from scholarly.

I was hit with "Please show you're not a robot" with my first request ever. Since no error was thrown, I just assumed I was using it wrong. Do we need a PR to at least throw an error when the "Please show you're not a robot" page is returned?

I'll add that I was able to get it working by replacing _HEADERS with just: _HEADERS = {'cookie': 'SID=XXXX; HSID=XXXX; SSID=XXXX'}, where the XXXX values were obtained from the method described here: ckreibich/scholar.py#52 (comment)

from scholarly.

Addressed with proxy.

from scholarly.