Pismo Redirection Not Allowed about pismo HOT 5 OPEN

I would also appriciate allowing redirects. This guestion on stackoverflow seems to deal the same problem. Suppose open-uri is the cause of the problem.

from pismo.

@peterc I think this has been a problem forever in open-uri. Do you have a recommendation for what to do here? I tried adding this gem that monkey patches open-uri and provides a allow_redirects option. I tried this out with a New York Times article and it followed their redirect, but hit their paywall, which means the NYT link isn't useful in our app, but it still feels like there should be a built-in way to allow redirects, at least "safe" ones. https://github.com/andyatkinson/pismo/commit/39da3f68380e8260a787562e4f3e21c752a2791d

from pismo.

Hmm, well they must be detecting it somehow, so I'm guessing it's probably through the User-Agent on the request? If so, spoofing that User-Agent with that from a regular browser could work.

Pismo essentially includes open-uri to act as a sort of fallback if you didn't obtain the data in any other way or with a better/proper library, so I don't think going too far in implementing a more powerful HTTP client is a good idea, when they already exist separately (such as Typhoeus or even net/http). That said, spoofing the User-Agent is most likely the quick/cheap fix.

from pismo.

Ok. I'll mess around with the user-agent and try out a separate HTTP library. Thanks for the ideas!

from pismo.

If you find one that's pretty customizable and pure Ruby, let me know as it could become a dependency here. Any native/compiled stuff would be a no-no. Though to be fair, net/http could be used in a more advanced way here I guess..

from pismo.