Add custom user-agents to crawler curl and python-requests about apkcrawler HOT 2 CLOSED

Thanks for the quick commits @NicholasBuse!

Re: custom user agents, I don't have a strong opinion, because unless it's a forced feature that ensures the UA is unique, people will just either use the sample one or just leave it blank and default to yours. As long as I can tell this scraper from hammer bots, we're good.

from apkcrawler.

@archon810

I added the attached user-agent ( {'User-Agent': 'OpenGApps APKMirrorCrawler/1.0'}) as a default for the script. Maarten and I discussed adding an additional configurable user-agent via config file that we could have unique one. (So our user agent for the script would be different than others running this script). Does that have any value to you?

Also, I have a PR (opengapps/opengapps#425) to add a user-agent to our upload scripts (curl would use: -A "OpenGAppsUploader")

from apkcrawler.