Git Product home page Git Product logo

tumblr_download's Introduction

tumblr_download

pip3 install lxml gevent PySocks

Usage: tumblr.py [options]

Options:
  -h, --help            show this help message and exit
  -s SITES, --sites=SITES
                        sites split with ',', example:2013117,66666
  --type=TYPE           [photo|video|both]
  --thread=THREAD       threads
  --proxy=PROXY         socks5://127.0.0.1:1080

can be changed in source file:
RETRY: retry times, default 1
PROXY: because G.F.W must use proxy
TIMEOUT: time to wait net IO

example:
  python3 tumblr.py -s 2013117,66666 --type=both --thread=10

API:
  http://{site}.tumblr.com/api/read?type={photo|video}&num={pagesize}&start={start}

tumblr_download's People

Contributors

lc4t avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

tumblr_download's Issues

Can't support sock5

My sock5 is local shadowsocks, and the SS is used by me in daily work.
the tumblr_download can't support sock5 hosted by SS.
Would you have a look at the issue?

D:\GreenApplications\tumblr_download-master>python tumblr.py -s bjsxey --type=both --thread=10 --proxy=sock5://127.0.0.1:1199
[2017-01-10 20:13:40.749410] [WinError 183] Cannot create a file when that file already exists: 'bjsxey'
Traceback (most recent call last):
File "tumblr.py", line 177, in
tasks(site, options.type, int(options.thread))
File "tumblr.py", line 148, in tasks
total = crawler(site, types)
File "tumblr.py", line 128, in crawler
if requests.get('http://%s.tumblr.com/' % (site), proxies=PROXY).status_code != 200:
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 390, in send
conn = self.get_connection(request.url, proxies)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 290, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 194, in proxy_manager_for
**proxy_kwargs)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\poolmanager.py", line 363, in proxy_from_url
return ProxyManager(proxy_url=url, **kw)
File "D:\Users\ag\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\poolmanager.py", line 314, in init
raise ProxySchemeUnknown(proxy.scheme)
requests.packages.urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme sock5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.