Git Product home page Git Product logo

xunlei-lixian's People

Contributors

iambus avatar layerssss avatar leeroyding avatar leojay avatar sycx avatar tchaikov avatar wayhomeke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

xunlei-lixian's Issues

lx help, and more cli doc

  1. improve lx --help
  2. support lx help download
  3. support lx help readme (this should print the README, a full doc)
  4. support lx download --help

download incomplete bt task

when downloading with taskid/fileid, if the file is complete but the whole task is incomplete, current code won't download any file. should support this usage.

support lx rename

sometimes the default task name from bt includes invalid chars. support "lx rename" to rename it.

utf-8 编码问题

$ lx list

File "/workshop/xunlei-lixian/lixian_cli.py", line 709, in <module>
    execute_command()
File "/workshop/xunlei-lixian/lixian_cli.py", line 706, in execute_command
    commands[command](args[1:])
File "/workshop/xunlei-lixian/lixian_cli.py", line 526, in list_task
    tasks = client.read_all_tasks()
File "/workshop/xunlei-lixian/lixian.py", line 169, in read_all_tasks
    links, next_link = self.read_task_page(st)
File "/workshop/xunlei-lixian/lixian.py", line 162, in read_task_page
   return self.read_task_page_url(url)
File "/workshop/xunlei-lixian/lixian.py", line 145, in read_task_page_url
   page = req.read().decode('utf-8')
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
   return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xef in position 49351: invalid continuation byte

bt下载完成以后校验出错

Hashing bt ...
Traceback (most recent call last):
File "lx", line 738, in
execute_command()
File "lx", line 735, in execute_command
commandscommand
File "lx", line 471, in download_task
download_multiple_tasks(client, download, tasks, **download_args)
File "lx", line 327, in download_multiple_tasks
download_single_task(client, download, task, output_dir=output_dir, delete=d
elete, resuming=resuming, overwrite=overwrite, mini_hash=mini_hash)
File "lx", line 310, in download_single_task
verified = lixian_hash_bt.verify_bt(filename, lixian_hash_bt.bdecode(torrent
_file)['info'], file_set=file_set, progress_callback=bar.update)
File "/thunder/lixian_hash_bt.py", line 219, in verify_bt
return verify_bt_multiple(path, info, file_set=file_set, progress_callback=p
rogress_callback)
File "/thunder/lixian_hash_bt.py", line 179, in verify_bt_multiple
if sha1sum.digest() != sha1_stream.next_sha1() and complete_piece:
File "/thunder/lixian_hash_bt.py", line 111, in next_sha1
self.progress_callback(float(self.processed)/self.total)
File "lx", line 228, in update
bar = '{:>3}%[{:<40}]'.format(percent, bar)
ValueError: zero length field name in format

运行环境是python 2.6.4-3

python2.6运行lixian_cli.py发生错误以及aria2的支持。

python2.6运行lixian_cli.py发生一些错误,自己修改一下正常了,还有aria2输出文件名是--out选项。
以下是我的修改。
gdriveid = str(client.get_gdriveid())

  • aria2_opts = ['aria2c', '--header=Cookie: gdriveid='+gdriveid, download_url]
  • aria2_opts.extends(('--split', '5')) #AttributeError: 'list' object has no attribute 'extends'(python 2.6.5)
  • aria2_opts.extends(('--output-file', filename)) # TODO: this option seems not supported by aria2c ... use --out for aria2
  • aria2_opts = ['aria2c', '--header=Cookie: gdriveid='+gdriveid, '--split=5', '--out='+filename, download_url]
  • #aria2_opts.extends(('--split', '5'))
  • #aria2_opts.extends(('--out', filename)) # TODO: this option seems not supported by aria2c ...
    if resuming:
    aria2_opts.append('-c')
    exit_code = subprocess.call(aria2_opts)
    @@ -212,7 +212,7 @@ def aria2_download(client, download_url, filename, resuming=False):

TODO: support axel, ProZilla

def escape_filename(name):

  • name = re.sub(r'&(amp;)+', '&', name, flags=re.I) #TypeError: sub() got an unexpected keyword argument 'flags'
  • name = re.sub(r'&(amp;)+', '&', name, re.I)
    name = re.sub(r'[/:*?"<>|]', '-', name)
    return name

Exception: Probably login failed 登陆失效问题

使用的命令式 "lx login 迅雷号 密码" 和config后直接直接这两种方式都出现下面的登陆异常, 之前丢服务器上都没什么问题,跑了1个月, 这两天突然就这样。 但是web登陆离线就没有问题, 会不会是迅雷的登陆机制做了更改? 请帮忙看看。

运行环境是windows和centos都是python2.7

Traceback (most recent call last):
File "lixian_cli.py", line 693, in
execute_command()
File "lixian_cli.py", line 690, in execute_command
commandscommand
File "lixian_cli.py", line 68, in login
client = XunleiClient(args.username, args.password, args.cookies)
File "H:\resource\iambus-xunlei-lixian-01eedc4\lixian.py", line 35, in _init
_
self.login(username, password)
File "H:\resource\iambus-xunlei-lixian-01eedc4\lixian.py", line 107, in login
self.id = self.get_userid()
File "H:\resource\iambus-xunlei-lixian-01eedc4\lixian.py", line 63, in get_use
rid
raise Exception('Probably login failed')
Exception: Probably login failed

为何批量下载没有效果

lx list查看离线任务id。用lx download task-id1 task-id2 task-id3等进行下载,下完task-id1之后便退出了。用lx download --all也无法下载全部的离线任务,只下载第一个任务。不知道是什么问题。

强烈要求支持 axel

支持多线程,断点续传,Cookie 的好工具啊,下载速度比 wget 好多了。

add --mini-hash option

If file already exists, and the file size is completed (usually means a full hash was already done before), a full hash still will be performed. Add a --mini-hash option, in such case, just run a dcid hash (supposed to be very fast). This is used to verify ID (whether user is providing a different file), not content (whether user is providing a corrupted file).

谢谢,很好用。

请问未来迅雷会封杀这个分析工具吗?

另外你的N16是使用 原装固件还是TT或者DD-WRT呢?

可以加你gtalk或者我们email聊聊吗? 可能有很多话题可以讨论。:D
[email protected]

retry次数问题

似乎只能重试3次?

以下是aria2c跟的结果:

Status Legend:
(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.
Traceback (most recent call last):
File "lixian_cli.py", line 709, in
execute_command()
File "lixian_cli.py", line 706, in execute_command
commandscommand
File "lixian_cli.py", line 393, in download_task
download_multiple_tasks(client, download, tasks, download_args)
File "lixian_cli.py", line 271, in download_multiple_tasks
download_single_task(client, download, task, options)
File "lixian_cli.py", line 264, in download_single_task
download2(client, download_url, filename, task)
File "lixian_cli.py", line 196, in download2
download1(client, url, path, size)
File "lixian_cli.py", line 178, in download1
download(client, url, path)
File "lixian_cli.py", line 128, in aria2_download
raise Exception('aria2c exited abnormaly')
Exception: aria2c exited abnormaly
因为上面都是进度条,所以看不到重试了多少次,只能重试3次是wget下跟的结果。在我的网络环境下,重试一般都是有效的,但过一段时间(不定),又会出现502或504错误,希望可以间隔30秒或60秒自动重试一次,并且不限制重试次数。

--mini-hash is confusing

Actually --no-hash is a true mini hash, --mini-hash is not...need figure out a better name...

HTTP Error 504: Gateway Time-out

need retry...

Traceback (most recent call last):
File "/root/bin/lx_batch", line 14, in
download_batch(sys.argv[1:])
File "/root/bin/lx_batch", line 11, in download_batch
lixian_cli.execute_command(['download', '--input', f, '--delete', '--continue'])
File "/opt/root/projects/xunlei-lixian/lixian_cli.py", line 666, in execute_command
commandscommand
File "/opt/root/projects/xunlei-lixian/lixian_cli.py", line 424, in download_task
download_multiple_tasks(client, download, tasks, *_download_args)
File "/opt/root/projects/xunlei-lixian/lixian_cli.py", line 328, in download_multiple_tasks
download_single_task(client, download, task, output_dir=output_dir, delete=delete, resuming=resuming, overwrite=overwrite)
File "/opt/root/projects/xunlei-lixian/lixian_cli.py", line 324, in download_single_task
client.delete_task(task)
File "/opt/root/projects/xunlei-lixian/lixian.py", line 290, in delete_task
self.delete_task_by_id(task['id'])
File "/opt/root/projects/xunlei-lixian/lixian.py", line 287, in delete_task_by_id
self.delete_tasks_by_id([id])
File "/opt/root/projects/xunlei-lixian/lixian.py", line 282, in delete_tasks_by_id
response = json.loads(re.match(r'^delete_task_resp((.+))$', self.urlopen(url).read()).group(1))
File "/opt/root/projects/xunlei-lixian/lixian.py", line 44, in urlopen
return self.opener.open(urllib2.Request(url, *_args))
File "/opt/lib/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/opt/lib/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/opt/lib/python2.7/urllib2.py", line 438, in error
return self._call_chain(_args)
File "/opt/lib/python2.7/urllib2.py", line 372, in _call_chain
result = func(_args)
File "/opt/lib/python2.7/urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 504: Gateway Time-out

上传种子时报异常

Traceback (most recent call last):
File "lixian_cli.py", line 625, in
execute_command()
File "lixian_cli.py", line 622, in execute_command
commandscommand
File "lixian_cli.py", line 551, in add_task
tasks = find_torrents_task_to_download(client, links)
File "lixian_cli.py", line 339, in find_torrents_task_to_download
client.add_torrent_task_by_content(torrent, os.path.basename(link))
File "/Users/Jackie/scripts/xunlei-lixian/lixian.py", line 261, in add_torrent_task_by_content
response = self.urlopen(commit_url, data=data).read()
File "/Users/Jackie/scripts/xunlei-lixian/lixian.py", line 40, in urlopen
args['data'] = urllib.urlencode(args['data'])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1294, in urlencode
v = quote_plus(str(v))

好像是编码问题,python 2.7不兼容?

wget下载速度很慢,经常掉线!

直接用lx download id 下载,速度只有10K左右,而且很不稳定,经常掉线,不知是不是迅雷离线服务器的问题。换过几个资源下载都一样。重试几次后会出现如下错误。
2012-03-01 23:03:58 (2.29 KB/s) - Connection closed at byte 3385312. Retrying.

--2012-03-01 23:04:03-- (try: 6) http://dl.u.lixian.vip.xunlei.com/download?fid=xzjdc7io3itr3LFrz02b0uScHCZoWKrRAAAAAMTghvxDVkHVt6OggyFBBhCbZWRd&mid=666&threshold=150&tid=68EA12CD709F53EDCA046FC796B92CCE&srcid=4&verno=1&g=C4E086FC435641D5B7A3A083214106109B65645D&scn=t9&i=C4E086FC435641D5B7A3A083214106109B65645D&t=1&ui=15748173&ti=83104037640&s=3517601896&m=0&n=01115D85262D65706C4F03D46E322E30324F00D57173756E640443883E6E642E76121F852D73656E610D1FD36D30702E6D0A47E45F00000000&ff=0&co=FC27817ECF041EB4734C0132236F01BA&vod=vod25.c10
Connecting to dl.u.lixian.vip.xunlei.com|118.186.160.13|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 3517601896 (3.3G), 3514216584 (3.3G) remaining [application/octet-stream]
Saving to: `/mnt/sda1/Download/fairplay-epl.2012.02.11.sunderland.vs.arsenal.720p.mkv'

0% [ ] 3,649,472 --.-K/s in 2m 29s

2012-03-01 23:06:35 (1.74 KB/s) - Connection closed at byte 3649472. Retrying.

--2012-03-01 23:06:41-- (try: 7) http://dl.u.lixian.vip.xunlei.com/download?fid=xzjdc7io3itr3LFrz02b0uScHCZoWKrRAAAAAMTghvxDVkHVt6OggyFBBhCbZWRd&mid=666&threshold=150&tid=68EA12CD709F53EDCA046FC796B92CCE&srcid=4&verno=1&g=C4E086FC435641D5B7A3A083214106109B65645D&scn=t9&i=C4E086FC435641D5B7A3A083214106109B65645D&t=1&ui=15748173&ti=83104037640&s=3517601896&m=0&n=01115D85262D65706C4F03D46E322E30324F00D57173756E640443883E6E642E76121F852D73656E610D1FD36D30702E6D0A47E45F00000000&ff=0&co=FC27817ECF041EB4734C0132236F01BA&vod=vod25.c10
Connecting to dl.u.lixian.vip.xunlei.com|118.186.160.13|:80... connected.
HTTP request sent, awaiting response... 502 Bad Gateway
2012-03-01 23:06:51 ERROR 502: Bad Gateway.

Traceback (most recent call last):
File "/bin/lx", line 710, in
execute_command()
File "/bin/lx", line 707, in execute_command
commandscommand
File "/bin/lx", line 416, in download_task
download_multiple_tasks(client, download, tasks, download_args)
File "/bin/lx", line 272, in download_multiple_tasks
download_single_task(client, download, task, options)
File "/bin/lx", line 265, in download_single_task
download2(client, download_url, filename, task)
File "/bin/lx", line 197, in download2
download1(client, url, path, size)
File "/bin/lx", line 179, in download1
download(client, url, path)
File "/bin/lx", line 106, in wget_download
raise Exception('wget exited abnormaly')
Exception: wget exited abnormaly

能否增加选择性下载全部的功能

很多人的离线帐号是两人共用的,添加任务时,可以加个前缀作为区别。
比如用户1的所有下载任务名前缀都是user1
能不能实现这样的功能。
lixian_cli.py download --all user1

呃……

BT文件数量BUG

经测试 list_bt(self, task)方法返回的JSON,有30的数量上限,当大于30个文件则会被忽略。

测试资源为:
--torrent bt://585b232d91bf6bace1e7882217826b9c4063f334

aria2c '--out'参数的问题

aria2 version 1.10.9
'--out'接受的参数会被整个当作一个文件名而不是路径来处理,所以最后指定任何绝对路径都会下载到lx所在目录下的相对路径下。
要解决的话需要同时传'--dir'和'--out'两个参数给aria2c。

不知可否实现「任务链接导出」和「根据任务添加时间不同而下载」?

1、能不能给 list 加一个 output 参数?这样可以直接把链接批量保存下来,和不太懂这个的朋友分享资源的时候直接给他一个txt文件,让他把里面的多个链接直接复制到下载工具的批量下载里就可以下载了。
比如:
~/xunlei-lixian/lixian_cli.py list --search 720X400 --download-url --no-id --no-name --no-status --output ~/downloads.txt

2、能不能给 download 加一个时间类的参数?
比如「下载刚刚被添加的任务」,「下载离线任务列表的前3个任务」或者「下载剩余保留天数为7天的任务」。

3、能不能给 add 加一个 prefix 参数可以用于给任务名添加各种前缀以便于过滤?比如时间前缀。
比如:
~/xunlei-lixian/lixian_cli.py add --input list.txt --prefix-currenttime
这样 list.txt 里面每一个链接被添加到离线任务列表的时候都会在任务名称前添加当前时间作为前缀。
比如 【20120310122605】The.Big.Bang.Theory.S05E15.720p.HDTV.x264-DIMENSION.mkv

谢谢~真是已经开发了太棒的工具。

在config中加入Output-dir参数

这个脚本似乎会默认将文件下载到放置这个脚本的目录下面,请问能不能在lx config 中加入选项output-dir能够指定文件默认的下载目录?

路由上多任务后台运行问题

ssh到路由运行的添加多个任务id后多任务下载如何避免ssh断开后进程结束导致下载停止?尝试采用lx download id1 id2 id3 --output-dir xxx &后发现断开ssh后,后台只执行download id1下载任务,id2 id3并没有后台下载,这个不是脚本问题,而是进程后台运行问题,想请教一下iambus是如何在路由上后台下载的。

[Bug] utf-8 解码问题

因为与人共用账号,不知道他们离线了什么文件,资源的文件名包含有 0xe3 这样的非法字符。当获取资源列表时就会发生 utf-8 解码错误,请问是否可以考虑 req.read().decode('utf-8', 'replace') 避免此类问题?

support lx config

Add lx config to save default settings:
lx config --password ...
lx config --tool asyn

hash校验

hash校验部分能否作为一个参数,下载大文件校验的时候路由cpu占用极高,甚至程序会无响应,导致队列里未下载任务中断。

请教迅雷离线下载在python2.6下的问题

在openwrt的python2.6环境下迅雷离线下载 lx download id之后出现
root@DreamBox:~# lx download 75688984323
Traceback (most recent call last):
File "/bin/lx", line 711, in
execute_command()
File "/bin/lx", line 708, in execute_command
commandscommand
File "/bin/lx", line 417, in download_task
download_multiple_tasks(client, download, tasks, download_args)
File "/bin/lx", line 273, in download_multiple_tasks
download_single_task(client, download, task, options)
File "/bin/lx", line 212, in download_single_task
filename = os.path.join(output_dir, filename)
File "/usr/lib/python2.6/posixpath.py", line 67, in join
elif path == '' or path.endswith('/'):
AttributeError: 'bool' object has no attribute 'endswith'
如何解决?有没有db120wg能用的python2.7的ipk?

执行download命令下载时,会出现WindowsError [Error 2]

Downloading U2Game.Net-Toaru.Kagaku.no.Chou.Denjihou.CRK.JPN.科学超电磁炮.日版ISO-1191M.7z ...
Traceback (most recent call last):
File "F:\develop\xunlei-lixian\lixian_cli.py", line 690, in
execute_command()
File "F:\develop\xunlei-lixian\lixian_cli.py", line 687, in execute_command
commandscommand
File "F:\develop\xunlei-lixian\lixian_cli.py", line 409, in download_task
download_multiple_tasks(client, download, tasks, download_args)
File "F:\develop\xunlei-lixian\lixian_cli.py", line 265, in download_multiple_tasks
download_single_task(client, download, task, options)
File "F:\develop\xunlei-lixian\lixian_cli.py", line 258, in download_single_task
download2(client, download_url, filename, task)
File "F:\develop\xunlei-lixian\lixian_cli.py", line 194, in download2
download1(client, url, path, size)
File "F:\develop\xunlei-lixian\lixian_cli.py", line 176, in download1
download(client, url, path)
File "F:\develop\xunlei-lixian\lixian_cli.py", line 102, in wget_download
exit_code = subprocess.call(wget_opts)
File "f:\python27\lib\subprocess.py", line 486, in call
return Popen(_popenargs, *_kwargs).wait()
File "f:\python27\lib\subprocess.py", line 672, in init
errread, errwrite)
File "f:\python27\lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2]

感觉好像是中文任务名的问题。

[FEATURE REQUEST] wish to download all completed job by one command

Hi, do you have a plan to support another command argument to download all completed items one time.

Now I can specify jobs' id to download them manually. However, if I could run a batch command, maybe lx download --completed, then all completed items are downloaded one by one automatically. I think it should be more useful :)

wget: no response from server

Connecting to gdl.lixian.vip.xunlei.com (125.39.149.75:80)
wget: no response from server
Traceback (most recent call last):
File "/bin/lx", line 709, in
execute_command()
File "/bin/lx", line 706, in execute_command
commandscommand
File "/bin/lx", line 393, in download_task
download_multiple_tasks(client, download, tasks, download_args)
File "/bin/lx", line 271, in download_multiple_tasks
download_single_task(client, download, task, options)
File "/bin/lx", line 247, in download_single_task
download2(client, download_url, path, f)
File "/bin/lx", line 196, in download2
download1(client, url, path, size)
File "/bin/lx", line 178, in download1
download(client, url, path)
File "/bin/lx", line 105, in wget_download
raise Exception('wget exited abnormaly')
Exception: wget exited abnormaly

需要手动重新lx download id 好几次才行,建议加个检测
要不添加上任务后,出了这个问题,下载任务就停了。

磁力链接input报错

单个下没问题,放文本文件里调用报错。任务已存在的话也没问题。

lx download --input data\ed2k.txt

Adding below tasks:
magnet:?xt=urn:btih:UBAHAW54JZ2QPBN7OZ36UOSY6VZIQO6B

File "lixian_cli.py", line 355, in find_tasks_to_download
raise NotImplementedError('task not found, wired: '+link)
NotImplementedError: task not found, wired: magnet:?xt=urn:btih:UBAHAW54JZ2QPBN7
OZ36UOSY6VZIQO6B

--all --continue 参数的问题

有正在下载的任务时,如果修改lixian.vip.xunlei.com的任务列表(增加新任务),之后执行 --all --continue 会删除掉之前未完成的任务,并从最后一个添加的任务开始重新下载

环境为DD-WRT+Python2.7+wget完整版

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.