oldshensheep / v2ex_scrapy Goto Github PK

包都pip install了
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/xinmans/mydev/v2ex_scrapy/v2ex_scrapy/spiders/V2exSpider.py", line 7, in
from v2ex_scrapy.DB import DB
File "/home/xinmans/mydev/v2ex_scrapy/v2ex_scrapy/DB.py", line 7, in
from v2ex_scrapy.items import (
File "/home/xinmans/mydev/v2ex_scrapy/v2ex_scrapy/items.py", line 13, in
class Base(DeclarativeBase):
File "/home/xinmans/mydev/v2ex_scrapy/v2ex_scrapy/items.py", line 15, in Base
list[str]: JSON,
TypeError: 'type' object is not subscriptabl

rt

2023-07-06 14:50:53 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://www.v2ex.com/t/326> (referer: None)
2023-07-06 14:50:53 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.v2ex.com/t/326>: HTTP status code is not handled or not allowed
2023-07-06 14:50:53 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://www.v2ex.com/t/327> (referer: None)
2023-07-06 14:50:54 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.v2ex.com/t/327>: HTTP status code is not handled or not allowed
2023-07-06 14:50:54 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://www.v2ex.com/t/328> (referer: None)
2023-07-06 14:50:54 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.v2ex.com/t/328>: HTTP status code is not handled or not allowed
2023-07-06 14:50:56 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://www.v2ex.com/t/329> (referer: None)
2023-07-06 14:50:56 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.v2ex.com/t/329>: HTTP status code is not handled or not allowed

建议加一些伪造useragent等逻辑

比如加入“收藏量”、“感谢量”等