Comments (5)
Contributions are welcome!
from scrapy-redis.
I would be happy to get in (I'm maintaining a big scrapper with scrapy-redis), but I never did such a task before. Do you (does anyone) have some guidance on how to proceed?
from scrapy-redis.
First thing it would be to try to setup your project in Python 3.11.
If everything installs fine then try to run it and see if you get the same behavior.
If something goes wrong you can report back here and we can figure it out.
from scrapy-redis.
Coll, cool, cool, that was my plan, I just stopped seeing, that scrapy-redis
supports =< 3.10
. I'm starting to create a new project then, and, eventually, 'll come back here.
from scrapy-redis.
@rmax @lion137
I've already built scrapy_redis
with python3.11
, it's compatible, I'll update the necessary files in these days.
Then we can issue next version.
from scrapy-redis.
Related Issues (20)
- Is there a way to stop spider check duplicate with redis ? HOT 5
- [docs] Chinese documentation
- [style] Fix pylint style
- Redis data persistence HOT 4
- Scrapy 2.6.1 Unsupport function make_requests_from_url to use HOT 5
- error object has no attribute 'make_requests_from_url' HOT 2
- [spiders] remove duplicate check setting types
- why I can not see request record in redis HOT 2
- 你好,大佬,请问一个问题,我用scrapy-redis执行爬虫的时候,设置最大并发是20,但是怎么感觉像创建20个队列一样,上一个20执行完之后,下一个20再执行,大佬,您能为我解答一下吗,感激不尽 HOT 2
- How to use scrapy-redis if I'm using start_requests() instead of start_urls in my spider? HOT 3
- 日志报警:String request is deprecated
- 警告: Passing a 'spider' argument to ExecutionEngine.crawl is deprecated HOT 4
- make_request_from_data implementation in RedisMixin HOT 2
- Cleanup requirements HOT 1
- Playwright? HOT 2
- Scrapy 2.8.0 deprecated function scrapy.utils.request.request_fingerprint() warning HOT 1
- [dev] Add Type annotations
- Add Type annotations pep-0483
- How does the CrawlSpider work?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scrapy-redis.