hentai-chan / hentai Goto Github PK
View Code? Open in Web Editor NEWImplements a wrapper class around nhentai's RESTful API.
Home Page: https://www.hentai-chan.dev/projects/hentai
License: GNU General Public License v3.0
Implements a wrapper class around nhentai's RESTful API.
Home Page: https://www.hentai-chan.dev/projects/hentai
License: GNU General Public License v3.0
I'm not sure but the Hentai.exists method doesn't seem to return false
Is your feature request related to a problem? Please describe.
At the moment, this library uses the same user agent over and over again. I recenlty ran into an interesting article that showcase how to fake and rotate between multiple user agents.
Describe the solution you'd like
See the aforementioned article.
Describe alternatives you've considered
I also saw some interesting libraries that look very promising on GitHub. Maybe we can put off some work on a third-party dependency? If the proposed library is well-maintained and offers a good solution, I would be willing to take this approach into consideration as well.
Additional context
See also the current contributing guidelines and the project's wiki. Don't forget to add a unit test to the test suite at hentai/tests/
once everything is said and done.
This is a WIP that is being developed on a private repository. The website will be hosted by pythonanywhere at
https://hentaichan.pythonanywhere.com/projects/hentai
While the website is under construction, it remains inaccessible to the general public.
I got this error everytime I try to use any function of the api:
requests.exceptions.ConnectionError:` HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/123456789 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='nhentai.net', port=443): Read timed out. (read timeout=5)"))
I don't have any idea to fix it
im using:
I don't think it's a bug, so i did not used the template to make this issue ^^
Utils.get_homepage
currently doesn't fetch the Popular Now
section, as It should. So I was thinking fo making a new function in Utils
to fetch the current trending Doujins.
I am not actually familiar with the json requests I need to make in order to fetch so currently I don't have any code that does the job but if anyone knows, you can comment the format and I'll implement the function
I got a error like this when try to execute a code
Exception has occurred: RetryError
HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /random (Caused by ResponseError('too many 503 error responses'))
This is my code
from hentai import Hentai, Format
doujin = Hentai(177013)
print(doujin.url)
Is your feature request related to a problem? Please describe.
As you detailed in this issue, a reason for a lack of Python 3.6 support was the implementation of dataclasses. I followed up with PyPy now supporting 3.7 and now support dataclasses.
I did a test and yes, dataclasses are supported.
Python 3.7.4 (87875bf2dfd8, Sep 24 2020, 07:27:16)
[PyPy 7.3.2-alpha0 with MSC v.1927 32 bit] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>> import hentai
>>>> doujin = hentai.Hentai(177013)
>>>> print(doujin.tags[0])
Tag(id=19018, type='tag', name='dark skin', url='/tag/dark-skin/', count=21056)
>>>> doujin.tags[0].id
19018
>>>> type(doujin.tags[0])
<class 'hentai.hentai.Tag'>
Describe the solution you'd like
As Python 3.7 has dataclasses inbuilt, there is no need for a backport of dataclasses and supporting Python 3.7 would only require the following patch:
- assert sys.version_info.minor > 7
+ assert sys.version_info.minor >= 7
- raise RuntimeError("Hentai requires Python 3.8+!")
+ raise RuntimeError("Hentai requires Python 3.7+!")
Is your feature request related to a problem? Please describe.
As it stands now, the unit test only check if the test file's meta data are in sync with https://nhentai.net.
Describe the solution you'd like
Rewrite everything using patch
from unittest.mock
this time.
Describe alternatives you've considered
Additional context
See also: unittest.mock
The problem happens when I try to use the library in a system that does not have write permission to /var/log
the issue happens when assigning the file handler for logger
Add the ability to change the logging file location or disable logging completely
A clear and concise description of any alternative solutions or features you've considered.
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe.
Implement a function that stores data as a JSON file to disk for processing in other languages.
Describe the solution you'd like
from hentai import Hentai
popular_loli = Hentai.search_all_by_query('tag:loli')
# dest should default to CWD
# keyword arguments narrow down the properties to store
# Hentai.All should save all properties bound to a Hentai object
Hentai.export(popular_loli, dest='./target/path/filename.json', options=[Hentai.ID, Hentai.UploadDate])
This function should also work with collections coming from
search_by_query
get_homepage
browse_homepage
Hentai
objectDescribe alternatives you've considered
If there is an easier way to specify which properties to include (like Hentai.ID
, etc.), go for it. Either way, this function should be able to to influence the result of the outcome in order to reduce the final file size, but also to make it easier to inspect the data later on by dismissing unnecessary information upfront.
Example 1
>>> from hentai import Hentai
>>> doujin = Hentai(177013)
>>> # save doujin.json to disk
>>> doujin.export()
Example 2
>>> from hentai import Hentai
>>> doujin = Hentai(177013)
>>> # save doujin.json to disk
>>> doujin.export(options=[Hentai.ID, Hentai.UploadDate, Hentai.NumFavorites])
Example 1
{'id': 177013, 'media_id': '987560', 'title': {'english': '[ShindoLA] METAMORPHOSIS (Complete) [English]', ...
Example 2
{'id': 177013, 'upload_date': '2016-10-18 14:28:49', 'num_favorites': 44548}
Lists of doujins coming from search_by_query
or get_homepage
should export the information using the same format wrapped around a list, e.g.
doujins = Hentai.get_homepage()
Hentai.export(doujins, options=[Hentai.ID, Hentai.UploadDate, Hentai.NumFavorites])
[
{
'id': 177013,
'upload_date': "2016-10-18 14:28:49",
'num_favorites': 44548
},
{
'id': 269582,
'upload_date': "2019-04-20 14:54:07",
'num_favorites': 6163
},
...
]
Additional context
See also the current contributing guidelines and the project's wiki. Don't forget to add a unit test to the test suite at hentai/test_hentai
once everything is said and done.
from hentai import Hentai
doujin = Hentai(177013)
print(Hentai.exists(doujin.id))
| OS Name | Python Version | App Version |
|:---------- --:|:-----------------:|:-----------:|
| Windows
| 3.10.2
| 3.2.10
|
Traceback (most recent call last):
File "C:\Python310\lib\site-packages\requests\adapters.py", line 440, in send
resp = conn.urlopen(
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
[Previous line repeated 2 more times]
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 866, in urlopen
retries = retries.increment(method, url, response=response, _pool=self)
File "C:\Python310\lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\xxx\Desktop\hentai scrping\hentai\test.py", line 3, in <module>
doujin = Hentai(177013)
File "C:\Python310\lib\site-packages\hentai\hentai.py", line 531, in __init__
self.__response = self.handler.get(self.api)
File "C:\Python310\lib\site-packages\hentai\hentai.py", line 479, in get
response = self.session.get(url, timeout=self.timeout, proxies=self.proxies or getproxies(), **kwargs)
File "C:\Python310\lib\site-packages\requests\sessions.py", line 542, in get
return self.request('GET', url, **kwargs)
File "C:\Python310\lib\site-packages\requests\sessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python310\lib\site-packages\requests\sessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:\Python310\lib\site-packages\requests\adapters.py", line 510, in send
raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))
I know that we all have a hard time scraping nhentai due to Cloudflare. Just wondering if there is a good solution now? Also is there any instruction on how to set up OAuth2 with this package?
Describe the bug
The make_request
paramter doesn't work if the CWD isn't set to the package's root directory.
To Reproduce
Steps to reproduce the behavior:
pip install hentai
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\hentai-chan\AppData\Roaming\Python\Python38\site-packages\hentai\hentai.py", line 630, in get_random_id
with open("./data/ids.csv", mode='r', encoding='utf-8') as file_handler:
FileNotFoundError: [Errno 2] No such file or directory: './data/ids.csv'
>>> from hentai import Utils
>>> Utils.get_random_id(make_request=False)
Additional context
Useful links that help you fix that error:
[1] Python Docs on importlib.resources
[2] Read the Docs for importlib_resources
The project's file system layout needs to adapt to these changes without breaking backwards compatibility. As laid down in commit c7046e0, this is just an experimental feature so this bug fix is a low priority on my bucket list. For anyone else interested in fixing this issue, feel free to have a shot at it!
I'm using version 3.2.10 and get an error while trying to interact with the api in any form.
Any interaction is failing which makes me think it's an API change or something.
`from hentai import Hentai as H
print(H(177013))`
| OS Name | Python Version | App Version |
|:----------:|:----------:|:--------------:|:-----------:|
| Windows
| 3.10.2
| 3.2.10
|
Traceback (most recent call last):
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 440, in send
resp = conn.urlopen(
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
return self.urlopen(
[Previous line repeated 2 more times]
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 866, in urlopen
retries = retries.increment(method, url, response=response, _pool=self)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Projects\discordbot\test.py", line 3, in <module>
print(H(177013))
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\hentai\hentai.py", line 531, in __init__
self.__response = self.handler.get(self.api)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\hentai\hentai.py", line 479, in get
response = self.session.get(url, timeout=self.timeout, proxies=self.proxies or getproxies(), **kwargs)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 542, in get
return self.request('GET', url, **kwargs)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:\Users\imagi\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 510, in send
raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))```
### Code samples
```cli
from hentai import Hentai as H
print(H(177013))
I'm expecting it to return a Hentai object.
When I try to run
from hentai import Hentai
The error occured
OS Name | Build No. | Python Version | App Version |
---|---|---|---|
Arch Linux |
5.10.30-1-lts |
3.9.5 |
`` |
```bash
Traceback (most recent call last):
File "/usr/lib/python3.9/pkgutil.py", line 416, in get_importer
importer = sys.path_importer_cache[path_item]
KeyError: PosixPath('/home/lordronz/.local/lib/python3.9/site-packages/faker/providers')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/lordronz/.local/lib/python3.9/site-packages/hentai/__init__.py", line 5, in <module>
from .hentai import *
File "/home/lordronz/.local/lib/python3.9/site-packages/hentai/hentai.py", line 45, in <module>
from faker import Faker
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/__init__.py", line 1, in <module>
from faker.factory import Factory
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/factory.py", line 7, in <module>
from faker.config import AVAILABLE_LOCALES, DEFAULT_LOCALE, PROVIDERS
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/config.py", line 11, in <module>
PROVIDERS = find_available_providers(
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/utils/loading.py", line 57, in find_available_providers
for mod in list_module(providers_mod) if mod != '__pycache__'
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/utils/loading.py", line 35, in list_module
return [name for _, name, is_pkg in pkgutil.iter_modules([path]) if is_pkg]
File "/home/lordronz/.local/lib/python3.9/site-packages/faker/utils/loading.py", line 35, in <listcomp>
return [name for _, name, is_pkg in pkgutil.iter_modules([path]) if is_pkg]
File "/usr/lib/python3.9/pkgutil.py", line 130, in iter_modules
for i in importers:
File "/usr/lib/python3.9/pkgutil.py", line 420, in get_importer
importer = path_hook(path_item)
File "<frozen importlib._bootstrap_external>", line 1601, in path_hook_for_FileFinder
File "<frozen importlib._bootstrap_external>", line 1476, in __init__
File "<frozen importlib._bootstrap_external>", line 177, in _path_isabs
AttributeError: 'PosixPath' object has no attribute 'startswith'
from hentai import Hentai
Successfully imported
None
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.