Git Product home page Git Product logo

Comments (15)

SINC-G avatar SINC-G commented on August 20, 2024

这边测试正常。

授权信息没获取到,可能是网络问题?可以再尝试多运行几次。

from bilibili2bangumi.

CberYellowstone avatar CberYellowstone commented on August 20, 2024

这边测试正常。

授权信息没获取到,可能是网络问题?可以再尝试多运行几次。

建议加一个手动填写code的功能

from bilibili2bangumi.

SINC-G avatar SINC-G commented on August 20, 2024

建议加一个手动填写code的功能

一开始用的输入,现在监听回调不是更好吗

from bilibili2bangumi.

CberYellowstone avatar CberYellowstone commented on August 20, 2024

建议加一个手动填写 code 的功能

一开始用的输入,现在监听回调不是更好吗

两个一起可以排除到底是哪个环节问题

from bilibili2bangumi.

CberYellowstone avatar CberYellowstone commented on August 20, 2024

@SINC-G 上述bug确认是缓降问题,复现条件不明
新的bug:

** 异常被引发!
** aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://api.bgm.tv/subject/26449')
** 3 次尝试均失败,退出!
Traceback (most recent call last):
File ".\bili2bgm.py", line 44, in
loop.run_until_complete(main())
File "C:\Users\28448\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File ".\bili2bgm.py", line 38, in main
await get_and_update(bili2bgm_map, bili_auth_data, BILI_UID, bgm_auth_data)
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 241, in get_and_update
await data.update_bgm_data_task
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 137, in update_bgm_data
await gather(*data.update_one_bgm_data_tasks)
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 70, in update_one_bgm_data
lambda: client.get(
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\utilities.py", line 94, in try_for_times_async_json
depth=depth+1
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\utilities.py", line 69, in try_for_times_async_chain
result = await f(result)
File "C:\Users\28448\AppData\Local\Programs\Python\Python37\lib\site-packages\aiohttp\client_reqrep.py", line 1103, in json
headers=self.headers,
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://api.bgm.tv/subject/26449')

已经多次复现

from bilibili2bangumi.

SINC-G avatar SINC-G commented on August 20, 2024

@NKID00 麻烦过来看一下

** 异常被引发!
** aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://api.bgm.tv/subject/26449')
** 3 次尝试均失败,退出!
Traceback (most recent call last):
File ".\bili2bgm.py", line 44, in
loop.run_until_complete(main())
File "C:\Users\28448\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File ".\bili2bgm.py", line 38, in main
await get_and_update(bili2bgm_map, bili_auth_data, BILI_UID, bgm_auth_data)
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 241, in get_and_update
await data.update_bgm_data_task
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 137, in update_bgm_data
await gather(*data.update_one_bgm_data_tasks)
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\update.py", line 70, in update_one_bgm_data
lambda: client.get(
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\utilities.py", line 94, in try_for_times_async_json
depth=depth+1
File "C:\Users\28448\Administrator\Temp\Bilibili2Bangumi\utilities.py", line 69, in try_for_times_async_chain
result = await f(result)
File "C:\Users\28448\AppData\Local\Programs\Python\Python37\lib\site-packages\aiohttp\client_reqrep.py", line 1103, in json
headers=self.headers,
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://api.bgm.tv/subject/26449')

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

报用json解码html的一般都是被反爬挡了然后要人机验证之类的…

要么就是网站服务器抽了

我找找解决方案

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

试试看调低 CONNECTION_LIMIT_PER_HOST(每站点最大并发连接数)?

但是调低之后运行速度可能会慢一点

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

要不加个更友好的提示之类的?

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

报用json解码html的一般都是被反爬挡了然后要人机验证之类的…

要么就是网站服务器抽了

我找找解决方案

bgm的api用了cloudflare家自带反爬的cdn就很难受

还是遇到就弹个手动验证人机验证?

from bilibili2bangumi.

SINC-G avatar SINC-G commented on August 20, 2024

@NKID00 速度慢点可以,只要保证能正确迁移就行。

from bilibili2bangumi.

SINC-G avatar SINC-G commented on August 20, 2024

话说这反爬机制对我们这种更新也生效嘛,我们也没在爬数据啊,如果是还好,我们还可以用镜像站

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

写了一些调试代码试了下貌似高并发的情况下会被 503

在 503 的时候主动停一会应该就能解决问题了

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

现在遇到 HTTP 状态不对的时候不会出现令人费解的 Attempt to decode JSON ... 了而是 ** HTTP 状态 500 或者 ** HTTP 状态 503,别慌,稍等片刻即可

遇到 503 时还会主动阻塞消息循环 0.5 秒防止过度并发,就算并发数调到 150 都能正常运行了。

from bilibili2bangumi.

NKID00 avatar NKID00 commented on August 20, 2024

现在在 feat-#12 分支上新加了遇到响应异常时输出很多调试信息的功能,以后遇到令人费解的 Attempt to decode JSON ... 把调试输出打开再复现一遍就会有详细信息了。

from bilibili2bangumi.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.