tbdsux / kuryana Goto Github PK
View Code? Open in Web Editor NEWA serverless web scraper api for MyDramaList.com
Home Page: kuryana.vercel.app
License: MIT License
A serverless web scraper api for MyDramaList.com
Home Page: kuryana.vercel.app
License: MIT License
Hi there,
I got the following error while loading content with MDL slug.
https://kuryana.vercel.app/id/716895-zui-shi-ren-jian-yan-huo-se
Can you help to take a look?
[ERROR] AttributeError: 'NoneType' object has no attribute 'text'
Traceback (most recent call last):
File "/var/task/vc__handler__python.py", line 305, in vc_handler
response = asgi_cycle(__vc_module.app, body)
File "/var/task/vc__handler__python.py", line 208, in call
loop.run_until_complete(asgi_task)
File "/var/lang/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/var/task/fastapi/applications.py", line 269, in call
await super().call(scope, receive, send)
File "/var/task/starlette/applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "/var/task/starlette/middleware/errors.py", line 184, in call
raise exc
File "/var/task/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/var/task/starlette/middleware/cors.py", line 84, in call
await self.app(scope, receive, send)
File "/var/task/starlette/exceptions.py", line 93, in call
raise exc
File "/var/task/starlette/exceptions.py", line 82, in call
await self.app(scope, receive, sender)
File "/var/task/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/var/task/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 670, in call
await route.handle(scope, receive, send)
File "/var/task/starlette/routing.py", line 266, in handle
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 65, in app
response = await func(request)
File "/var/task/fastapi/routing.py", line 231, in app
raw_response = await run_endpoint_function(
File "/var/task/fastapi/routing.py", line 160, in run_endpoint_function
return await dependant.call(**values)
File "./api/main.py", line 39, in fetch
code, r = await fetch_func(query=drama_id, t="drama")
File "/var/task/api/utils.py", line 45, in fetch_func
f._get()
File "/var/task/api/fetch.py", line 82, in _get
self._get_main_container()
File "/var/task/api/fetch.py", line 35, in get_main_container
container.find("div", class="show-synopsis")
Sometime I Got ValueError
File "/var/task/api/fetch.py", line 81, in _get self._get_main_container() File "/var/task/api/fetch.py", line 27, in _get_main_container (container.find("div", class_="col-film-rating").find("div").text) ValueError: could not convert string to float: 'N/A'
I was wondering if another endpoint for the api for the site's List pages, https://mydramalist.com/list/ (eg. https://mydramalist.com/list/1xrQlbW3) These can also be private and for people instead of drama's.
I think it would be a nice addition to the api, moreover me along with many people might have a use for it. Let me know if its possible from your side!
Trying to search for a drama but it's not available on MDL yet then it will return error:
500: INTERNAL_SERVER_ERROR
Code: FUNCTION_INVOCATION_FAILED
ID: sin1::hz578-1678765839345-26a0508481ef
[ERROR] AttributeError: 'NoneType' object has no attribute 'find'
Traceback (most recent call last):
File "/var/task/vc__handler__python.py", line 305, in vc_handler
response = asgi_cycle(__vc_module.app, body)
File "/var/task/vc__handler__python.py", line 208, in __call__
loop.run_until_complete(asgi_task)
File "/var/lang/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/var/task/fastapi/applications.py", line 269, in __call__
await super().__call__(scope, receive, send)
File "/var/task/starlette/applications.py", line 124, in __call__
await self.middleware_stack(scope, receive, send)
File "/var/task/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/var/task/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/var/task/starlette/middleware/cors.py", line 84, in __call__
await self.app(scope, receive, send)
File "/var/task/starlette/exceptions.py", line 93, in __call__
raise exc
File "/var/task/starlette/exceptions.py", line 82, in __call__
await self.app(scope, receive, sender)
File "/var/task/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/var/task/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 670, in __call__
await route.handle(scope, receive, send)
File "/var/task/starlette/routing.py", line 266, in handle
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 65, in app
response = await func(request)
File "/var/task/fastapi/routing.py", line 231, in app
raw_response = await run_endpoint_function(
File "/var/task/fastapi/routing.py", line 160, in run_endpoint_function
return await dependant.call(**values)
File "./api/main.py", line 39, in search
code, r = await search_func(query=query)
File "/var/task/api/utils.py", line 23, in search_func
f._get_search_results()
File "/var/task/api/search.py", line 85, in _get_search_results
title = result.find("h6", class_="text-primary title").find("a")
Hi any plans to add casts to the api?
Hey,
I cannot seem to make the search work, all I get is an empty array, no matter the search query I use.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.