Git Product home page Git Product logo

kuryana's Introduction

kuryana

A simple MyDramaList.com scraper api.

This scrapes on the go so it might be slow.

THIS SERVICE IS ONLY CREATED TO SATISFY THE NEED FOR AN API FOR [MYDRAMALIST.COM](https://mydramalist.com). THIS WILL BE STOPPED ONCE AN OFFICIAL API WILL BE RELEASED.

Deploy Your Own

Deploy with Vercel

API Use

https://kuryana.vercel.app/search/q/{yourquery}
https://kuryana.vercel.app/id/{mydramalist-slug}
https://kuryana.vercel.app/id/{mydramalist-slug}/cast
https://kuryana.vercel.app/id/{mydramalist-slug}/reviews
https://kuryana.vercel.app/people/{people-id}
https://kuryana.vercel.app/seasonal/{year}/{quarter}
https://kuryana.vercel.app/list/{id}
https://kuryana.vercel.app/dramalist/{user_id}

Error Messages

// mainly on all endpoints except `search`
// sample: /list/unknown-random-id
{
  "code": 400,
  "error": true,
  "description": {
    "title": "This list is private.",
    "info": "You can see this page because the URL you are accessing cannot be found."
  }
}
// could also be this (only on `/search`) endpoint
{
  "error": true,
  "code": 404,
  "description": "404 Not Found"
}

Development

  • Minimum Python Version : 3.9,

  • Always make sure to create a virtualenvironment

    python -m venv venv
    source ./venv/bin/activate.sh # change depends on your shell and os
  • Install the dependencies

    pip install -r requirements.txt

Dev Server

You can start the development server in two ways:

  • Using the vercel CLI (localhost:3000)

    vercel dev
    • The api will be http://localhost:3000/api/*
  • Or with uvicorn (localhost:8000)

    uvicorn api.main:app --reload
    • The api will be http://localhost:8000/*

NOTE

All Requests and SCRAPED Datas are not cached by Vercel or the API itself.

© TheBoringDude

kuryana's People

Contributors

bifeldy avatar debakarr avatar leonguyen52 avatar renovate-bot avatar tbdsux avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

kuryana's Issues

500: INTERNAL_SERVER_ERROR

Example Link

Sometime I Got ValueError

File "/var/task/api/fetch.py", line 81, in _get self._get_main_container() File "/var/task/api/fetch.py", line 27, in _get_main_container (container.find("div", class_="col-film-rating").find("div").text) ValueError: could not convert string to float: 'N/A'

cast?

Hi any plans to add casts to the api?

500: INTERNAL_SERVER_ERROR while searching but no result

Trying to search for a drama but it's not available on MDL yet then it will return error:

500: INTERNAL_SERVER_ERROR
Code: FUNCTION_INVOCATION_FAILED
ID: sin1::hz578-1678765839345-26a0508481ef

[ERROR] AttributeError: 'NoneType' object has no attribute 'find'
Traceback (most recent call last):
  File "/var/task/vc__handler__python.py", line 305, in vc_handler
    response = asgi_cycle(__vc_module.app, body)
  File "/var/task/vc__handler__python.py", line 208, in __call__
    loop.run_until_complete(asgi_task)
  File "/var/lang/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/var/task/fastapi/applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "/var/task/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/var/task/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/var/task/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/var/task/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/var/task/starlette/exceptions.py", line 93, in __call__
    raise exc
  File "/var/task/starlette/exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "/var/task/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/var/task/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/var/task/starlette/routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "/var/task/starlette/routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "/var/task/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/var/task/fastapi/routing.py", line 231, in app
    raw_response = await run_endpoint_function(
  File "/var/task/fastapi/routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "./api/main.py", line 39, in search
    code, r = await search_func(query=query)
  File "/var/task/api/utils.py", line 23, in search_func
    f._get_search_results()
  File "/var/task/api/search.py", line 85, in _get_search_results
    title = result.find("h6", class_="text-primary title").find("a")

500: INTERNAL_SERVER_ERROR Code: FUNCTION_INVOCATION_FAILED

Hi there,

I got the following error while loading content with MDL slug.
https://kuryana.vercel.app/id/716895-zui-shi-ren-jian-yan-huo-se

Can you help to take a look?

[ERROR] AttributeError: 'NoneType' object has no attribute 'text'
Traceback (most recent call last):
File "/var/task/vc__handler__python.py", line 305, in vc_handler
response = asgi_cycle(__vc_module.app, body)
File "/var/task/vc__handler__python.py", line 208, in call
loop.run_until_complete(asgi_task)
File "/var/lang/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
return future.result()
File "/var/task/fastapi/applications.py", line 269, in call
await super().call(scope, receive, send)
File "/var/task/starlette/applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "/var/task/starlette/middleware/errors.py", line 184, in call
raise exc
File "/var/task/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/var/task/starlette/middleware/cors.py", line 84, in call
await self.app(scope, receive, send)
File "/var/task/starlette/exceptions.py", line 93, in call
raise exc
File "/var/task/starlette/exceptions.py", line 82, in call
await self.app(scope, receive, sender)
File "/var/task/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/var/task/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 670, in call
await route.handle(scope, receive, send)
File "/var/task/starlette/routing.py", line 266, in handle
await self.app(scope, receive, send)
File "/var/task/starlette/routing.py", line 65, in app
response = await func(request)
File "/var/task/fastapi/routing.py", line 231, in app
raw_response = await run_endpoint_function(
File "/var/task/fastapi/routing.py", line 160, in run_endpoint_function
return await dependant.call(**values)
File "./api/main.py", line 39, in fetch
code, r = await fetch_func(query=drama_id, t="drama")
File "/var/task/api/utils.py", line 45, in fetch_func
f._get()
File "/var/task/api/fetch.py", line 82, in _get
self._get_main_container()
File "/var/task/api/fetch.py", line 35, in get_main_container
container.find("div", class
="show-synopsis")

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.