Git Product home page Git Product logo

pixiv.ts's Introduction

About

This is a wrapper for the Pixiv API that includes typings and various utility functions to make getting content from pixiv a lot easier. You can also mass download illusts and download ugoiras as gifs!

Insall

npm install pixiv.ts

Getting Started

In order to receive an access token from pixiv, you must login using your username and password. All subsequent logins after the first will be done using the refresh token you receive on first login, and it will be regenerated automatically whenever it expires.

Useful Links

Notes on password login

Pixiv stopped supporting user/password logins, so you can only login with refresh token for now. Follow this guide to obtain your refresh token, and login instead with await Pixiv.refreshLogin(process.env.PIXIV_REFRESH_TOKEN).

Searching for illusts, novels, and manga

import Pixiv from "pixiv.ts"

async function useAPI() {
    /*Logging in is an asynchronous function. Don't try to use the constructor, all the properties will be undefined!*/
    const pixiv = await Pixiv.login(process.env.PIXIV_USERNAME, process.env.PIXIV_PASSWORD)

    /*If you wish, you can regenerate and return your refresh token manually if it has expired*/
    const refreshToken = await pixiv.refreshToken()

    /*You can get an illust very easily with it's url or id. Most endpoints will have a get() method
    that will parse the id out of the url automatically.*/
    const illust = await pixiv.illust.get("https://www.pixiv.net/en/artworks/76833012")

    /*You could also get the most bookmarked illust from the query. This uses search internally, so you can
    specify the parameters in the second argument.*/
    const shortcut = await pixiv.illust.get("gabriel", {r18: true})

    /*To parse the id out of any url, you can use util.parseID()*/
    const id = await pixiv.util.parseID("https://www.pixiv.net/en/artworks/75788934") //75788934

    /*You can search illusts with a query. The nextURL is stored in pixiv.search.nextURL.*/
    let illusts = await pixiv.search.illusts({word: "gabriel dropout"})
    /*There is also an utility to sort by most bookmarked.*/
    illusts = pixiv.util.sort(illusts)

    /*Filter parameters: en to search for english tags, type to filter by type, r18 to filter r18 illusts,
    and bookmarks to filter by minimum bookmarks. By default tags are translated to japanese, but you can change
    that behavior by changing en to true.*/
    const filteredSearch = await pixiv.search.illusts({word: "megumin", r18: true, type: "illust", bookmarks: "100"})
    const englishSearch = await pixiv.search.illusts({word: "cute", en: true})

    /*You can also search through the rankings, popular previews, etc.*/
    const rankings = await pixiv.illust.ranking({mode: "day_r18"})
    const popularPreviews = await pixiv.illust.popularPreview({word: "sagiri izumi"})

    /*And get all the illusts from a user.*/
    const userIllusts = await pixiv.user.illusts({user_id: 18590546})

    /*Getting novels is practically identical to illusts. The alternative to the get() method is
    to query the api for the details directly.*/
    const novel = await pixiv.novel.detail({novel_id: 11826198}).then((n) => n.novel)

    /*Novels obviously have text, and you can retrieve it with the text() method.*/
    const text = await pixiv.novel.text({novel_id: 11826198}).then((n) => n.novel_text)

    /*There is also manga, which 90% of the time will have multiple pages. You can get the
    urls of all the pages with the getPages() method.*/
    const manga = await pixiv.manga.get("https://www.pixiv.net/en/artworks/77333204")
    const pages = await pixiv.manga.getPages(manga)
}
useAPI()

Searching for users, bookmarks, tags, and articles

async function useAPI() {
    /*Again, you can use get() on the user class.*/
    const user = await pixiv.user.get("https://www.pixiv.net/member.php?id=35096162")

    /*You can also search for users using a query.*/
    const users = await pixiv.search.users({word: "kawaii"})

    /*You can retrieve a lot of info on bookmarks with the api, such as 
    the details, tags, and ranges.*/
    const bookmarkDetails = await pixiv.illust.bookmarkDetail({illust_id: 75788934}).then((b) => bookmark_detail)
    const bookmarkTags = await pixiv.illust.bookmarkTags().then((b) => b.bookmark_tags)
    const bookmarkRanges = await pixiv.illust.bookmarkRanges({word: "cute"}).then((b) => b.bookmark_ranges)

    /*Of course, you can also get all of the bookmarks of a user.*/
    const bookmarks = await pixiv.user.bookmarksIllust({user_id: 21479436})

    /*To get articles from pixiv vision, you can use the spotlight endpoint.*/
    const articles = await pixiv.spotlight.articles()
}

Downloading Illusts and Converting Ugoiras to gifs

async function useAPI() {
  /*You can download any illust locally with the function downloadIllust().*/
  await pixiv.util.downloadIllust("https://www.pixiv.net/en/artworks/72668134", "./illust", "large")

  /*One of my personal favorite methods is mass-downloading illusts and mapping them into separate
  folders based on the tags that they have.*/
  await pixiv.util.downloadIllusts("black tights", "./illust", "large", [{folder: "stockings", tag: "black tights"}])

  /*You can retrieve all of the metadata for a ugoira, including all of the image frames, the delay
  between each frame, and the url for the zip download.*/
  const metadata = await pixiv.ugoira.metadata({illust_id: 77329939})

  /*Using the above endpoint internally, there is an utility to download the zip file and extract
  it to a local path automatically.*/
  await pixiv.util.downloadZip("https://www.pixiv.net/en/artworks/77359698", "./ugoira")

  /*A ton of png/jpg files are not that useful... which is why you can also convert and download a 
  ugoira as a gif! This uses downloadZip() and encodeGif() internally. The third parameter has some
  options for speed and whether or not it's played in reverse.*/
  await pixiv.util.downloadUgoira("https://www.pixiv.net/en/artworks/68064543", "./ugoira", {speed: 1.0, reverse: false})
}

Obtaining subsequent API results

async function useAPI() {
  /*You can obtain all subsequent search with util.multiCall(). The optional limit specifies how many extra api calls to make.*/
  let limit = 100
  let illusts = await pixiv.search.illusts({word: "word"})
  if (pixiv.search.nextURL) illusts = await pixiv.util.multiCall({next_url: pixiv.search.nextURL, illusts}, limit)
}

Common Parameters

  • illust_id: ID of the illust.
  • user_id: ID of the user.
  • novel_id: ID of the novel.
  • series_id: ID of the series.
  • word: The search query to search.
  • type: The type of content to search: "illust" | "novel" | "manga" | "ugoira"
  • restrict: Restricts the bookmarks you search: "public" | "private" | "all"
  • search_target: The matching options in the search endpoint: "partial_match_for_tags" | "exact_match_for_tags" | "title_and_caption"
  • mode: For searching rankings, either: "day" | "week" | "month" | "day_male" | "day_female" | "week_original" | "week_rookie" | "day_r18" | "day_male_r18" | "day_female_r18" | "week_r18" | "week_r18g" | "day_manga" | "week_manga" | "month_manga" | "week_rookie_manga" | "day_r18_manga" | "week_r18_manga" | "week_r18g_manga"
  • duration: Relative search duration: "within_last_day" | "within_last_week" | "within_last_month"

Common Types

PixivIllust
export interface PixivIllust {
    id: number
    title: string
    type: string
    image_urls: {
      square_medium: string
      medium: string
      large?: string
    }
    caption: string
    restrict: number
    user: PixivUser
    tags: PixivTag[]
    tools: string[]
    create_date: string
    page_count: number
    width: number
    height: number
    sanity_level: number
    meta_single_page: {
      original_image_url?: string
    }
    meta_pages: PixivMetaPage[]
    total_view: number
    total_bookmarks: number
    is_bookmarked: boolean
    visible: boolean
    x_restrict: number
    is_muted: boolean
    total_comments: number
  }
PixivUser
export interface PixivUser {
    id: number
    name: string
    account: string
    profile_image_urls: {
      medium: string
    }
    comment: string
    is_followed: boolean
}
PixivUserDetail
export interface PixivUserDetail {
    user: PixivUser
    profile: {
      webpage: string
      gender: string
      birth: string
      birth_day: string
      birth_year: number
      region: string
      address_id: number
      country_code: string
      job: string
      job_id: number
      total_follow_users: number
      total_mypixiv_users: number
      total_illusts: number
      total_manga: number
      total_novels: number
      total_illust_bookmarks_public: number
      total_illust_series: number
      background_image_url: string
      twitter_account: string
      twitter_url: string
      pawoo_url: string
      is_premium: boolean
      is_using_custom_profile_image: boolean
    }
    profile_publicity: {
      gender: string
      region: string
      birth_day: string
      birth_year: string
      job: string
      pawoo: boolean
    }
    workspace: {
      pc: string
      monitor: string
      tool: string
      scanner: string
      tablet: string
      mouse: string
      printer: string
      desktop: string
      music: string
      desk: string
      chair: string
      comment: string
      workspace_image_url: string | null
    }
}
PixivComment
export interface PixivComment {
    id: number
    comment: string
    date: string
    user: PixivUser
    parent_comment: PixivComment
}

pixiv.ts's People

Contributors

chinhongtan avatar dependabot[bot] avatar diamond-frost avatar rayriffy avatar sotrx avatar tenpi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

pixiv.ts's Issues

util.multiCall seems to be not working

I'm not very sure if I am using it wrongly, here's my code snippet:

const pixiv = await Pixiv.default.refreshLogin(refreshToken);
const word = 'ロリ';
let illusts = await pixiv.search.illusts({ word: word, r18: false, type:'illust', bookmarks: '1000' });
if (pixiv.search.nextURL) illusts = [...illusts, ...await pixiv.util.multiCall({ next_url: pixiv.search.nextURL, illusts }, 2)];

I noticed that no matter what number I use for limit (which is 2 in the code above), it always ends up returning Error: Request failed with status code 400. If no limit is provided, it will end up returning Error: Request failed with status code 403.

Also, using await Pixiv.refreshLogin(refreshToken) like the example in the readme.md will not work. An error will pop up saying TypeError: Pixiv.refreshLogin is not a function. It took me quite a while to figure out that I have to use Pixiv.default.refreshLogin instead.

login invalid_grant error

Tried to login but I get an error:
image

Relevant code:

import Pixiv from 'pixiv.ts';
import { env } from './environment';

let client: Pixiv;

export async function usePixivClient() {
  if (!client) {
    client = await Pixiv.login(env.pixiv.username, env.pixiv.password);
  }

  async function getUserIllustrations(params: Parameters<typeof client['user']['illusts']>[0]) {
    return await client.user.illusts(params);
  }

  return {
    getUserIllustrations,
  };
}

Third time's the charm. await client.util.downloadIllust does not wait

I literally just took your example with the hashes and it doesn't work.

await client.util.downloadIllust(url, "./downloads", "original");

const files = fs.readdirSync(path.join(__dirname, "../downloads")).map((p) => {
    return path.join(__dirname, "../downloads", p)
})
const hash = crypto.createHash("md5").update(fs.readFileSync(files[0])).digest("hex")
console.log(hash);
for(let i = 1; i < 5; i++) {
    setTimeout(() => {
        const hash2 = crypto.createHash("md5").update(fs.readFileSync(files[0])).digest("hex")
        console.log(hash2)
        console.log(hash === hash2)
    }, i * 1000)
}

the output is as follows

2a96de86c25cd98a85c89debfed3fe4e
7cbe45c3c4fdf5cf4c3cd0ae78ee1a1b -- after a second
false
aad8d30495ccc35463511dc27bd0e52d -- after 2 seconds
false
f56fb56448ec3df54eb0986a81ce0554 -- after 3 seconds
false
4528e847c998aa0617f56412ca11a624 -- after 4 seconds
false
25cd98a8aaaad7f56412e4e2a96de86c -- after 5 seconds
false

and also, i get this

please dont close the damn issue, let's discuss this, I just don't see where this is my problem

here is the post I tried it on: https://www.pixiv.net/en/artworks/98079929

It contains 3 images by the way

await util.downloadIllust does not work

Basically I want to download images to a folder and after they are downloaded, convert and move them to another folder.

This does not work and crashed with [Error: VipsJpeg: Premature end of input file] because files didn't finish downloading yet

const illust = await client.illust.get(url);
const img_dir = "./downloaded";
await sendToChannel(channel, `📥 Downloading from pixiv`);
await client.util.downloadIllust(illust, img_dir, "original");

let images = walk(img_dir);

for (const image of images) {
    const new_file_name = getFileName(image);
    const new_file_path = await getFilePath(new_file_name, channel);
    if(new_file_path) {
    await processAndSaveImage(fs.createReadStream(image));
    fs.unlinkSync(image);
}

However, if i add sleep(5000); before walk(img_dir); it works, which means that await client.util.downloadIllust(illust, img_dir, "original"); does not actually wait untill all images are saved

reopen. client.util.downloadIllust does not wait

I am reopening this issue because the previous one was closed but the issue is still there.

Here is what i came up with to 'solve' the issue

const img_dir = "./downloaded";
await client.util.downloadIllust(url, img_dir, "original");

// should be downloaded by now
let exit = false;
let prev_hash = "";
let images: string[] = [];
while(true) {
    let images_stats = fs.readdirSync(img_dir);
    let curr_hash = "";
    images = [];
    for (const image_stats of images_stats) {
        const file = img_dir + "/" + image_stats;
        images.push(file);
        curr_hash += await getFileHash(file);
    }
    if(exit) {
        break;
    }
    if (prev_hash != curr_hash || !curr_hash.length) {
        debug(`hash mismatch, waiting... (${prev_hash} || ${curr_hash})`);
    }
    exit = prev_hash == curr_hash && curr_hash.length > 0;
    prev_hash = curr_hash;
    await sleep(1000);
}

// by this point the images are downloaded, but the solution is mega-crap

here is the output

hash mismatch, waiting... (c397c6d991c6ddc033cc089b33a412355 || af1349b9f5f9a1a6a0404dea36dcc9499)
hash mismatch, waiting... (af1349b9f5f9a1a6a0404dea36dcc9497 || efb763f51eb0780232d81f86cdc86135c)
hash mismatch, waiting... (a2344fc328fe26a3ff5f241c10ba91071 || 9183ef0552ac039c012bec3885eb85d79)
hash mismatch, waiting... (2b90f3912a26806e03a96f0a107870f16 || 015537fcb4eee269792ed88ad1d7bdb84)
hash mismatch, waiting... (b90f3912a26806e03a96f0a107870f167 || 12a26806e03a96f0a107870f167258f9a)

as you can see, the hash changes which means that the files are still downloading, please don't close this issue blaming me, this is clearly a problem in the library

Log in with cookies?

Can I log in with cookies (PHPSESSID) that I could get from pixiv page?

I tried using Pixiv.refreshLogin() with my cookies as the parameter and it doesn't work.
Can we improve this method to support login with cookies too?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.