Git Product home page Git Product logo

imageboard-downloader-rs's Introduction

Imageboard Downloader

imageboard-downloader-rs is a command-line multi image gallery downloader made in Rust with a very simple, yet extensible API.

It is a cross-platform tool with speed, simple cli interface and multiple simultaneous downloads as its main focus.

imageboard_downloader_rs has a hardcoded limit of 100 pages per download session to prevent API rate-limiting and put less strain on the imageboard's servers.

โš  Avoid downloading single tag selections that span ~100k posts alone without using the download limiter. Be reasonate!

Running example

Features

  • Multiple simultaneous downloads.
  • Authentication and user blacklist.
  • Download limit.
  • Custom websites support.
  • Global blacklist. See more
  • Store downloads in cbz file. See more

Installation

Currently, you can install the latest version using cargo or download from Releases

cargo install imageboard_downloader

Or by cloning this repository and building it yourself

git clone https://gitlab.com/FerrahWolfeh/imageboard-downloader-rs.git

cd imageboard-downloader-rs

cargo build --release

cargo run --release -- search "your_tag" "your_another_tag_(cool)" -o ~/

The final binary will be located at target/release/imageboard_downloader

Windows releases coming someday...

Usage

The utility has 3 main operating modes:

1. Tag Search

This mode is the former default mode of the utility, where it will fetch all posts with a tag-based search

cargo run --release -- search [OPTIONS] <TAGS>...

2. Post download

This mode is meant for downloading a single or a select few posts byt inputting their id

cargo run --release --  post [OPTIONS] <POST_IDS>...

3. Pool download

This mode is for downloading entire groups of organized posts (pools)

cargo run --release -- pool [OPTIONS] <POOL_ID>

Each mode has their own unique set of options, see more details with imageboard_downloader --help or cargo run --release -- --help.


Examples

Download images from danbooru with specified tags

imageboard_downloader search "skyfire_(arknights)"

In case you want to authenticate with danbooru or e621, use the --auth flag only once. Then all subsequent downloads will use authentication as well.


Download images starting from page 10

imageboard_downloader search "skyfire_(arknights)" -s 10

Download only images with "safe" rating from e621

imageboard_downloader search -i e621 "ash_(pokemon)" "pikachu" --safe-mode

Download images from rule34 with 20 simultaneous downloads

imageboard_downloader search -i rule34 -d 20 "moe"

Save downloaded images with their id instead of md5 as filename

imageboard_downloader search -i e621 "wolf" "anthro" --id

By default, the program will download files to your current dir. In case you want to download files to another place use:

imageboard_downloader "kroos_(arknights)" -o /any/other/dir

This will save files in /any/other/dir/<file>.png If the specified directory does not exist, it will be created.

Download posts with annotated tags

In order to download posts and save their tags along with them in a .txt file, just run the app like this:

cargo run --release -- post -o /whenever --annotate 123 456 69420

Inspiration and References

imageboard-downloader-rs's People

Contributors

ferrahwolfeh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

quackdoc

imageboard-downloader-rs's Issues

Problem with specific tag names.

Hello there, i'm trying to download "emilia_(re:zero)" tagged images from konachan, and it seems to not work because the filesystem doesn't support the ":" character in folder names.

Program crashes after finding 201,000 posts

The error message:

Found 201000 posts
Error: Connection Error
Caused by:
0: error decoding response body: expected value at line 1 column 1
1: expected value at line 1 column 1

I'm curious about the purpose of pre-finding the posts because it makes the command take a few minutes to start downloading with tags that have a large amount of posts, and if it's over 200,00 it just crashes apparently

REQ: Allow custom Board server / URL

Since there are a lot of image boards around but most of them share the same board software as others, it would be really nice if we could do something like imageboard_downloader -s moebooru -i https://yande.re ... this would help a lot instead of needing to hard code many sites that may not be popular.

as for custom syntax, maybe regex?

Instead of hoping it works out well, perhaps a config file might be better and more flexible?

#based on konachan
[yandere]
server = "moebooru"
useragent = ""
extractor_user_agent = ""
base_url = "https://yande.re"
post_url = ""
post_list_url = "https://yande.re/post.json"
pool_idx_url = ""
max_post_limit = 100
auth_url = ""

Empty output folder when using --cbz

Running imageboard_downloader --cbz --pool 6837 -i e621 -o . leads to all the images being downloaded and an output folder called e621 created, but once finished the folder is empty with no .cbz file to be found

image

Download e621 pools with correct page order

Hello! I'm not sure if this is out of scope for this project since it's designed to work with many imageboards, but I'm looking for a tool that can download comics (pools) from e621 (and zip it up as a .cbz) with the correct page order.

Currently trying this with pool:<id> as the tag will download a pool but all of the pages will be out of order and it's clearly not meant for this use case.

Thank you for making this tool regardless, I'll find it very useful!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.