Git Product home page Git Product logo

iwara-dl's Introduction

iwara-dl

This program downloads iwara.tv videos

Due to the Recent Iwara website update, This repo can support the following function

Update on 07/31/2023

To-do list:

  • Download by id
  • Login
  • Download by user
  • Updater V1
  • Download by playlist
  • Updater V2
  • Download by subscription Please check later for the updated features.

Dependency

bash
curl
jq
nkf

Usage:

usage: iwara-dl.sh [-u [U]] [-p [P]] [-i [n]] [-rhftcsdn] [-F [M]] [-l [f]] [url [url ...]]

positional arguments:
  url

optional arguments:
  -h --help                 show this help message and exit
  --username [U]            username
  --userpass [P]            password
  --load-ignore-list [File] load the list in file that should not download
  -r --resume               try resume download
  --retry [count]           Max time to retry the download fail
  --user                    treat input url as usernames
  --quiet-mode              quiet mode
  --login                   log in upfront
  --accept-insecure         accept insecure https connection
  --name-only               output downloaded file name only(hides curl download bar)

  --shallow-update          only download users first page
  --updater-v1              cd to each username folder; update each folder;
  --updater-v2              create ./dl/ folder and update;

  --rm [File]               add a name to iwara ignore list and delete the file

  --follow  [M]             Download videos of people you are following. M:MaxPage
  --dl-list [f]             Download using the VideoID in the [F] VideoID List file.

extra:
  .iwara_ignore file => newline-saperated list of filenames of skipping download
  dl/.iwara_downloaded file => newline-saperated list of VideoID of skipping download
# Download. Video page url, Video id, playlist, and user page are supported
iwara-dl.sh https://ecchi.iwara.tv/videos/ooxxzz

# You can also use env to set your login cred
IWARA_USER="Jacky" IWARA_PASS="password" iwara-dl.sh videoid

iwara-dl's People

Contributors

amamiya-yuuko-1225 avatar hare1039 avatar momoandbanana22 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

iwara-dl's Issues

Failed download, returning error: 404

When I try to download all videos from a user, I get this error "curl: (22) The requested URL returned a error: 404"

When I try to play the video in the Iwara website, I get an error saying "The media could not be loaded, either because the server or network failed or because the format is not supported."

I am running the script in Windows using WSL2 Ubuntu

Is this the website's fault?

What is the license for this software?

I want to add a feature to this wonderful software. But I am not sure about the license of this software. So, I don't know if it is OK to add features.

Random videos came up when downloading videos from users

This bug cannot be consistently reproduced, but I have encountered this for several times.

屏幕截图 2023-11-13 121950

However, I didn't find anything wrong as I went through the related code.

I would be greatly appreciated if you could solve this problem😭

Thanks a lot.

Best regards,

Download different users' videos to different dirs

Hi,
When I download several different users' videos in a single command like this:
./iwara-dl.sh --user aaa bbb ccc ddd
the downloaded videos are not grouped by dirs.
Is there a way to tackle with the problem? If not, since I've written a script to group them using the terminal output info, I may try to integrate it into the project in the future.
Best regards

Curious about if it is legal to crawl videos on Iwara in the US

I am also considering writing one myself and adding more handy features. I saw you are living in the US and wrote this crawler. Anyone has reached out to you and alarmed you? Or any form of bad things ever happened to you?

Thank you so much for your kind help!

Error when running iwara-dl

Firstly, Is it fine to run this using Git Bash for Windows? I tried executing the command as follows:
sh iwara-dl.sh [Playlist URL]

to download a playlist but it returns with an error
./iwaralib.sh: line 11: `calc-argc': not a valid identifier

So I checked the file using notepad and changed - to _ for all the functions until it stopped throwing that error

Then when I try running it, I get the following error
iwara_dl.sh: line 142: calc-argc: command not found
Missing [urls/ids]

I tried sh iwara-dl.sh url [Playlist URL] & sh iwara-dl.sh -url [Playlist URL] but the issue still persists...

I'm new to coding so apologies if this problem sounds very basic :)

PS/Edit: Unrelated to this, but do u have any solution for videos in playlists just randomly disappearing? I had around 140 but now it only shows 30 :(

New Iwara site

iwara has a new website. Therefore, login/download seems to be no longer available with this script (iwara-dl).

Script interface is broken. Need refractor for interface

I want to make sure the interfaces are similar to other dl tools. Like youtube-dl url => output one file megadl 'path' => write a file, not in a ./dl/ directory. It actually breaks my daily updater after I fetch the new master branch...

So recently, I think we will refractor the getopts part to make the user interface more intuitive. (e.g --opts value etc)

If you have any suggestions, please write below. @momoandbanana22

Currently, I think in the next version, we can

iwaradl.sh 
  -h --help       => show this help message and exit
  --username [U]  => username
  --userpass [P]  => password
  --load-ignore-list [File] => load the list of files that should not download
  -r --resume     => try resume download
  --retry [count] => Max time to retry the download fail
  --user          => treat input url as usernames
  --quiet-mode    => quiet mode
  --name-only     => output downloaded file name only(hides curl download bar)

  --shallow-update => only download users first page
  --updater-v1     => cd to each username folder; update each folder;
  --updater-v2     => create ./dl/ folder and update;

  --rm [File]      =>  add a name to iwara ignore list and delete the file

  --follow [M]  => Download videos of people you are following. M:MaxPage
  --dl-list [f] => Download using the VideoID in the [F] VideoID List file.

Question about the 'retry' argument

Hi,
Due to the usage part in readme, there is a retry argument like this:
--retry [count] Max time to retry the download fail
However, the count parameter is not accepted, since the code in iwara-dl.sh reads like:
--retry ) export IWARA_RETRY="TRUE"; shift; ;;
and in func iwara-dl-retry-dl() in iwaralib.sh:
if [[ "$IWARA_RETRY" != "FALSE" ]] ; then
It seems that the [count] parameter is not supported yet, so maybe it's better to modify the help info about [retry].
Best regards

Problem about checking downloaded files' integrity

As I noticed, the script uses the http status code returned from cURL to identify whether a file is successfully downloaded.

if [[ "$http_return_code" == "416" ]] || [[ "$http_return_code" == "200" ]]; then
    add-downloaded-id "$videoid";

However, when cURL ends up with an error
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
It returns 200 even the file is not successfully downloaded.

屏幕截图 2023-11-12 233513

I am not very familiar with shell scripts and commands, so I am wondering if you have a better way to solve this problem.

As for now , I mannually copy the progress bar of cURL and check download integrity, since I don't know how to read the progress bar in commands or write them to files. It maybe a feasbile way to solve this issue if you can get the message in progress bar.

What's more, the '.file.size' argument in $video_stat got in function iwara-dl-by-videoid is not reliable. I don't know why there is difference between the actual size and the one provided by API. Maybe it's useful if you know the correlation between these 2 figures.

Thanks a lot.

Best regards,

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.