Git Product home page Git Product logo

yt-channelarchiver's Introduction

Supported platform License

YouTube Channel Archiver

Bootstrap and ready to use script to archive all videos from YouTube channel including metadata like comments and live chats. NOW supporting Docker, see here.

Note: If you are here to archive the channel Rushia Ch. 潤羽るしあ. There is a third-party website which archives hololive channels and has all videos of her channel:

https://archive.ragtag.moe/channel/UCl_gCybOJRIgOXw6Qb4qJzQ

Quickstart

Tl;dr: On Debian & Ubuntu to archive a channel run this two lines (replace youtube link with target channel url):

sudo apt install -y python3 python-is-python3 python3-pip wget curl tar && curl -fsSL https://raw.githubusercontent.com/Synertry/YT-ChannelArchiver/main/deploy/ytca-setup.sh | bash
ytca.sh https://www.youtube.com/channel/UCl_gCybOJRIgOXw6Qb4qJzQ

Dependencies

  • Python3
  • yt-dlp
  • Your YouTube cookies

To fulfill dependencies follow Installation

Installation

Install all cli dependencies

sudo apt install -y python3 python-is-python3 python3-pip wget curl tar

Run the setup script

curl -fsSL https://raw.githubusercontent.com/Synertry/YT-ChannelArchiver/main/deploy/ytca-setup.sh | bash

Directory structure will be created and the main script ytca.sh added to your path

Cookies

Place your cookies in the yt-dlp folder in your home directory. DO NOT modify the name. It needs to be exactly youtube.com_cookies.txt, like /home/<user>/yt-dlp/youtube.com_cookies.txt

Read here for the browser plugin.

Usage

Use your desired link to channel

ytca.sh https://www.youtube.com/channel/UCl_gCybOJRIgOXw6Qb4qJzQ

If you have membership access to the specified channel add an member subcommand to also archive membership-only videos

ytca.sh member https://www.youtube.com/channel/UCl_gCybOJRIgOXw6Qb4qJzQ

If you do not have access to membership, then you will be redirected to /featured, which would likely mean duplicate videos

Docker

Instead of provisioning a whole vm/os or modifying your system with my script there is now a Docker way.

docker pull synertry/ytca:v1

Alternatively you can build the image yourself with, which may take about 5 minutes. Download the Dockerfile into a folder of your choosing (maybe not the default Downloads). In the shell in the same folder run:

docker build -t ytca .

You can then archive a channel without even entering the container with:

docker run --rm -v "./download:/root/yt-dlp/download" ytca https://www.youtube.com/channel/UCl_gCybOJRIgOXw6Qb4qJzQ

The -v flag with the download path is important to persist your downloads after container destruction.

Server

For unattended usage on servers I recommend screen or tmux and technical experience.

Uninstall

WARNING: This is destructive as it removes the python packages, without checking if they were installed beforehand.

Your files still in yt-dlp/download will be moved to /media/videos/YouTube while force overwriting. OS apt packages will not be removed.

curl -fsSL https://raw.githubusercontent.com/Synertry/YT-ChannelArchiver/main/deploy/ytca-uninstall.sh | bash

TODO

  • use config files
  • scaffold directory structure
  • install/setup script
  • central main script
  • url input checks
  • proper README
  • additional all-in-one config file
  • uninstall script
  • support membership-only download
  • code cleanup by looping
  • bash options and subcommands
  • dockerfile
  • playlists
  • featured
  • server background task (overkill?)
  • Windows support with powershell

More

As these scripts are prepared for more general user usage, it might seem bloated with all checks and comments. So see my orignal script for the unbloated version without conf-files: https://gist.github.com/Synertry/b2445850c09f59fd0838ab13e31e0f1b

There is a ytca-all-in-one.conf if you don't want three different conf files, but it does not embed subs.

Trivia

The termination of Uruha Rushias contract and deletion of her channel motivated me to write this. It took me about 10 hours to understand yt-dlp, bash script options and write the scripts + configs.

yt-channelarchiver's People

Contributors

synertry avatar

Stargazers

 avatar

Watchers

 avatar

yt-channelarchiver's Issues

Network Spikes

Postprocessing on per video basis creates network downtimes. Which is inefficient for high-bandwith >1GB/s

Either create parent process which spawns new processes or postpone postprocessing after batch download.

Live-Chat download is slow

Downloading the live-chat part slows down the whole script and delaying the next download e.g. membership too far ahead.

Either manage it with parent process which could solve #1 or seperate it completely (again).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.