glubsy / livestream_saver Goto Github PK

View Code? Open in Web Editor NEW

305.0 15.0 39.0 2.47 MB

Monitor a Youtube channel and download live-streams from the first segment

License: GNU General Public License v3.0

Python 97.38% Shell 2.35% Dockerfile 0.27%

livestream youtube channel-monitor stream-downloader

livestream_saver's People

Contributors

Stargazers

Watchers

livestream_saver's Issues

Pytube error: "get_throttling_function_name: could not find match for multiple"

Hello! I don't know if YouTube has changed something again?
I'm using the latest version & I get this error now as of today:

2021-08-26 19:35:51,525 - WARNING - download.TrR7hzSyXAs - An output directory already existed. We assume a failed download attempt. Last segment available was 1.
2021-08-26 19:35:51,526 - INFO - download.TrR7hzSyXAs - Will start downloading from segment number 0.
2021-08-26 19:35:51,836 - INFO - download.TrR7hzSyXAs - Stream seems to be viewed live. Good.
2021-08-26 19:36:01,884 - INFO - download.TrR7hzSyXAs - Stream status Status.OK
2021-08-26 19:36:02,589 - ERROR - livestream_saver - Got error in stream download but continuing...
get_throttling_function_name: could not find match for multiple

(I'm currently using a old version (5/26/2021) that's holding up in the mean time with no issues atm.)

Unable to start livestream_saver on Ubuntu

I'm getting this error (below) when i try to initialize it.

I followed the install instructions.

python3.8 livestream_saver.py monitor --help
Traceback (most recent call last):

File "livestream_saver.py", line 12, in
from livestream_saver import extract, util
File "/home/hobbes/livestream_saver.py", line 12, in
from livestream_saver import extract, util
ImportError: cannot import name 'extract' from partially initialized module 'livestream_saver' (most likely due to a circular import) (/home//hobbes/livestream_saver.py)

What I did do wrong?

Please, let me know

404 not found

Hello is this normal behavior?


2022-07-09 11:33:12,723 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="243"      mime_type="video/webm" res="360p"       fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,723 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="244"      mime_type="video/webm" res="480p"       fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,723 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="278"      mime_type="video/webm" res="144p"       fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,724 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="247"      mime_type="video/webm" res="720p"       fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,724 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="248"      mime_type="video/webm" res="1080p"      fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,724 - INFO - download.gTMzVB6p9JQ - Available <Stream:      itag="271"      mime_type="video/webm" res="1440p"      fps="30fps"     vcodec="vp9"    progressive="False"     type="video">
2022-07-09 11:33:12,724 - INFO - download.gTMzVB6p9JQ - Selected video <Stream: itag="137" mime_type="video/mp4" res="1080p" fps="30fps" vcodec="avc1.640028" progressive="False" type="video">
2022-07-09 11:33:12,724 - INFO - download.gTMzVB6p9JQ - selected audio <Stream: itag="140" mime_type="audio/mp4" abr="128kbps" acodec="mp4a.40.2" progressive="False" type="audio">
Downloading segment 630...2022-07-09 11:38:13,618 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 11:38:15,308 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 1310...2022-07-09 11:43:15,034 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 11:43:16,804 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 2060...2022-07-09 11:48:17,725 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 11:48:19,317 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 2810...2022-07-09 11:53:18,750 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 11:53:20,637 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 3560...2022-07-09 11:58:22,910 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 11:58:24,863 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 4440...2022-07-09 12:03:24,054 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 12:03:25,791 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 5430...2022-07-09 12:08:26,371 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 12:08:28,122 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 6430...2022-07-09 12:13:29,099 - INFO - download.gTMzVB6p9JQ - Forcing update of download URLs.
2022-07-09 12:13:30,859 - WARNING - download.gTMzVB6p9JQ - Could not find any stream descriptor in the response! Loaded streams from MPD instead.
Downloading segment 7179...2022-07-09 12:16:54,173 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:16:54,174 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7180...2022-07-09 12:16:57,460 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:16:57,460 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7180...2022-07-09 12:17:00,665 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:00,665 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7181...2022-07-09 12:17:03,982 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:03,982 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7182...2022-07-09 12:17:07,327 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:07,327 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7182...2022-07-09 12:17:10,428 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:10,428 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7183...2022-07-09 12:17:13,797 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:13,797 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7183...2022-07-09 12:17:16,980 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:16,980 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7184...2022-07-09 12:17:20,382 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:20,382 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7185...2022-07-09 12:17:23,730 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:23,730 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7186...2022-07-09 12:17:27,062 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:27,062 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7186...2022-07-09 12:17:30,290 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:30,290 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7187...2022-07-09 12:17:33,647 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:33,648 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7187...2022-07-09 12:17:36,779 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:36,780 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7188...2022-07-09 12:17:40,285 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:40,285 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7189...2022-07-09 12:17:43,714 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:43,714 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7189...2022-07-09 12:17:46,941 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:46,941 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7190...2022-07-09 12:17:50,359 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:50,359 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7191...2022-07-09 12:17:53,709 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:53,709 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7192...2022-07-09 12:17:57,078 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:17:57,078 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7192...2022-07-09 12:18:00,291 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:00,291 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7193...2022-07-09 12:18:03,683 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:03,684 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7193...2022-07-09 12:18:06,813 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:06,814 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 2/10)
Downloading segment 7194...2022-07-09 12:18:10,238 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:10,239 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7195...2022-07-09 12:18:13,580 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:13,580 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7196...2022-07-09 12:18:16,912 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-07-09 12:18:16,912 - WARNING - download.gTMzVB6p9JQ - Waiting for 3 seconds before retrying... (attempt 1/10)
Downloading segment 7196...2022-07-09 12:18:20,046 - CRITICAL - download.gTMzVB6p9JQ - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found

Slow download rate: Youtube throttling

Youtube throttles downloads from URL endpoints not signed with some kind of cryptographic token.

Pytube implemented a fix (pytube linked issue),

invidious implemented a fix,

yt-dlp issue about this topic.

Possible workaround but needs more testing: having the javascript youtube player active in a web browser (ie. the video page open, but the video itself probably does not need to be playing).

Possible solutions: currently attempting to merge some of Pytube methods in the hope to get a signed URL for every stream. Will it need to be refreshed? And how frequently? No idea yet.

Also looking into integrating yt-dlp and streamlink somehow but it will take time.

[feature request] Monitor Membership tab for streams not attached to posts

Not sure if this is a recent change on Youtube, but one channel that I have set for monitoring has recently posted some members-only streams that were not attached to any Community posts. Because of this, they do not show up in the Community tab at all, but do show up in the Membership tab as just a video/stream entry (like the kind you see if you scroll down any Membership tab for a bit). Due to this, the channel monitoring function was not able to see the streams to automatically download them.

I double checked to make sure it wasn't a cookie issue by replacing my cookie file, and I also still see older member-only streams (that are attached to Community posts) listed in the "Currently listed community videos" list. Also, if I use the plain Download function and manually specify the member-only video URL, the stream is seen and downloads fine.

From what I'm gathering, it seems a new function to monitor the Membership tab for new streams (not just ones attached to posts) might be needed to cover these cases.

ERROR - livestream_saver - apply_descrambler() takes 1 positional argument but 2 were given

Linux Mint 20.3, livestream_saver.py commit ee2a9fe

python3 livestream_saver.py download https://www.youtube.com/watch?v=QtBWg6j8yzs 2022-05-05 12:05:29,838 - INFO - livestream_saver.cookies - No cookie path submitted. Using a blank cookie jar. 2022-05-05 12:05:30,079 - WARNING - download.QtBWg6j8yzs - An output directory already existed. We assume a failed download attempt. Last segment available was 1. 2022-05-05 12:05:30,079 - INFO - download.QtBWg6j8yzs - Will start downloading from segment number 0. 2022-05-05 12:05:30,315 - INFO - download.QtBWg6j8yzs - Stream seems to be viewed live. Good. 2022-05-05 12:05:30,316 - INFO - download.QtBWg6j8yzs - Stream status Status.OK 2022-05-05 12:05:30,316 - ERROR - livestream_saver - apply_descrambler() takes 1 positional argument but 2 were given Traceback (most recent call last): File "livestream_saver.py", line 963, in main error = args["func"](config, args) File "livestream_saver.py", line 614, in download_mode dl.download(config.getfloat("download", "scan_delay", vars=args)) File "/home/ra/Local/Downloads/Direct/livestream_saver-main/livestream_saver/download.py", line 964, in download self.update_download_urls() File "/home/ra/Local/Downloads/Direct/livestream_saver-main/livestream_saver/download.py", line 853, in update_download_urls video_quality, audio_quality = self.get_best_streams( File "/home/ra/Local/Downloads/Direct/livestream_saver-main/livestream_saver/download.py", line 1246, in get_best_streams avail_streams = self.streams File "/home/ra/Local/Downloads/Direct/livestream_saver-main/livestream_saver/download.py", line 498, in streams query = pytube.StreamQuery(self.fmt_streams) File "/home/ra/Local/Downloads/Direct/livestream_saver-main/livestream_saver/download.py", line 1188, in fmt_streams pytube.extract.apply_descrambler(self.player_config_args, fmt) TypeError: apply_descrambler() takes 1 positional argument but 2 were given

I tried half a dozen livestreams that are currently broadcasting, with the same result.

Merge failed after download

I was running a download with python stream_downloader.py <youtube-link> and it failed after the download like this:

2021-04-14 21:11:01,865 - INFO - livestream_saver.util - GET https://www.youtube.com/watch?v=x
2021-04-14 21:11:02,233 - WARNING - download.x - The stream is not live anymore. Done.
2021-04-14 21:11:02,233 - INFO - download.x - Finished downloading x.
Traceback (most recent call last):
  File "stream_downloader.py", line 65, in <module>
    merge(info=dl.video_info, data_dir=dl.output_dir, delete_source=args.delete_source)
NameError: name 'merge' is not defined

Add Docker Support

This looks like an awesome project and one I would love to run on my server. The easiest way to do that is with a docker. Any chance you could create one?

Implement parallel monitoring

It would be nice to have the ability to monitor all [monitor] sections at the same time. This would prevent having to run several python interpreter instances at the same time. async would be the preferable method.

Raised JSONDecodeError: Extra data

The issue just started today. I tried multiple channels and confirmed the same issue. Basically JSON decoding is failed. Could someone confirm it?

Using the latest code and it runs on Windows.

F:\livestream_saver>python3 livestream_saver.py monitor --delete-source https://www.youtube.com/c/FoxNews
2022-04-25 22:52:33,827 - INFO - livestream_saver.cookies - No cookie path submitted. Using a blank cookie jar.
2022-04-25 22:52:34,014 - INFO - livestream_saver - Monitoring channel: FoxNews
2022-04-25 22:52:34,441 - CRITICAL - livestream_saver.extract - Error loading JSON from string: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:34,443 - CRITICAL - livestream_saver.monitor - Got an invalid upcoming JSON: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:34,444 - INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0

2022-04-25 22:52:34,873 - CRITICAL - livestream_saver.extract - Error loading JSON from string: Extra data: line 1 column 137908 (char 137907)
2022-04-25 22:52:34,874 - CRITICAL - livestream_saver.monitor - Got an invalid community JSON: Extra data: line 1 column 137908 (char 137907)
2022-04-25 22:52:34,874 - INFO - livestream_saver.monitor - Currently listed community videos: 0

2022-04-25 22:52:35,296 - CRITICAL - livestream_saver.extract - Error loading JSON from string: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:35,297 - CRITICAL - livestream_saver.monitor - Got an invalid public JSON: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:35,297 - INFO - livestream_saver.monitor - Currently listed public videos: 0

2022-04-25 22:52:35,301 - CRITICAL - livestream_saver.extract - Error loading JSON from string: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:35,302 - ERROR - livestream_saver - Extra data: line 1 column 234985 (char 234984)
Traceback (most recent call last):
  File "livestream_saver.py", line 485, in monitor_mode
    "Live videos found for channel "
  File "F:\livestream_saver\livestream_saver\monitor.py", line 45, in get_channel_name
    _json = self.public_json
  File "F:\livestream_saver\livestream_saver\monitor.py", line 98, in public_json
    self._public_json = extract.str_as_json(
  File "F:\livestream_saver\livestream_saver\extract.py", line 83, in str_as_json
    j = loads(string)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 234985 (char 234984)
2022-04-25 22:52:35,304 - INFO - livestream_saver.monitor - Sleeping for 18.40 minutes (1104.16 seconds)...
Traceback (most recent call last):
  File "livestream_saver.py", line 991, in <module>
    exit(main())
  File "livestream_saver.py", line 963, in main
    error = args["func"](config, args)
  File "livestream_saver.py", line 495, in monitor_mode
    wait_block(min_minutes=scan_delay, variance=3.5)
  File "F:\livestream_saver\livestream_saver\monitor.py", line 563, in wait_block
    sleep(wait_time_sec)
KeyboardInterrupt
^C

Missing Videos tab endpoint data for Livestream VODS

Looks like Youtube has added a new tab (LIVE) to store Livestream VODs....If the channel does not have any non live stream VOD videos, an exception is thrown in monitor.

Cant download 1440p60 quality video

I try to download stream and i got 1080p, in despite of the video has 1440p resolution max.

What is the problem? Is it possible to download better than 1080p ? Is it stricts of youtube or programm cant download?

If it need for debug: I'm downloading streams from this youtuber. He is removing his streams always.
https://www.youtube.com/channel/UC1XLQIzXJd_KaLOuEVFESVw

Fails to run on windows (setsid)

It failed to run on windows for me because the import from os import setsid doesn't exist. Simply removing that specific import makes it work fine with downloading and merging but it probably breaks something on another os?

Edit: It's imported in the main file but has no use there and it gets imported in the hooks.py file and used as preexec_fn=setsid

Unhandled exception: Missing streamingData key in json causing downloads to be interrupted

I've noticed this issue multiple times while downloading long streams recently. The download fails and goes back to monitoring.
PFA the relevant screenshot with the error.

Segment 0 missing from capture / doesn't get recorded

Hello! I noticed that segment '0' is missing / doesn't get recorded; resulting in the first 1,2 or 5 seconds missing from a live stream.
Tested with stream downloader & channel monitor. It should start at segment 0 when it starts at 1!

TypeError: apply_descrambler() takes 1 positional argument but 2 were given

I get this error upon trying to download/monitor for every release after the Jul 6, 2021 build.

My setup:

Windows 8.1 x64
Python 3.10.0
pytube 11.0.1
ffmpeg-4.4.1-full_build

Release/Build + Command:

livestream_saver-main (Jul 6, 2021)
python livestream_saver.py download https://www.youtube.com/watch?v=-bvg9HHONTI

RESULTS:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\USER\Downloads\APPS\livestream_saver-main>python livestream_saver.py d
ownload https://www.youtube.com/watch?v=G9RnKxnQxwA
2021-11-11 10:26:23,754 - INFO - livestream_saver.util - GET https://www.youtube
.com/watch?v=G9RnKxnQxwA
2021-11-11 10:26:24,317 - INFO - download.G9RnKxnQxwA - Will start downloading f
rom segment number 0.
2021-11-11 10:26:24,460 - INFO - livestream_saver.util - GET https://www.youtube
.com/watch?v=G9RnKxnQxwA
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Stream seems to be viewe
d live. Good.
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Stream status Status.OK
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 136      qualityLabel: 720p      mimeType: video/mp4; codecs="avc1.4d401f
"       bitrate: 2684050        quality: hd720  fps: 30
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 247      qualityLabel: 720p      mimeType: video/webm; codecs="vp9"
bitrate: 1040000        quality: hd720  fps: 30
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 135      qualityLabel: 480p      mimeType: video/mp4; codecs="avc1.4d401f
"       bitrate: 1350025        quality: large  fps: 30
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 244      qualityLabel: 480p      mimeType: video/webm; codecs="vp9"
bitrate: 528000 quality: large  fps: 30
2021-11-11 10:26:24,897 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 134      qualityLabel: 360p      mimeType: video/mp4; codecs="avc1.4d401e
"       bitrate: 1008250        quality: medium fps: 30
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 243      qualityLabel: 360p      mimeType: video/webm; codecs="vp9"
bitrate: 292000 quality: medium fps: 30
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 133      qualityLabel: 240p      mimeType: video/mp4; codecs="avc1.4d4015
"       bitrate: 456228 quality: small  fps: 30
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 242      qualityLabel: 240p      mimeType: video/webm; codecs="vp9"
bitrate: 166000 quality: small  fps: 30
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 160      qualityLabel: 144p      mimeType: video/mp4; codecs="avc1.42c00b
"       bitrate: 212465 quality: tiny   fps: 15
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available video quality:
 itag: 278      qualityLabel: 144p      mimeType: video/webm; codecs="vp9"
bitrate: 111000 quality: tiny   fps: 30
2021-11-11 10:26:24,913 - WARNING - download.G9RnKxnQxwA - Chosen video quality:
     itag 136; height: 720p     type: video/mp4; codecs="avc1.4d401f" bitrate: 2
684050
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Available audio quality:
 itag: 140      audioQuality: AUDIO_QUALITY_MEDIUM      mimeType: audio/mp4; cod
ecs="mp4a.40.2" bitrate: 144000 audioSampleRate: 44100
2021-11-11 10:26:24,913 - WARNING - download.G9RnKxnQxwA - Chosen audio quality:
     itag 140; height: AUDIO_QUALITY_MEDIUM     type: audio/mp4; codecs="mp4a.40
.2" bitrate: 144000
2021-11-11 10:26:24,913 - INFO - download.G9RnKxnQxwA - Downloading segment 0...

2021-11-11 10:26:31,678 - INFO - download.G9RnKxnQxwA - Downloading segment 1...

Traceback (most recent call last):
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver.py"
, line 262, in <module>
    main()
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver.py"
, line 258, in main
    args.func(args)
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver.py"
, line 212, in _download
    dl.download(args.scan_delay)
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 337, in download
    self.do_download()
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 409, in do_download
    if not self.write_to_file(in_stream, video_segment_filename):
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 568, in write_to_file
    buf = fsrc_read(length)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\http\client.p
y", line 464, in read
    s = self.fp.read(amt)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\socket.py", l
ine 705, in readinto
    return self._sock.recv_into(b)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\ssl.py", line
 1273, in recv_into
    return self.read(nbytes, buffer)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\ssl.py", line
 1129, in read
    return self._sslobj.read(len, buffer)
KeyboardInterrupt
^C
C:\Users\USER\Downloads\APPS\livestream_saver-main>

And if I run the following manual merge command afterwards, the above test clip will successfully merge and work.
python livestream_saver.py merge stream_capture_G9RnKxnQxwA

Now if I use ANY newer release/build AFTER Jul 6, 2021, including Nov 11, 2021, I get the following error...

Release/Build + Command:

livestream_saver-main (Nov 11, 2021)
python livestream_saver.py download https://www.youtube.com/watch?v=G9RnKxnQxwA

RESULTS:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\USER\Downloads\APPS\livestream_saver-main>python livestream_saver.py d
ownload https://www.youtube.com/watch?v=G9RnKxnQxwA
2021-11-11 10:31:04,280 - INFO - livestream_saver.cookies - No cookie path submi
tted. Using a blank cookie jar.
2021-11-11 10:31:04,499 - INFO - download.G9RnKxnQxwA - Will start downloading f
rom segment number 0.
2021-11-11 10:31:04,968 - INFO - download.G9RnKxnQxwA - Stream seems to be viewe
d live. Good.
2021-11-11 10:31:04,968 - INFO - download.G9RnKxnQxwA - Stream status Status.OK
2021-11-11 10:31:04,968 - ERROR - livestream_saver - apply_descrambler() takes 1
 positional argument but 2 were given
Traceback (most recent call last):
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver.py"
, line 655, in main
    error = args["func"](config, args)
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver.py"
, line 425, in download_mode
    dl.download(config.getfloat("download", "scan_delay", vars=args))
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 705, in download
    self.update_download_urls()
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 642, in update_download_urls
    video_quality, audio_quality = self.get_best_streams(
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 971, in get_best_streams
    avail_streams = self.streams
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 320, in streams
    return pytube.StreamQuery(self.fmt_streams)
  File "C:\Users\USER\Downloads\APPS\livestream_saver-main\livestream_saver\dow
nload.py", line 915, in fmt_streams
    pytube.extract.apply_descrambler(self.player_config_args, fmt)
TypeError: apply_descrambler() takes 1 positional argument but 2 were given

C:\Users\USER\Downloads\APPS\livestream_saver-main>

Any assistance to resolve this error would be great, thank you!

ImportError: cannot import name 'wait' from 'os'

Hey! In Windows 10, when trying to download a stream, the following error occurred:

Traceback (most recent call last): File "E:\STREAMS\youtube_stream_capture\livestream_saver-main\livestream_saver.py", line 12, in <module> from livestream_saver.download import YoutubeLiveStream File "E:\STREAMS\youtube_stream_capture\livestream_saver-main\livestream_saver\download.py", line 2, in <module> from os import sep, path, makedirs, listdir, wait ImportError: cannot import name 'wait' from 'os' (C:\Users\User\AppData\Local\Programs\Python\Python310\lib\os.py)

Fixed by replacing the second line in livestream_saver\download.py with:
from os import sep, path, makedirs, listdir

"wait" is not used there by the way

AttributeError: module 'pytube' has no attribute 'RegexMatchError'

Does code work fine?
I see this error.

For pytube there is an open issue
pytube/pytube#1199

Please help

I don't how to use ur code
please say me how to do it from first
I could not download my YouTube classes because it made private after class please help

Error 403 during mid download

22 minutes into downloading a public livestream I got the following.
Downloading segment 1379...2021-10-12 11:02:36,624 - CRITICAL - download.LJ3EFe_hyoc - <class 'urllib.error.HTTPError'>: HTTP Error 403: Forbidden 2021-10-12 11:02:36,625 - INFO - download.LJ3EFe_hyoc - Forbidden 2021-10-12 11:02:37,236 - WARNING - download.LJ3EFe_hyoc - The stream is not live anymore. Done. 2021-10-12 11:02:37,237 - INFO - download.LJ3EFe_hyoc - Finished downloading LJ3EFe_hyoc.

I did check the stream did not go offline.

livestream_saver does not handle age-restricted videos correctly

Hey. This app worked for a long time, but the streamer I watch started to change his streams to age-restricted, and now livestream_saver no longer works.

I have included a gif showing what happens, even after providing correct login credentials.

Please look into why livestream_saver is not handling age restricted streams.

Thank you.

Merge not working

I get this error on Python 3.9:

2022-05-24 20:10:28,887 - INFO - download.spO_hiEoeAM - No missing segment detected. All good.
2022-05-24 20:10:29,067 - DEBUG - download.spO_hiEoeAM - Probed "0000000000_video.ts". codec_name: h264, duration: 0.567, start_time: 0.0.
2022-05-24 20:10:29,185 - DEBUG - download.spO_hiEoeAM - Probed "0000000000_audio.ts". codec_name: aac, duration: 0.580499, start_time: 0.0.
2022-05-24 20:10:29,186 - INFO - download.spO_hiEoeAM - Performing concat method NativeConcatFile
2022-05-24 20:10:29,186 - ERROR - download.spO_hiEoeAM - unsupported operand type(s) for /: 'str' and 'str'
Traceback (most recent call last):
  File "/home/artrix/livestream_saver/livestream_saver/merge.py", line 553, in merge
    concat_video_file = methods[attempt](
  File "/home/artrix/livestream_saver/livestream_saver/merge.py", line 296, in __init__
    super().__init__(*args, **kwargs)
  File "/home/artrix/livestream_saver/livestream_saver/merge.py", line 73, in __init__
    self._final_file: Path = output_dir / \
TypeError: unsupported operand type(s) for /: 'str' and 'str'
2022-05-24 20:10:29,186 - ERROR - livestream_saver - Missing concat video file: None
Traceback (most recent call last):
  File "/home/artrix/livestream_saver/livestream_saver.py", line 963, in main
    error = args["func"](config, args)
  File "/home/artrix/livestream_saver/livestream_saver.py", line 636, in merge_mode
    written_file = merge(
  File "/home/artrix/livestream_saver/livestream_saver/merge.py", line 611, in merge
    raise Exception(f"Missing concat video file: {concat_video_file}")
Exception: Missing concat video file: None```

Manual merge fails

I tried to merge a downloaded video and it seems to fail but reports success:

python3 merge.py  stream_capture_bvxA9yzUpSo/
2021-04-22 06:35:08,522 - DEBUG - download.bvxA9yzUpSo - Using thumbnail: /volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/thumbnail. Type: jpeg.
2021-04-22 06:35:08,786 - DEBUG - download.bvxA9yzUpSo - Calling subprocess: ['ffmpeg', '-hide_banner', '-y', '-i', '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/bvxA9yzUpSo_video_v2_ffmpeg.mp4', '-i', '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/bvxA9yzUpSo_audio_v2_ffmpeg.m4a', '-i', '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/thumbnail', '-map', '0', '-map', '1', '-map', '2', '-c:v:2', 'jpeg', '-disposition:v:1', 'attached_pic', '-metadata', 'title=BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate', '-metadata', 'artist=BVH e.V.', '-metadata', 'date=2021-04-21', '-c', 'copy', '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/BVH e.V. [2021-04-21] BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate_[1080]_bvxA9yzUpSo.mp4']
2021-04-22 06:35:08,787 - DEBUG - download.bvxA9yzUpSo - FFmpeg STDERR:Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/bvxA9yzUpSo_video_v2_ffmpeg.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.36.100
  Duration: 01:54:35.17, start: 0.000000, bitrate: 905 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 903 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/bvxA9yzUpSo_audio_v2_ffmpeg.m4a':
  Metadata:
    major_brand     : M4A
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf56.36.100
  Duration: 01:54:34.99, start: 0.000000, bitrate: 129 kb/s
    Stream #1:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[mjpeg @ 0x2095e20] Changeing bps to 8
Input #2, jpeg_pipe, from '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/thumbnail':
  Duration: N/A, bitrate: N/A
    Stream #2:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
[ @ 0x7ffcd816a130] [Eval @ 0x7ffcd8169560] Undefined constant or missing '(' in 'attached_pic'
[ @ 0x7ffcd816a130] Unable to parse option value "attached_pic"
Output #0, mp4, to '/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/BVH e.V. [2021-04-21] BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate_[1080]_bvxA9yzUpSo.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.36.100
    title           : BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate
    artist          : BVH e.V.
    date            : 2021-04-21
    Stream #0:0(und): Video: h264, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 903 kb/s, 30 fps, 30 tbr, 90k tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac, 44100 Hz, stereo, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
    Stream #0:2: Video: mjpeg, yuvj420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 tbr, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)
  Stream #2:0 -> #0:2 (copy)
    Last message repeated 1 times

2021-04-22 06:35:08,787 - DEBUG - download.bvxA9yzUpSo - Removing temporary audio/video concatenated files...
2021-04-22 06:35:08,955 - INFO - download.bvxA9yzUpSo - Successfully wrote file "/volume1/homes/jo/tools/livestream_saver/stream_capture_bvxA9yzUpSo/BVH e.V. [2021-04-21] BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate_[1080]_bvxA9yzUpSo.mp4".

New Youtube UI update, Livestreams moved to new tab

Looks like Youtube has made yet another UI update, this time actually affecting livestream locations.
Livestreams are now listed in a "Live" tab, located at URL https://www.youtube.com/channel/zzz/streams , with the Videos tab now being only for uploaded videos.
Because of this location change, some of the channel monitoring is broken:

INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0
INFO - livestream_saver.monitor - Currently listed public videos: 0

Community and Membership monitoring appears to be fine.
Just thought I'd give a heads-up!

having trouble downloading streaming video. HTTP Error 404: Not Found

I tried to download live streaming video but it kept throwing this
Downloading segment 0...2022-11-17 11:45:02,114 - CRITICAL - download.xAgXQE06u10 - <class 'urllib.error.HTTPError'>: HTTP Error 404: Not Found
2022-11-17 11:45:02,114 - INFO - download.xAgXQE06u10 - HTTP Error 404: Not Found
2022-11-17 11:45:02,471 - WARNING - download.xAgXQE06u10 - It seems the stream has not really ended. Retrying in 5 secs... (attempt 0/15)

[Feature request] Ability to rewind for a peirod of time

Is it possible to specify the rewind time and recording duration? It would helps a lot while making real time short clips.

Could not find ffmpeg or ffprobe.

There is an exception which states that "Exception: Could not find ffmpeg or ffprobe! Make sure it is installed and discoverable from your PATH environment variable". Due to which merge is not working

Handle corrupt or missing segments by filling with blank data

Have to look into a way to replace corrupt video/audio segments with blank data instead of failing the concatenation step entirely.

Can't download continuous streams that last longer than the 12-hour rewind limit

For example streams like this that go on for weeks/months:
https://www.youtube.com/watch?v=nWB0ZnYfO6I
It tries to download from segment 0, but obviously it has expired long ago, so it gets stuck trying to download it and eventually fails. ytarchive has the same issue. Is there an option to specify a segment range you can download?

https://gist.github.com/cheadrian/b661fb68a6a87ea64069e641cef68c3e
This script is able to specify a segment range and download it, but you have to manually get to network tab, get the proper links and manually tweak the script and then concatenate yourself, lots of work.

[Feature request] Download all stream from start if possible

For example stream started at 6pm
then at 7pm I run "livestream_saver.py"

Current behavior:
Loaded mp4 starts at 7pm, but on youtube i can watch steam from 6pm using timeline
additional info: livestream_saver logs

Expected behavior:
Load 2 parts at the same time:
part 1: from 6pm to 7 pm
part 2: from 7pm to the live end

Monitoring seems not working

Hello,

Community or public video are empty even with stream curently live.
Cookie return lot of warning like cookies expired, timestamp return 04-04-2022 00:00:04, so not expired.

root@ks:~/livestream_saver$ python livestream_saver.py monitor --cookie cookies.txt --scan-delay 1 https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw
2021-10-21 01:19:46,861 - WARNING - livestream_saver.cookies - <Cookie APISID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,861 - WARNING - livestream_saver.cookies - <Cookie HSID=[redacted] for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,862 - WARNING - livestream_saver.cookies - <Cookie LOGIN_INFO=[redacted]  for .youtube.com/> is expired (1649019661)! Might want to renew it.
2021-10-21 01:19:46,862 - WARNING - livestream_saver.cookies - <Cookie NID=[redacted]  for .youtube.com/> is expired (1646176669)! Might want to renew it.
2021-10-21 01:19:46,862 - WARNING - livestream_saver.cookies - <Cookie PREF=[redacted]  for .youtube.com/> is expired (1649526651)! Might want to renew it.
2021-10-21 01:19:46,862 - WARNING - livestream_saver.cookies - <Cookie SAPISID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,863 - WARNING - livestream_saver.cookies - <Cookie SID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,863 - WARNING - livestream_saver.cookies - <Cookie SIDCC=[redacted]  for .youtube.com/> is expired (1666275424)! Might want to renew it.
2021-10-21 01:19:46,863 - WARNING - livestream_saver.cookies - <Cookie SSID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,863 - WARNING - livestream_saver.cookies - <Cookie VISITOR_INFO1_LIVE=[redacted]  for .youtube.com/> is expired (1649019661)! Might want to renew it.
2021-10-21 01:19:46,864 - WARNING - livestream_saver.cookies - <Cookie YSC=[redacted]  for .youtube.com/> is expired (0)! Might want to renew it.
2021-10-21 01:19:46,864 - WARNING - livestream_saver.cookies - <Cookie __Secure-1PAPISID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,864 - WARNING - livestream_saver.cookies - <Cookie __Secure-1PSID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,864 - WARNING - livestream_saver.cookies - <Cookie __Secure-3PAPISID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,865 - WARNING - livestream_saver.cookies - <Cookie __Secure-3PSID=[redacted]  for .youtube.com/> is expired (1649023204)! Might want to renew it.
2021-10-21 01:19:46,865 - WARNING - livestream_saver.cookies - <Cookie __Secure-3PSIDCC=[redacted]  for .youtube.com/> is expired (1666275424)! Might want to renew it.
2021-10-21 01:19:46,865 - WARNING - livestream_saver.cookies - <Cookie wide=0 for .youtube.com/> is expired (0)! Might want to renew it.
2021-10-21 01:19:46,961 - INFO - livestream_saver - Monitoring channel: UCTI0uTrhBxjRiUDdJJIhmGw
2021-10-21 01:19:47,185 - INFO - livestream_saver.monitor - Currently listed community videos:

2021-10-21 01:19:53,155 - INFO - livestream_saver.monitor - Currently listed public videos:

2021-10-21 01:19:53,156 - INFO - livestream_saver.monitor - Sleeping for 1.35 minutes (81.14 seconds)...
2021-10-21 01:21:14,727 - INFO - livestream_saver.monitor - Sleeping for 4.24 minutes (254.55 seconds)...

root@ks:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal
root@ks:~$ python --version
Python 3.8.10

I want to record sub only stream from this channel id.
Is there something missing in options or is it a bug ?

Thanks for your help :)

Monitoring always return empty lists

Returned lists of "Currently listed public videos" and "Currently listed community videos" are always empty for any channels I tried.

[feature request] Computer shut down after a stream is over

Please add if possible. Or a suggestion on how this could be implemented in Windows would be appreciated 🙏

Cannot monitor or download (public videos = 0)

I was having no issues until a few days ago.
Issues:

Fails to monitor livestreams
Fails to download livestreams
Shows public videos as zero

Debug log example:

2022-11-06 08:06:50,790 - DEBUG - livestream_saver.request - Setting consent cookie: <Cookie CONSENT=YES+cb.20210328-17-p0.en+F+214 for .youtube.com/>
2022-11-06 08:06:50,794 - INFO - livestream_saver - Monitoring channel: Vaush
2022-11-06 08:06:50,797 - DEBUG - livestream_saver.request - Request https://www.youtube.com/c/Vaush/videos?view=2&live_view=502
2022-11-06 08:06:50,880 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/c/Vaush/videos?view=2&live_view=502 -> response url: https://www.youtube.com/c/Vaush/videos?view=2&live_view=502
2022-11-06 08:06:50,882 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Sun, 06 Nov 2022 08:06:50 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Cross-Origin-Opener-Policy-Report-Only: same-origin-allow-popups; report-to="youtube_main"
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-platform=*, ch-ua-platform-version=*
Report-To: {"group":"youtube_main","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/youtube_main"}]}
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
Server: ESF
X-XSS-Protection: 0
Set-Cookie: GPS=1; Domain=.youtube.com; Expires=Sun, 06-Nov-2022 08:36:50 GMT; Path=/; Secure; HttpOnly
Set-Cookie: __Secure-YEC=; Domain=.youtube.com; Expires=Mon, 10-Feb-2020 08:06:50 GMT; Path=/; Secure; HttpOnly; SameSite=lax
Set-Cookie: VISITOR_INFO1_LIVE=ykmKFmKy84k; Domain=.youtube.com; Expires=Fri, 05-May-2023 08:06:50 GMT; Path=/; Secure; HttpOnly; SameSite=none
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked


2022-11-06 08:06:51,028 - INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0

2022-11-06 08:06:51,029 - DEBUG - livestream_saver.request - Request https://www.youtube.com/c/Vaush/community
2022-11-06 08:06:51,115 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/c/Vaush/community -> response url: https://www.youtube.com/c/Vaush/community
2022-11-06 08:06:51,117 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Sun, 06 Nov 2022 08:06:51 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-platform=*, ch-ua-platform-version=*
Report-To: {"group":"youtube_main","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/youtube_main"}]}
Cross-Origin-Opener-Policy-Report-Only: same-origin-allow-popups; report-to="youtube_main"
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
Server: ESF
X-XSS-Protection: 0
Set-Cookie: __Secure-YEC=; Domain=.youtube.com; Expires=Mon, 10-Feb-2020 08:06:51 GMT; Path=/; Secure; HttpOnly; SameSite=lax
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked


2022-11-06 08:06:51,208 - INFO - livestream_saver.monitor - Currently listed community videos: 0

2022-11-06 08:06:51,209 - DEBUG - livestream_saver.request - Request https://www.youtube.com/c/Vaush/membership
2022-11-06 08:06:51,293 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/c/Vaush/membership -> response url: https://www.youtube.com/c/Vaush/membership
2022-11-06 08:06:51,295 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Sun, 06 Nov 2022 08:06:51 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Cross-Origin-Opener-Policy-Report-Only: same-origin-allow-popups; report-to="youtube_main"
Report-To: {"group":"youtube_main","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/youtube_main"}]}
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-platform=*, ch-ua-platform-version=*
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
Server: ESF
X-XSS-Protection: 0
Set-Cookie: __Secure-YEC=; Domain=.youtube.com; Expires=Mon, 10-Feb-2020 08:06:51 GMT; Path=/; Secure; HttpOnly; SameSite=lax
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked


2022-11-06 08:06:51,408 - INFO - livestream_saver.monitor - Currently listed membership videos: 0

2022-11-06 08:06:51,409 - DEBUG - livestream_saver.request - Request https://www.youtube.com/c/Vaush/videos?view=2&live_view=501
2022-11-06 08:06:51,489 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Sun, 06 Nov 2022 08:06:51 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Cross-Origin-Opener-Policy-Report-Only: same-origin-allow-popups; report-to="youtube_main"
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-platform=*, ch-ua-platform-version=*
Report-To: {"group":"youtube_main","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/youtube_main"}]}
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
Server: ESF
X-XSS-Protection: 0
Set-Cookie: __Secure-YEC=; Domain=.youtube.com; Expires=Mon, 10-Feb-2020 08:06:51 GMT; Path=/; Secure; HttpOnly; SameSite=lax
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked


2022-11-06 08:06:51,593 - INFO - livestream_saver.monitor - Currently listed public videos: 0

2022-11-06 08:06:51,593 - DEBUG - livestream_saver - Live videos found for channel "Vaush": None
2022-11-06 08:06:51,595 - INFO - livestream_saver.monitor - Sleeping for 15.39 minutes (923.25 seconds)...

Support monitoring multiple channels in one instance of the script

In the interest of archiving all available streams featuring certain channels/people it would be a good idea to allow the script to monitor multiple channels at once without needing a new instance of the script set up for each channel.
the use case would be setting up a regex filter for someone's name and applying it to a list of channels in the event there's an unannounced collab.
as is, i'd have to run a separate script for each friend of the channel i am archiving which can get pretty cluttered.

Downloading fails if the stream quality is higher than 1080p

I get this error when trying to download livestreams which are at a quality higher than 1080p. Have tried --max-video-quality 1080, but get the same error.

python livestream_saver.py monitor https://www.youtube.com/c/DrLupo

2022-01-07 17:18:25,046 - DEBUG - livestream_saver.request - CookieJar after extract_cookies(): <MozillaCookieJar[<Cookie CONSENT=YES+cb.20210328-17-p0.en+F+581 for .youtube.com/>, <Cookie GPS=1 for .youtube.com/>, <Cookie VISITOR_INFO1_LIVE=I7ayvkFUTlg for .youtube.com/>, <Cookie YSC=aWqWi4EP6lQ for .youtube.com/>]>
2022-01-07 17:18:26,760 - ERROR - livestream_saver - Got error in stream download but continuing...
 'NoneType' object has no attribute 'url'
Traceback (most recent call last):
  File "D:\livestream_saver-main\livestream_saver.py", line 529, in monitor_mode
    livestream.download()
  File "D:\livestream_saver-main\livestream_saver\download.py", line 812, in download
    self.update_download_urls()
  File "D:\livestream_saver-main\livestream_saver\download.py", line 734, in update_download_urls
    self.video_base_url = self.video_itag.url
AttributeError: 'NoneType' object has no attribute 'url'
2022-01-07 17:18:26,761 - INFO - livestream_saver.monitor - Sleeping for 18.47 minutes (1108.35 seconds)...

Full log
https://pastebin.com/zY5FnSgL

python livestream_saver.py download https://www.youtube.com/watch?v=eCar7tz_J6I

2022-01-07 17:18:49,453 - DEBUG - livestream_saver.request - CookieJar after extract_cookies(): <MozillaCookieJar[<Cookie CONSENT=YES+cb.20210328-17-p0.en+F+809 for .youtube.com/>, <Cookie GPS=1 for .youtube.com/>, <Cookie VISITOR_INFO1_LIVE=QG44tSXh0hY for .youtube.com/>, <Cookie YSC=5gibyeIC6tI for .youtube.com/>]>
2022-01-07 17:18:50,639 - INFO - download.eCar7tz_J6I - Selected video None
2022-01-07 17:18:50,639 - INFO - download.eCar7tz_J6I - selected audio None
2022-01-07 17:18:50,963 - DEBUG - download.eCar7tz_J6I - Selected video itag None / Selected audio itag:None
2022-01-07 17:18:50,963 - ERROR - livestream_saver - 'NoneType' object has no attribute 'url'
Traceback (most recent call last):
  File "D:\livestream_saver-main\livestream_saver.py", line 964, in main
    error = args["func"](config, args)
  File "D:\livestream_saver-main\livestream_saver.py", line 615, in download_mode
    dl.download(config.getfloat("download", "scan_delay", vars=args))
  File "D:\livestream_saver-main\livestream_saver\download.py", line 812, in download
    self.update_download_urls()
  File "D:\livestream_saver-main\livestream_saver\download.py", line 734, in update_download_urls
    self.video_base_url = self.video_itag.url
AttributeError: 'NoneType' object has no attribute 'url'

Full log
https://pastebin.com/gTBjGeJz

Reports successful download although merging failed

I downloaded a stream and in the end, I see a success message despite of exceptions. The mp4 file is empty.

2021-04-21 21:54:42,556 - INFO - download.bvxA9yzUpSo - Downloading segment 6876...
2021-04-21 21:54:50,007 - ERROR - download.bvxA9yzUpSo - IncompleteRead(14323 bytes read)
Traceback (most recent call last):
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 560, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 527, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 592, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 562, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/volume1/homes/jo/tools/livestream_saver/livestream_saver/download.py", line 359, in do_download
    if not self.write_to_file(in_stream, video_segment_filename):
  File "/volume1/homes/jo/tools/livestream_saver/livestream_saver/download.py", line 520, in write_to_file
    buf = fsrc_read(length)
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 463, in read
    n = self.readinto(b)
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 497, in readinto
    return self._readinto_chunked(b)
  File "/var/packages/py3k/target/usr/local/lib/python3.8/http/client.py", line 608, in _readinto_chunked
    raise IncompleteRead(bytes(b[0:total_bytes]))
http.client.IncompleteRead: IncompleteRead(14323 bytes read)
2021-04-21 21:54:50,010 - INFO - download.bvxA9yzUpSo - IncompleteRead(14323 bytes read)
2021-04-21 21:54:50,144 - INFO - livestream_saver.util - GET https://www.youtube.com/watch?v=bvxA9yzUpSo
2021-04-21 21:54:50,356 - WARNING - download.bvxA9yzUpSo - The stream is not live anymore. Done.
2021-04-21 21:54:50,356 - INFO - download.bvxA9yzUpSo - Finished downloading bvxA9yzUpSo.
2021-04-21 21:55:45,666 - INFO - download.bvxA9yzUpSo - Successfully wrote file ".//stream_capture_bvxA9yzUpSo/BVH e.V. [2021-04-21] BVH Börsenführerschein Teil 3 - Futures, Optionen & Zertifikate_[1080]_bvxA9yzUpSo.mp4".

Downloading segments too slow

Downloading segments are too slow in current version.
In previous versions it was fast.
First step for downloading already existed stream takes a lot of time. So author can be on time to close and delete his steam.

Check this please.

Streams no longer using Videos/Community tab to list live stream.

I've been having issues with the channel here:
https://www.youtube.com/channel/UCpXBGqwsBkpvcYjsJBQ7LEQ

It appears they no longer list their live streams under the Videos/Community tabs, and only show them under home or live.

Is it possible to implement scanning of the Live tab?

AttributeError: 'NoneType' object has no attribute 'span'

python3.8 livestream_saver.py download https://www.youtube.com/watch?v=u3jsBaC029Q
2022-02-28 21:09:52,166 - INFO - livestream_saver.cookies - No cookie path submitted. Using a blank cookie jar.
2022-02-28 21:09:52,956 - WARNING - download.u3jsBaC029Q - An output directory already existed. We assume a failed download attempt. Last segment available was 1.
2022-02-28 21:09:52,957 - INFO - download.u3jsBaC029Q - Will start downloading from segment number 0.
2022-02-28 21:09:53,261 - INFO - download.u3jsBaC029Q - Stream seems to be viewed live. Good.
2022-02-28 21:09:53,262 - INFO - download.u3jsBaC029Q - Stream status Status.OK
2022-02-28 21:09:55,603 - ERROR - livestream_saver - 'NoneType' object has no attribute 'span'
Traceback (most recent call last):
File "livestream_saver.py", line 963, in main
error = args["func"](config, args)
File "livestream_saver.py", line 614, in download_mode
dl.download(config.getfloat("download", "scan_delay", vars=args))
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 942, in download
self.update_download_urls()
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 831, in update_download_urls
video_quality, audio_quality = self.get_best_streams(
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 1224, in get_best_streams
avail_streams = self.streams
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 480, in streams
query = pytube.StreamQuery(self.fmt_streams)
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 1168, in fmt_streams
pytube.extract.apply_signature(self.player_config_args, fmt, self.js)
File "/home/bigpupik/.local/lib/python3.8/site-packages/pytube/extract.py", line 442, in apply_signature
cipher = Cipher(js=js)
File "/media/sf_Downloads/livestream_saver-main/livestream_saver/download.py", line 121, in patched__init__
self.throttling_plan = pytube.cipher.get_throttling_plan(js)
File "/home/bigpupik/.local/lib/python3.8/site-packages/pytube/cipher.py", line 387, in get_throttling_plan
raw_code = get_throttling_function_code(js)
File "/home/bigpupik/.local/lib/python3.8/site-packages/pytube/cipher.py", line 301, in get_throttling_function_code
code_lines_list = find_object_from_startpoint(js, match.span()[1]).split('\n')
AttributeError: 'NoneType' object has no attribute 'span'

Keep getting cookies expired error

I have got the program working, but I want to get rid of the errors in case I want to record age-restricted stuff.
I have used several different chrome extensions to export my cookies into netscape format. The readers tell me the expiration date is a year from the current time but the program keeps telling me the VISITOR INFO1 LIVE and YSC cookies are expired when they clearly are not. I have tried to manipulate the values and the errors persist no matter what I try. I tried removing those keys as well and that obviously didnt work either.
Any insight is appreciated, thank you.

Exception while accessing title of video (possibly a pytube issue)

Downloads have started failing with this error recently. It is non-recoverable even when I try running the program again.

2023-03-17 07:10:29,798 - DEBUG - livestream_saver.monitor - Parsing json of selected tab: "Live"...
2023-03-17 07:10:29,798 - DEBUG - livestream_saver - Live videos found for channel "pirateaba": [{'videoId': 'OZk0x_ifOHg', 'thumbnail': {'thumbnails': [{'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEbCKgBEF5IVfKriqkDDggBFQAAiEIYAXABwAEG&rs=AOn4CLBeMWCNasP14wOx-16yEtnZZYALcw', 'width': 168, 'height': 94}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEbCMQBEG5IVfKriqkDDggBFQAAiEIYAXABwAEG&rs=AOn4CLAQ5OwusXr1Nz7Hvj3jG8PetEq_rA', 'width': 196, 'height': 110}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEcCPYBEIoBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLBoOHzG0XFB8lq0Zd1srD5baZaIWw', 'width': 246, 'height': 138}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEcCNACELwBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLBOHg9wJYTzaobPqcYpqNzrpvKdfQ', 'width': 336, 'height': 188}]}, 'isLiveNow': True, 'isLive': True, 'title': 'Interlude - I Have No Idea, Stream 1. Short chapter, interruptions, tired author.', 'url': '/watch?v=OZk0x_ifOHg'}, {'videoId': 'OZk0x_ifOHg', 'thumbnail': {'thumbnails': [{'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEbCKgBEF5IVfKriqkDDggBFQAAiEIYAXABwAEG&rs=AOn4CLBeMWCNasP14wOx-16yEtnZZYALcw', 'width': 168, 'height': 94}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEbCMQBEG5IVfKriqkDDggBFQAAiEIYAXABwAEG&rs=AOn4CLAQ5OwusXr1Nz7Hvj3jG8PetEq_rA', 'width': 196, 'height': 110}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEcCPYBEIoBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLBoOHzG0XFB8lq0Zd1srD5baZaIWw', 'width': 246, 'height': 138}, {'url': 'https://i.ytimg.com/vi/OZk0x_ifOHg/hqdefault_live.jpg?sqp=-oaymwEcCNACELwBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLBOHg9wJYTzaobPqcYpqNzrpvKdfQ', 'width': 336, 'height': 188}]}, 'isLiveNow': True, 'isLive': True, 'title': 'Interlude - I Have No Idea, Stream 1. Short chapter, interruptions, tired author.', 'url': '/watch?v=OZk0x_ifOHg'}]
2023-03-17 07:10:29,798 - DEBUG - livestream_saver.util - Creating output_dir: /Users/rajathreghunath/livestream_saver/UCGpHzPfE3vUwNIZDSSt4Uaw/stream_capture_OZk0x_ifOHg...
2023-03-17 07:10:29,798 - INFO - livestream_saver - Found live: OZk0x_ifOHg title: Interlude - I Have No Idea, Stream 1. Short chapter, interruptions, tired author.. Downloading...
2023-03-17 07:10:29,798 - DEBUG - livestream_saver.request - Making API request... endpoint='https://www.youtube.com/youtubei/v1/player?key=AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w&prettyPrint=false'
data={'context': {'client': {'clientName': 'ANDROID', 'clientVersion': '17.31.35', 'androidSdkVersion': 30, 'userAgent': 'com.google.android.youtube/17.31.35 (Linux; U; Android 11) gzip', 'hl': 'en'}}, 'videoId': 'OZk0x_ifOHg'}
headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:89.0) Gecko/20100101 Firefox/89.0', 'Accept-Language': 'en-US,en', 'Content-Type': 'application/json', 'Origin': 'https://www.youtube.com', 'X-YouTube-Client-Name': '3', 'X-YouTube-Client-Version': '17.31.35'}
2023-03-17 07:10:30,034 - DEBUG - livestream_saver.request - Making API request... endpoint='https://www.youtube.com/youtubei/v1/player?key=AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w&prettyPrint=false'
data={'context': {'client': {'clientName': 'ANDROID', 'clientVersion': '17.31.35', 'androidSdkVersion': 30, 'userAgent': 'com.google.android.youtube/17.31.35 (Linux; U; Android 11) gzip', 'hl': 'en'}}, 'videoId': 'OZk0x_ifOHg'}
headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:89.0) Gecko/20100101 Firefox/89.0', 'Accept-Language': 'en-US,en', 'Content-Type': 'application/json', 'Origin': 'https://www.youtube.com', 'X-YouTube-Client-Name': '3', 'X-YouTube-Client-Version': '17.31.35'}
2023-03-17 07:10:30,335 - ERROR - livestream_saver - Got error in stream download but continuing...
 Exception while accessing title of https://www.youtube.com/watch?v=OZk0x_ifOHg. Please file a bug report at https://github.com/pytube/pytube
Traceback (most recent call last):
  File "/Users/rajathreghunath/livestream_saver/livestream_saver/download.py", line 584, in title
    self._title = self.player_response['videoDetails']['title']
KeyError: 'videoDetails'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/rajathreghunath/livestream_saver/livestream_saver.py", line 552, in monitor_mode
    livestream.download()
  File "/Users/rajathreghunath/livestream_saver/livestream_saver/download.py", line 923, in download
    f"Checking metadata items {(self.title, self.description)} against"
  File "/Users/rajathreghunath/livestream_saver/livestream_saver/download.py", line 591, in title
    raise pytube.exceptions.PytubeError(
pytube.exceptions.PytubeError: Exception while accessing title of https://www.youtube.com/watch?v=OZk0x_ifOHg. Please file a bug report at https://github.com/pytube/pytube
2023-03-17 07:10:30,336 - INFO - livestream_saver.monitor - Sleeping for 17.53 minutes (1051.70 seconds)...

ffmpeg takes forever to read the concatenated ts

Lately I noticed an issue: when downloading long livestream, it would take forever to merge.

After some quick tests, it appears that the part that is very slow is to read the natively concatenated ts files.

No matter if it's just to ffprobe it, or to read it as input in ffmpeg, it is extremely slow and ffmpeg will throw thousands of [mov,mp4,m4a,3gp,3g2,mj2 @ 00000000004476c0] Found duplicated MOOV Atom. Skipped it with it.

Here is one sample (merely 5h+): https://1drv.ms/f/s!Akq11jtCTJYwg7YYldMf_jj0esbgUA

ffprobe audio.ts alone would take around 30s on my dated computer.

Now, try to mux it ffmpeg -i audio.ts -c copy output.mp4:

It took a whopping 993s (16min) to just remux audio. Video file would take even longer.

The most interesting about this is that the time it takes isn't linear to the duration of the track. It feels like exponential.

In contrast, using mkvmerge, it does take longer time than typical to read it too, but at least it can re-mux it relatively fast:

mkvmerge audio.ts -o audio.mka - 22s

I understand this is an external dependency issue, so probably nothing can be done here; just think it might be worth a note.

TypeError

Hello, when I tried to download a livestream, I got this error:

2021-08-10 19:03:01,270 - INFO - livestream_saver.cookies - No cookie path submitted. Using a blank cookie jar.
2021-08-10 19:03:01,462 - INFO - download.PbA78pEwsKw - Will start downloading from segment number 0.
2021-08-10 19:03:01,965 - INFO - download.PbA78pEwsKw - Stream seems to be viewed live. Good.
2021-08-10 19:03:01,965 - INFO - download.PbA78pEwsKw - Stream status Status.OK
2021-08-10 19:03:01,966 - ERROR - livestream_saver - apply_descrambler() takes 1 positional argument but 2 were given
Traceback (most recent call last):
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver.py", line 588, in main
    error = args["func"](config, args)
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver.py", line 371, in download_mode
    dl.download(config.getfloat("download", "scan_delay", vars=args))
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver\download.py", line 669, in download
    self.update_download_urls()
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver\download.py", line 606, in update_download_urls
    video_quality, audio_quality = self.get_best_streams(
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver\download.py", line 918, in get_best_streams
    avail_streams = self.streams
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver\download.py", line 284, in streams
    return pytube.StreamQuery(self.fmt_streams)
  File "D:\YoutubeDL\Livestreamsaver\livestream_saver\download.py", line 862, in fmt_streams
    pytube.extract.apply_descrambler(self.player_config_args, fmt)
TypeError: apply_descrambler() takes 1 positional argument but 2 were given

Download does not start when stream is live

I experienced trouble that the download did not begin when the live stream has started. Instead, there was always the same error message. I was able to download with another tool then.

Add webhook notification

Hello, I've seen your project and I think it could be quite useful to me and some of my friends. However I was wondering if it would be possible to add webhook support as an alternative to email notifications. We use Discord a lot and it's seems like the perfect way to notify our community.

I may be able to submit a pull request with the corresponding code if I have the time and you allow it.

Syntax Error when monitoring a channel

Hi there.

I followed the installation instructions and made a virtual environment, the dependencies also installed correctly, but when I launch the script using python livestream_saver.py monitor CHANNEL_URL I get the following error:

File "livestream_saver.py", line 400
    if level := config.get(mode_str, "log_level", vars=args):
              ^
SyntaxError: invalid syntax

This is with Python 3.7.3

Merge fails (outputs an mp4 without audio)

Both, automatic and manual merge produce an mp4 without audio

D:\livestream_saver>python merge.py stream_capture_R-wCsDY2jTA
2021-05-24 21:08:01,518 - DEBUG - download.R-wCsDY2jTA - Calling subprocess: ['ffmpeg', '-hide_banner', '-loglevel', 'panic', '-y', '-i', 'D:\\livestream_saver\\stream_capture_R-wCsDY2jTA\\concat_R-wCsDY2jTA_video.ts', '-c', 'copy', 'D:\\livestream_saver\\stream_capture_lIIl4H_k0KY\\R-wCsDY2jTA_video_v2_ffmpeg.mp4']
Traceback (most recent call last):
  File "D:\livestream_saver\merge.py", line 55, in <module>
    written_file = merge.merge(\
  File "D:\livestream_saver\livestream_saver\merge.py", line 118, in merge
    ffmpeg_output_path_video = concat("video", info.get('id'), video_files, data_dir)
  File "D:\livestream_saver\livestream_saver\merge.py", line 64, in concat
    logger.debug("FFmpeg STDERR:\n" + ffmpeg_stderr)
TypeError: can only concatenate str (not "NoneType") to str

merging error

I have problem in merging please sort out

C:\Users\USER\Downloads\livestream_saver-main\livestream_saver-main>python lives
tream_saver.py merge C:\Users\USER\Downloads\livestream_saver-main\livestream_sa
ver-main\stream_capture_19gEHNEkakM{19gEHNEkakM}
Traceback (most recent call last):
File "C:\Users\USER\Downloads\livestream_saver-main\livestream_saver-main\live
stream_saver\merge.py", line 18, in get_metadata_info
with open(path / "metadata.json", 'r', encoding='utf-8') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\USER\Downlo
ads\livestream_saver-main\livestream_saver-main\stream_capture_19gEHNEkakM{19
gEHNEkakM}\metadata.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "livestream_saver.py", line 655, in main
error = args["func"](config, args)
File "livestream_saver.py", line 440, in merge_mode
info = get_metadata_info(data_path)
File "C:\Users\USER\Downloads\livestream_saver-main\livestream_saver-main\live
stream_saver\merge.py", line 21, in get_metadata_info
logger.exception(f"Exception while trying to load metadata.json: {e}")
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1469, in excepti
on
self.error(msg, *args, exc_info=exc_info, **kwargs)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1463, in error
self.log(ERROR, msg, args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1577, in log
self.handle(record)
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1587, in handle
self.callHandlers(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1649, in callHan
dlers
hdlr.handle(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 950, in handle
self.emit(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1182, in emit
self.stream = self.open()
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1172, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\USER\Downlo
ads\livestream_saver-main\livestream_saver-main\stream_capture_19gEHNEkakM{19
gEHNEkakM}\merge.log'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "livestream_saver.py", line 683, in
exit(main())
File "livestream_saver.py", line 660, in main
logger.exception(e)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1469, in excepti
on
self.error(msg, *args, exc_info=exc_info, **kwargs)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1463, in error
self.log(ERROR, msg, args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1577, in log
self.handle(record)
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1587, in handle
self.callHandlers(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1649, in callHan
dlers
hdlr.handle(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 950, in handle
self.emit(record)
File "C:\ProgramData\Anaconda3\lib\logging_init_.py", line 1182, in emit
self.stream = self.open()
File "C:\ProgramData\Anaconda3\lib\logging_init.py", line 1172, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\USER\Downlo
ads\livestream_saver-main\livestream_saver-main\stream_capture_19gEHNEkakM{19
gEHNEkakM}\merge.log'

Merging fails due to duplicated MOOV Atoms

I get a lot of

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5569c3c60440] Found duplicated MOOV Atom. Skipped it
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5569c3c60440] Failed to add index entry

and the merged file only has one segment in it.

A different way of concatenation is described here, which worked in my case:

https://stackoverflow.com/questions/26260925/ffmpeg-2-4-2-concatenate-mp4-files

glubsy / livestream_saver Goto Github PK

livestream_saver's People

Contributors

Stargazers

Watchers

Forkers

livestream_saver's Issues

Recommend Projects

Recommend Topics

Recommend Org