Comments (8)
glubsy
commented on June 20, 2024
The warning messages regarding cookies are irrelevant (I should probably get rid of them as they are not very useful anyway). The cookies are loaded regardless.
It could be a simple display bug. I suspect the script would actually start downloading live streams if one came up anyway. If that was not the case, please post your monitor debug log to a service like Pastebin or as a gist.
It could also be that your cookies are not being loaded properly, and youtube reset your cookies for the session (the monitor debug logs would show that, with a bunch of Set-Cookie: SID=EXPIRED for example).
I edited your original message to redact the cookie values. Those are not supposed to be shared with others, they should be kept secret. Although there shouldn't be a problem here as the server probably refused your cookies anyway.
I'll keep an eye on this.
from livestream_saver.
BiRooT
commented on June 20, 2024
@glubsy There really seems to be a problem with the cookies.
I have extracted my youtube cookies several times from different browsers and different extensions (and I use this same method via yt-dlp to download videos that are only accessible when you are a member of a channel, and it works)
If I extract the cookies and put them in a file, I get this :
python3 livestream_saver.py monitor -c /var/www/stream/livestream_saver-main/livestream_saver.cfg
2022-01-17 16:58:38,071 - WARNING - livestream_saver.cookies - <Cookie CONSISTENCY=[redacted] for .youtube.com/> is expired (1642419916)! Might want to renew it.
2022-01-17 16:58:38,071 - WARNING - livestream_saver.cookies - <Cookie GPS=[redacted] for .youtube.com/> is expired (1642421088)! Might want to renew it.
2022-01-17 16:58:38,071 - WARNING - livestream_saver.cookies - <Cookie YSC=[redacted] for .youtube.com/> is expired (0)! Might want to renew it.
2022-01-17 16:58:38,919 - INFO - livestream_saver - Monitoring channel: UCTI0uTrhBxjRiUDdJJIhmGw
2022-01-17 16:58:39,110 - INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0
2022-01-17 16:58:39,320 - INFO - livestream_saver.monitor - Currently listed community videos: 0
2022-01-17 16:58:39,510 - INFO - livestream_saver.monitor - Currently listed public videos: 0
2022-01-17 16:58:39,515 - INFO - livestream_saver.monitor - Sleeping for 3.80 minutes (228.07 seconds)...
Not even possible to get the public videos.
If I disable cookie variable in cfg like this:
[monitor 1]
channel_name = 1
scan_delay = 1.0
URL = https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw
#cookie = /var/www/stream/livestream_saver-main/cookies.txt
I get:
python3 livestream_saver.py monitor -c /var/www/stream/livestream_saver-main/livestream_saver.cfg
2022-01-17 17:12:54,911 - INFO - livestream_saver.cookies - No cookie path submitted. Using a blank cookie jar.
2022-01-17 17:12:54,980 - INFO - livestream_saver - Monitoring channel: UCTI0uTrhBxjRiUDdJJIhmGw
2022-01-17 17:12:55,264 - INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0
2022-01-17 17:12:55,365 - INFO - livestream_saver.monitor - Currently listed community videos: 0
2022-01-17 17:12:55,624 - INFO - livestream_saver.monitor - Currently listed public videos: 30
<video_id> - <video_title>
...
...
...
2022-01-17 17:12:55,625 - INFO - livestream_saver.monitor - Sleeping for 4.41 minutes (264.66 seconds)...
So with cookies I can't list anything, without cookies I can list public video so I think there is a problem with cookie management.
And do you know why delay is not respected ?
Thanks for your help.
from livestream_saver.
glubsy
commented on June 20, 2024
The only way to know what is going on would be to check the logs. I would suggest using --log-level DEBUG and share the logs if you can.
Regarding the delay not being respected, that's normal: there is a variance added in the wait_block() function. I'll add this as a configurable variable in the config file soon.
from livestream_saver.
BiRooT
commented on June 20, 2024
I don't know if it will be usefull for you:
python3 livestream_saver.py monitor -c /var/www/stream/livestream_saver-main/livestream_saver.cfg
2022-01-17 19:48:04,928 - DEBUG - livestream_saver - Regex filters ["allow_regex: re.compile('.*archiv.*|.*karaoke.*|.*sing.*', re.IGNORECASE|re.MULTILINE)", 'block_regex: None']
2022-01-17 19:48:04,930 - WARNING - livestream_saver.cookies - <Cookie CONSISTENCY=[redacted] for .youtube.com/> is expired (1642419916)! Might want to renew it.
2022-01-17 19:48:04,931 - WARNING - livestream_saver.cookies - <Cookie GPS=[redacted] for .youtube.com/> is expired (1642421088)! Might want to renew it.
2022-01-17 19:48:04,934 - WARNING - livestream_saver.cookies - <Cookie YSC=[redacted] for .youtube.com/> is expired (0)! Might want to renew it.
2022-01-17 19:48:05,006 - DEBUG - livestream_saver.request - Initial req header items: [('Cookie', 'CONSENT=[redacted]; VISITOR_INFO1_LIVE=[redacted]; SID=[redacted]; __Secure-1PSID=[redacted]; __Secure-3PSID=[redacted]; HSID=[redacted]; SSID=[redacted]; APISID=[redacted]; SAPISID=[redacted]; __Secure-1PAPISID=[redacted]; __Secure-3PAPISID=[redacted]; LOGIN_INFO=[redacted]; PREF=[redacted]; SIDCC=[redacted]; __Secure-3PSIDCC=[redacted]'), ('Host', 'www.youtube.com'), ('User-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0'), ('Accept-language', 'en-US,en')]
2022-01-17 19:48:05,010 - DEBUG - livestream_saver.request - Initial res headers: Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Mon, 17 Jan 2022 18:48:05 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Report-To: {"group":"ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/encsid_ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"}]}
Cross-Origin-Opener-Policy-Report-Only: same-origin; report-to="ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=fr for more info."
Server: ESF
X-XSS-Protection: 0
Set-Cookie: YSC=[redacted]; Domain=.youtube.com; Path=/; Secure; HttpOnly; SameSite=none
Set-Cookie: SIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:05 GMT; path=/; domain=.youtube.com; priority=high
Set-Cookie: __Secure-3PSIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:05 GMT; path=/; domain=.youtube.com; Secure; HttpOnly; priority=high; SameSite=none
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
2022-01-17 19:48:06,119 - DEBUG - livestream_saver.request - Initial req cookies after extract: Set-Cookie: APISID=[redacted]
Set-Cookie: CONSENT=[redacted]
Set-Cookie: HSID=[redacted]
Set-Cookie: LOGIN_INFO=[redacted]
Set-Cookie: PREF=[redacted]
Set-Cookie: SAPISID=[redacted]
Set-Cookie: SID=[redacted]
Set-Cookie: SIDCC=[redacted]
Set-Cookie: SSID=[redacted]
Set-Cookie: VISITOR_INFO1_LIVE=[redacted]
Set-Cookie: __Secure-1PAPISID=[redacted]
Set-Cookie: __Secure-1PSID=[redacted]
Set-Cookie: __Secure-3PAPISID=[redacted]
Set-Cookie: __Secure-3PSID=[redacted]
Set-Cookie: __Secure-3PSIDCC=[redacted]
2022-01-17 19:48:06,119 - INFO - livestream_saver - Monitoring channel: UCTI0uTrhBxjRiUDdJJIhmGw
2022-01-17 19:48:06,120 - DEBUG - livestream_saver.request - Request https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=502
2022-01-17 19:48:06,121 - DEBUG - livestream_saver.request - Request headers: [('Cookie', 'CONSENT=[redacted]; VISITOR_INFO1_LIVE=[redacted]; SID=[redacted]; __Secure-1PSID=[redacted]; __Secure-3PSID=[redacted]; HSID=[redacted]; SSID=[redacted]; APISID=[redacted]; SAPISID=[redacted]; __Secure-1PAPISID=[redacted]; __Secure-3PAPISID=[redacted]; LOGIN_INFO=[redacted]; PREF=[redacted]; SIDCC=[redacted]; __Secure-3PSIDCC=[redacted]; YSC=[redacted]'), ('User-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0'), ('Accept-language', 'en-US,en')]
2022-01-17 19:48:06,183 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=502 -> response url: https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=502
2022-01-17 19:48:06,186 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Mon, 17 Jan 2022 18:48:06 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Cross-Origin-Opener-Policy-Report-Only: same-origin; report-to="ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"
Report-To: {"group":"ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/encsid_ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"}]}
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*
Server: ESF
X-XSS-Protection: 0
Set-Cookie: SIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; priority=high
Set-Cookie: __Secure-3PSIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; Secure; HttpOnly; priority=high; SameSite=none
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
2022-01-17 19:48:06,326 - INFO - livestream_saver.monitor - Currently listed public upcoming videos: 0
2022-01-17 19:48:06,327 - DEBUG - livestream_saver.request - Request https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/community
2022-01-17 19:48:06,327 - DEBUG - livestream_saver.request - Request headers: [('Cookie', 'CONSENT=[redacted]; VISITOR_INFO1_LIVE=[redacted]; SID=[redacted].; __Secure-1PSID=[redacted].; __Secure-3PSID=[redacted].; HSID=[redacted]; SSID=[redacted]; APISID=[redacted]; SAPISID=[redacted]; __Secure-1PAPISID=[redacted]; __Secure-3PAPISID=[redacted]; LOGIN_INFO=[redacted]; PREF=tz=[redacted]; SIDCC=[redacted]; __Secure-3PSIDCC=[redacted]; YSC=[redacted]'), ('User-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0'), ('Accept-language', 'en-US,en')]
2022-01-17 19:48:06,390 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/community -> response url: https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/community
2022-01-17 19:48:06,393 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Mon, 17 Jan 2022 18:48:06 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*
Report-To: {"group":"ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/encsid_ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"}]}
Cross-Origin-Opener-Policy-Report-Only: same-origin; report-to="ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"
Server: ESF
X-XSS-Protection: 0
Set-Cookie: SIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; priority=high
Set-Cookie: __Secure-3PSIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; Secure; HttpOnly; priority=high; SameSite=none
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
2022-01-17 19:48:06,498 - INFO - livestream_saver.monitor - Currently listed community videos: 0
2022-01-17 19:48:06,499 - DEBUG - livestream_saver.request - Request https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=501
2022-01-17 19:48:06,499 - DEBUG - livestream_saver.request - Request headers: [('Cookie', 'CONSENT=[redacted]; VISITOR_INFO1_LIVE=[redacted]; SID=[redacted].; __Secure-1PSID=[redacted].; __Secure-3PSID=[redacted].; HSID=[redacted]; SSID=[redacted]; APISID=[redacted]; SAPISID=[redacted]; __Secure-1PAPISID=[redacted]; __Secure-3PAPISID=[redacted]; LOGIN_INFO=[redacted]; PREF=[redacted]; SIDCC=[redacted]; __Secure-3PSIDCC=[redacted]; YSC=[redacted]'), ('User-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0'), ('Accept-language', 'en-US,en')]
2022-01-17 19:48:06,563 - DEBUG - livestream_saver.request - REQUEST https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=501 -> response url: https://www.youtube.com/channel/UCTI0uTrhBxjRiUDdJJIhmGw/videos?view=2&live_view=501
2022-01-17 19:48:06,566 - DEBUG - livestream_saver.request - Response Status code: 200.
Response headers:
Content-Type: text/html; charset=utf-8
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Mon, 17 Jan 2022 18:48:06 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000
Cross-Origin-Opener-Policy-Report-Only: same-origin; report-to="ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"
Report-To: {"group":"ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/encsid_ATmXEA_aXV-idIZ-e5x1JSbJUg8hfAx2dSl3lQ"}]}
Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*
Server: ESF
X-XSS-Protection: 0
Set-Cookie: SIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; priority=high
Set-Cookie: __Secure-3PSIDCC=[redacted]; expires=Tue, 17-Jan-2023 18:48:06 GMT; path=/; domain=.youtube.com; Secure; HttpOnly; priority=high; SameSite=none
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
2022-01-17 19:48:06,763 - INFO - livestream_saver.monitor - Currently listed public videos: 0
2022-01-17 19:48:06,763 - DEBUG - livestream_saver - Live videos found for channel "Ryu7z": None
2022-01-17 19:48:06,763 - INFO - livestream_saver.monitor - Sleeping for 2.96 minutes (177.47 seconds)...
from livestream_saver.
glubsy
commented on June 20, 2024
Thanks for the report. This is indeed very odd. Does this happen with other channels as well? It seems to me it only happens on this channel in particular for some reason.
Public live streams appear to be downloaded just fine at least. Perhaps member-only streams would work as well? I can't test that. I'll try to have a closer look as soon as I get some time.
from livestream_saver.
BiRooT
commented on June 20, 2024
I'm not member of another channel so I can't test :/
from livestream_saver.
glubsy
commented on June 20, 2024
Sorry I was not being very clear. I was wondering if, even though member-only streams are not listed, do they still get downloaded, or are they skipped altogether? I assume they get skipped but I just wanted to make sure.
Either way that is a bug I'll have to look at.
from livestream_saver.
BiRooT
commented on June 20, 2024
Yes it's totally skipped like I monitor a ghost channel without videos or livestream.
from livestream_saver.
Related Issues (20)
- ffmpeg takes forever to read the concatenated ts HOT 4
- livestream_saver does not handle age-restricted videos correctly HOT 14
- New Youtube UI update, Livestreams moved to new tab HOT 3
- Missing Videos tab endpoint data for Livestream VODS HOT 2
- Cannot monitor or download (public videos = 0) HOT 1
- Streams no longer using Videos/Community tab to list live stream. HOT 2
- having trouble downloading streaming video. HTTP Error 404: Not Found HOT 2
- Unable to start livestream_saver on Ubuntu HOT 9
- Support monitoring multiple channels in one instance of the script HOT 4
- Could not find ffmpeg or ffprobe. HOT 1
- Unhandled exception: Missing streamingData key in json causing downloads to be interrupted HOT 4
- Exception while accessing title of video (possibly a pytube issue) HOT 1
- I think the latest change broke something HOT 2
- "The following content is not available on this app." errors HOT 8
- Help getting yt-dlp to spawn on events? HOT 4
- Cookie HOT 2
- Outdated client error. Retrying shortly... HOT 1
- SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] HOT 2
- How to install? HOT 4
- No module named 'yt_dlp' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from livestream_saver.