Git Product home page Git Product logo

Comments (13)

vxbinaca avatar vxbinaca commented on June 4, 2024 2

@refeed here's the symptoms that trigger upload failures for some youtube channels:

  • A copyrighted video or deleted video will kill the channel rip on download. We should be able to gracefully handle this better despite that we're using --ignore-errors and it not working. Even if you complete a channel rip AROUND the errored video, it will still kill future updates when when using --use-download-archive. It will see the bad ID and kill the rip.

  • Sometimes randomly the hand off between a completed rip and upload fails. This particular problem started when you switched us to OOP. So this is your bug that would be nice to see fixed before your vanish for another year.

In the interim I'm going up add a warning about ripping entire channels.

Edit: any future low quality / no output / hidden command issues about errors related to channel rips will be closed automatically like S3 timeout issues in the interest of sanity.

from tubeup.

refeed avatar refeed commented on June 4, 2024

@rudolphos Could you help us by giving an URL to reproduce this issue?

from tubeup.

vxbinaca avatar vxbinaca commented on June 4, 2024

It seems like a JSON error:

tubeup --use-download-archive --metadata=collection:weedtubers https://www.periscope.tv/CoureyPie/1vAGRRqZVDyGl

[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.09.26
[debug] Python version 3.4.9 (CPython) - Linux-4.15.18-1-pve-x86_64-with-centos-7.5.1804-Core
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15
[debug] Proxy map: {}

An exception just occured, if you found this exception isn't related with any of your connection problem, please report this issue to https://github.com/bibanon/tubeup/issues
Traceback (most recent call last):
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/__main__.py", line 91, in main
    use_download_archive):
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/TubeUp.py", line 359, in archive_urls
    identifier, meta = self.upload_ia(basename, custom_meta)
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/TubeUp.py", line 273, in upload_ia
    with open(json_metadata_filepath) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/vxbinaca/.tubeup/downloads/Courey_-_Smoke_throwback_jams_hotboxtheinternet_420_stonergirls-1vAGRRqZVDyGl.info.json'

@rudolphos I'm going to make issue submission guidelines because of you. They'll require submission of the command you used with URL so we can test, and you'll have to use Github markup like I did in this post. Failure to either one of these will result in closure of the issue. We don't care that you back up right-wing content. Don't try to hide the URL from us we need it to fix the script.

from tubeup.

rudolphos avatar rudolphos commented on June 4, 2024

@vxbinaca I don't remember which channel it was, but it's possible it happens when some channels have videos put in limited state or geoblocked, I don't really know. That particular channel might have had a few videos in limited state+geoblocked.

Or maybe it's caused because youtubedl also outputs ids of videos that are already deleted/blocked due to copyright. So a video like this in a playlist / channel playlist could stop the program.

I just tried to archive this and got a similar error

sudo tubeup https://www.youtube.com/watch?v=4ekfBzeJBL8 --use-download-archive
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.09.18
[debug] Python version 3.6.6 (CPython) - Linux-4.4.0-17134-Microsoft-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.4-0ubuntu0.18.04.1, ffprobe 3.4.4-0ubuntu0.18.04.1
[debug] Proxy map: {}
Unable to extract video title
ERROR: "Tim Maia -..." The YouTube account associated with this video has been terminated due to multiple third-party notifications of copyright infringement.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/extractor/common.py", line 507, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.6/dist-packages/youtube_dl/extractor/youtube.py", line 1892, in _real_extract
    raise ExtractorError(error_message, expected=True)
youtube_dl.utils.ExtractorError: "Tim Maia -..." The YouTube account associated with this video has been terminated due to multiple third-party notifications of copyright infringement.


An exception just occured, if you found this exception isn't related with any of your connection problem, please report this issue to https://github.com/bibanon/tubeup/issues
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tubeup/__main__.py", line 91, in main
    use_download_archive):
  File "/usr/local/lib/python3.6/dist-packages/tubeup/TubeUp.py", line 356, in archive_urls
    urls, proxy, ydl_username, ydl_password, use_download_archive)
  File "/usr/local/lib/python3.6/dist-packages/tubeup/TubeUp.py", line 144, in get_resource_basenames
    self.create_basenames_from_ydl_info_dict(ydl, info_dict)
  File "/usr/local/lib/python3.6/dist-packages/tubeup/TubeUp.py", line 163, in create_basenames_from_ydl_info_dict
    info_type = info_dict.get('_type', 'video')
AttributeError: 'NoneType' object has no attribute 'get'

from tubeup.

vxbinaca avatar vxbinaca commented on June 4, 2024

@rudolphos No, you can hit the Up key and look at the last commands you ran. Limited state videos still show when you dump a channels IDs, geoblocked throw up an error when you try to rip them but still show.

What you posted was a terminated channel that can't be ripped and is useless to us as an error still.

This is what I'm talking about:

tubeup --use-download-archive --metadata=collection:tubeup Z-nLNHzsAD8

[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.10.05
[debug] Python version 3.4.9 (CPython) - Linux-4.15.18-1-pve-x86_64-with-centos-7.5.1804-Core
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15
[debug] Proxy map: {}
video doesn't have subtitles

An exception just occured, if you found this exception isn't related with any of your connection problem, please report this issue to https://github.com/bibanon/tubeup/issues
Traceback (most recent call last):
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/__main__.py", line 91, in main
    use_download_archive):
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/TubeUp.py", line 359, in archive_urls
    identifier, meta = self.upload_ia(basename, custom_meta)
  File "/home/vxbinaca/.local/lib/python3.4/site-packages/tubeup/TubeUp.py", line 273, in upload_ia
    with open(json_metadata_filepath) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/vxbinaca/.tubeup/downloads/le_monke_full_rap-Z-nLNHzsAD8.info.json'

Tubeup is wonky right now due to:

  • File parsing errors because Tubeup needs to be gone through with a fine tooth comb.
  • JSON errors that just make a video at random not upload and leaves the downloaded result in the .tubeup folder.

@rudolphos Do not comment on this issue further you're wasting our time with you purposely hiding URLs or when you do you giver us terminated channels. Just save it for next issue. Learn the rules. I assure you I run into issues before you do since I heavily use this script.

@refeed DM me your email adddress I want to add your Archive.org account to a collection I use to test Tubeup and keep things out of Community Video. I think you're testing one video at a time and I want you to start doing test rips of medium sized channels and comparing the results of whats on youtube to what was posted on IA. Theres a wonky bug that didn't exist before your re-base thats leaving videos in the tubeup folder and I want to nail this bug. I want you to start dumping into that channel so we don't piss off staff if you don't already have a collection of your own.

from tubeup.

refeed avatar refeed commented on June 4, 2024

About the .info.json missing issue, I think it's an intermittent issue produced by youtube-dl, I still can't reproduce it until now. https://pastebin.com/DB69PNyS

We do can make tubeup to re-run youtube-dl until we get the .info.json file which is crucial for ia metadata

from tubeup.

vxbinaca avatar vxbinaca commented on June 4, 2024

I've rerun it multiple times same result.

from tubeup.

refeed avatar refeed commented on June 4, 2024

from tubeup.

vxbinaca avatar vxbinaca commented on June 4, 2024

Yeah gimme a bit

from tubeup.

ealgase avatar ealgase commented on June 4, 2024

Any update? Also having this issue with https://www.youtube.com/user/sthetix360

from tubeup.

vxbinaca avatar vxbinaca commented on June 4, 2024

@ealgase use youtube-dl to output the channels videoIDs then have tubeup rip each video individually.

from tubeup.

ealgase avatar ealgase commented on June 4, 2024

OK.

from tubeup.

brandongalbraith avatar brandongalbraith commented on June 4, 2024

I believe this issue is fixed with the move to the new yt-dlp extractor. Please reopen if issues reoccurs with the latest version of tubeup. Thanks!

from tubeup.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.