Comments (8)
-
VTT is the default subtitle output for youtube, Archive.org doesn't yet process VTT to SRT so subtitles can be implemented, however I do believe they're collected and uploaded to Archive. Perhaps one day Archive will add a derive rule to process VTT. What's important is the information is saved.
-
The most common format for muxed container upload is MKV, although 20 percent of the time it's MP4. This does require post-processing on Archives end but it does allow for the best quality video and audio to be used. Later on when the files are re-derived ot something better, or perhaps they could be streamed in their uploaded format.
I'll give you an example of why forcing MP4 is bad: [A Fox In Sapce Episode 1][https://www.youtube.com/watch?v=uieM18rZdHY] was (fixed eventually) archived in a reduced quality when MP4 is forced. Currently I'm using bestaudio+bestvideo and letting youtube and youtube-dl sort it out, or whatever site is being ripped. It tends to work out for the best. I've done a lot of playing with this and tried to 'fix' the MKV 'problem', and in the end it's just easier to get the buxed best of both audio and video.
Instant streamability is iess important than fidelity.
from tubeup.
@rudolphos You can transcode MKV to MP4 with ffmpeg, and Archive.org derives to that format from MKV. The focus of this script is downloading video in the highest quality and transfering it, with metadata, to Archive.org and assembling a item for each video.
If this is acceptable to you, I'll close this issue.
from tubeup.
Your item got turned off it looks like, email [email protected] and ask why with the item identifier, ask that it be undarked if you want to write to it. Also note in the Readme my warning about uploading entire channels to "Community Video". Try writing 50 videos with admin permission, make an itel, transfer your already uploaded progress to a collection (you must request they be made), then continue to upload all videos from that channel into that collection. Theres flags to do it.
In closing, have a look at this: https://archive.org/details/youtube-uieM18rZdHY
I manually converted and re-uploaded the subtitles in SRT cormat, but eventually if/when Archive derives VTT or SRT, that's what it will look like. All the thumbnails and metadata are uploaded, the video is in top quality.
from tubeup.
What version of youtube-dl are you using? If you take my handle here and drop it into Archive.org you can see me ingestion of not just youtube but Periscope and other sites. Other than connection dropouts or Archive being overloaded and it dropping my uploads, I haven't had issues.
Did you install youtbue-dl from the Ubuntu repository or the pip repo? Pip is what you want to use. Remove Youtube-dl installed from Ubuntu and re-do it using the instructions on the Readme.
from tubeup.
I'll give you an example of why forcing MP4 is bad:
Didn;t know this.. I usually used mkv, but it was incompatible with editing software, so I switched my youtube-dl script to mp4 which works everywhere.
I'm gonna try this on a dedicated server, cloud9 was out of space (it only had 2 GB instead of 5 GB)
from tubeup.
Yeah it's acceptable. But is it possible to archive whole YT channel as one archive.org item ?
from tubeup.
Tried this script on a VPS, 10 videos successfully uploaded, but then it showed this error:
:: Upload Finished. Item information:
Title: ...
Upload URL: ...
:: Uploading /root/.tubeup/downloads/.....
2016-12-07 20:13:49,195 - internetarchive.item - ERROR - error uploading .....annotations.xml to youtube-...., Access Denied - You lack sufficient privileges to write to this item.
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/internetarchive/item.py", line 625, in upload_file
response.raise_for_status()
File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 893, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://s3.us.archive.org/.....annotations.xml
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/tubeup", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/tubeup/__main__.py", line 272, in main
identifier, meta = upload_ia(video, custom_meta=md)
File "/usr/local/lib/python3.5/dist-packages/tubeup/__main__.py", line 219, in upload_ia
item.upload(vid_files, metadata=meta, retries=30000, request_kwargs=dict(timeout=30000), delete=True)
File "/usr/local/lib/python3.5/dist-packages/internetarchive/item.py", line 751, in upload
request_kwargs=request_kwargs)
File "/usr/local/lib/python3.5/dist-packages/internetarchive/item.py", line 645, in upload_file
raise type(exc)(error_msg, response=exc.response, request=exc.request)
requests.exceptions.HTTPError: error uploading .....annotations.xml to youtube-...., Access Denied - You lack sufficient privileges to write to this item.
from tubeup.
Yes it's possible to rip an entire channel and upload it to archive.org. It's how I do my archival. Your error I'll look into in a bit.
from tubeup.
Related Issues (20)
- ERROR: Unable to extract uploader id HOT 5
- Proposal: Identify core/essential metadata and add upload safeties for missing MD HOT 4
- Bug report: Channels having YouTube shorts cause Tubeup to fail HOT 4
- Proposal: What to do about yt-dlps new nightly branch? HOT 4
- deleted HOT 1
- Bug report/feature request: Continue downloading other videos when one fails with a permanent error HOT 6
- Bug report: extremely slow downloads from youtube HOT 3
- Bug report: [native] nsig extraction failed HOT 4
- Possible NSIG fixes HOT 8
- Upgrade yt-dlp ASAP to at least 2023.07.06 HOT 5
- "Creator" field for Douyin needs update HOT 9
- Update internetarchive to 3.4.0/3.5.0? HOT 4
- Uploaded YT video thumbnails in .webp are not used for IA item tiles HOT 7
- PEP 668 compatability
- Add new release for 2023-08-10. HOT 2
- Bug report: Video impossible to upload when best quality stream is unavailable on the server-side HOT 11
- Bug report: Unable to archive Youtube video after premiere HOT 1
- TypeError: 'module' object is not callable HOT 8
- Troubles facing video archival
- Add a "local mode" to assist in uploading from disk HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tubeup.