revanced / restore-missing-youtube-watch-history Goto Github PK

A user pointed out that this script also restored shorts, which is not necessary since these are usually already preserved in the watch history.

Shorts can be skipped by affing filter:

if '#shorts' in event['title']:
    continue

Hi I need help, I accidently exported to html not json, and all the watch history is now gone from my google activity is there anyway to convert this html file to usable json file for the script? I've lost my watch history for the last 3 years...

The argument is unnecessary, as Shorts were always recorded on patched YouTube. Instead, just always remove them.

I did everything according to the instructions, but there was an error

Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

I have been trying to restore the YouTube watch history via Termux through the guide you provided but seems like in the end, it gives me an error

Found 18394 videos to watch
Found 18258 videos to watch after de-duplication
Traceback (most recent call last):
File "/storage/emulated/0/Download/Compressed/Revanced/restore-missing-youtube-watch-history-main/main.py", line 210, in
main()
File "/storage/emulated/0/Download/Compressed/Revanced/restore-missing-youtube-watch-history-main/main.py", line 109, in main
download_videos(kept, DONE_DIRECTORY, SLEEP_MIN,
File "/storage/emulated/0/Download/Compressed/Revanced/restore-missing-youtube-watch-history-main/main.py", line 176, in download_videos
with yt_dlp.YoutubeDL(opts) as ydl:
^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 687, in init
self._request_director = self.build_request_director(_REQUEST_HANDLERS.values(), _RH_PREFERENCES)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 4128, in build_request_director
cookiejar=self.cookiejar,
^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/functools.py", line 1001, in get
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 4049, in cookiejar
return load_cookies(
^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/cookies.py", line 91, in load_cookies
extract_cookies_from_browser(browser_name, profile, YDLLogger(ydl), keyring=keyring, container=container))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/cookies.py", line 112, in extract_cookies_from_browser
return _extract_chrome_cookies(browser_name, profile, keyring, logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/yt_dlp/cookies.py", line 273, in _extract_chrome_cookies
raise FileNotFoundError(f'could not find {browser_name} cookies database in "{search_root}"')
FileNotFoundError: could not find chrome cookies database in "/data/data/com.termux/files/home/.config/google-chrome"

Here is the screenrecord of the same: https://github.com/ReVanced/restore-missing-youtube-watch-history/assets/70900505/6a8b4c8b-d9dd-433d-aa5a-f85f6552d514

When I use Google Takeout to export 'YouTube and YouTube Music' in JSON format, the resulting watch_history.JSON file is missing the same watch history as the youtube web client and other apps are.

When looking at My Activity in Google, if I select to filter to YouTube only, the watch history is also missing.

If I select NO filtering, and manually scroll instead, I can find my missing watch history, where it is marked as from "youtube.com", as compared to the watch history which DOES show in the apps and IS in the watch_history.JSON file, which show as from"YouTube"

I don't know why I am different to other people but it's only the ReVanced client watched videos which have ended up this way - I am sure there are others who have this issue.

Can we adapt the script and instructions to be able to parse the missing watch history from a My Activity export to resolve this?

Image from my My Activity feed showing a properly recorded history entry (present in watch_history.JSON)
https://imgur.com/ksorTzk

Image from my My Activity feed showing a history entry created by ReVanced which is missing in watch_history.JSON and youtube history.
https://imgur.com/vWzsxSP

This repository is highly specialized to ReVanced and is not general-purpose. For that reason this should be stated, otherwise #8 would be unjust.

While executing the script (Using Google Chrome browser) as an Administrator, i got the following:

PS C:\Users\USER\Documents\GIT\restore-missing-youtube-watch-history> python.exe .\main.py
Found 18024 videos to watch
Found 16033 videos to watch after de-duplication
Traceback (most recent call last):
  File "C:\Users\USER\Documents\GIT\restore-missing-youtube-watch-history\main.py", line 210, in <module>
    main()
  File "C:\Users\USER\Documents\GIT\restore-missing-youtube-watch-history\main.py", line 109, in main
    download_videos(kept, DONE_DIRECTORY, SLEEP_MIN,
  File "C:\Users\USER\Documents\GIT\restore-missing-youtube-watch-history\main.py", line 176, in download_videos
    with yt_dlp.YoutubeDL(opts) as ydl:
         ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 687, in __init__
    self._request_director = self.build_request_director(_REQUEST_HANDLERS.values(), _RH_PREFERENCES)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 4128, in build_request_director
    cookiejar=self.cookiejar,
              ^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\functools.py", line 1001, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 4049, in cookiejar
    return load_cookies(
           ^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\cookies.py", line 91, in load_cookies
    extract_cookies_from_browser(browser_name, profile, YDLLogger(ydl), keyring=keyring, container=container))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\cookies.py", line 112, in extract_cookies_from_browser
    return _extract_chrome_cookies(browser_name, profile, keyring, logger)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\cookies.py", line 281, in _extract_chrome_cookies
    cursor = _open_database_copy(cookie_database_path, tmpdir)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\yt_dlp\cookies.py", line 1042, in _open_database_copy
    shutil.copy(database_path, database_copy_path)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 419, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\USER\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Network\\Cookies'

It seem linked to the way Chrome locks the cookies files (borisbabic/browser_cookie3#180 (comment)), sadly, the command line option --disable-features=LockProfileCookieDatabase does not seem to fix my issue
Shadow Copy work around can maybe fix this issue as mentioned in the previous comment.
Does anyone have a clue ?

YouTube Shorts is a product name. The code references them as "shorts" whereas it should be called "Shorts" or "YouTube Shorts" to be exact.