Upload and stream media from the cloud with or without encryption. Cache all new and recently streamed media locally to access quickly and reduce API calls
License: MIT License
I recently upgraded to the version using MergerFS. It turns out that I have the following error when copying a file :
root@1696351a9d23:/input cp /input/aFile.test /output/aFile.test
cp: error writing ‘/output/aFile.test’: Input/output error
cp: failed to extend ‘/output/aFile.test’: Input/output error
cp: failed to close ‘/output/aFile.test’: Input/output error
root@1696351a9d23:/input#
@Dulanic Looks like there's a connection between my mistake and the issue you reported on the MergerFS github.
So this morning I noticed my mount dropped and I tried to run mountcheck and this is all I see...
The only fix was for me to reboot. Mongodb is installed and running properly? Not sure why rebooting fixes it, but the script wont?
[ 2018-01-16@11:03:12 ] Mount has dropped. Remount in progress.
[ 2018-01-16@11:03:12 ] Unmounting...
[ 2018-01-16@11:03:12 ] Unmounting decrypted cloud mountpoint -> /opt/cloud/.cloud-decrypt
fusermount: failed to unmount /opt/cloud/.cloud-decrypt: Invalid argument
[ 2018-01-16@11:03:12 ] Unmounting encrypted cloud mountpoint -> /opt/cloud/.cloud-encrypt
fusermount: failed to unmount /opt/cloud/.cloud-encrypt: Invalid argument
[ 2018-01-16@11:03:12 ] Unmounting final library mountpoint -> /local-media
fusermount: failed to unmount /local-media: Invalid argument
[ 2018-01-16@11:03:12 ] Unmounted successfully
[ 2018-01-16@11:03:12 ] Mounting...
[ 2018-01-16@11:03:12 ] ###### Start Mount ######
[ 2018-01-16@11:03:17 ] Could not connect to Mongodb. Retrying in 5 seconds.
[ 2018-01-16@11:03:22 ] Could not connect to Mongodb. Retrying in 5 seconds.
[ 2018-01-16@11:03:27 ] Could not connect to Mongodb. Retrying in 5 seconds.
[ 2018-01-16@11:03:32 ] Could not connect to Mongodb. Retrying in 5 seconds.
[ 2018-01-16@11:03:37 ] Could not connect to Mongodb. Retrying in 5 seconds.
[ 2018-01-16@11:03:42 ] Could not connect to Mongodb. Please make sure Mongodb is running before mounting
[ 2018-01-16@11:03:42 ] Mount failed
[ 2018-01-16@11:03:42 ] Mounted successfully
I follow the manual untill last step. I even reinstall the system. it awyas hang here.
Do you want to start mounting now [Y/n]? y
This may take a while because Plexdrive needs to cache your files
[ 2018-02-26@13:37:39 ] ###### Start Mount ######
[ 2018-02-26@13:37:44 ] Mounting Google Drive with Plexdrive in mountpoint: /root/cloud-media-scripts/.cloud-encrypt
[ 2018-02-26@13:37:44 ] Mounted successfully with Plexdrive
[ 2018-02-26@13:37:49 ] Mounting decrypted Google Drive with Rclone in mountpoint: /root/cloud-media-scripts/.cloud-decrypt
[ 2018-02-26@13:37:49 ] Mounted successfully with Rclone
[ 2018-02-26@13:37:54 ] Could not find decrypted mountpoint: /root/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-26@13:38:04 ] Could not find decrypted mountpoint: /root/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-26@13:38:14 ] Could not find decrypted mountpoint: /root/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-26@13:38:24 ] Could not find decrypted mountpoint: /root/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-26@13:38:34 ] Could not find decrypted mountpoint: /root/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds.
[ 2018-02-26@13:38:44 ] Could not read mountpoint: /root/cloud-media-scripts/.cloud-decrypt
[ 2018-02-26@13:38:44 ] Mount failed
how clould I fix this?
Thanks
I'm still seeing issues.. just tried to save a file in the local-media folder and it won't allow it. All users are uid 1000, including the dockers. Could it be because root runs the scripts? Is there another workaround? I cant find one so far.
recated@mediaserver:/local-media/Media/Shows$ echo 123 > test
-bash: test: Permission denied
Couldn't import episode /downloads/complete/tv/*redacted*/*redacted*.mkv: Access to the path "/tv/*redacted*/Season 3" is denied.
/tv for Sonarr is mapped to /local-media/Media/Shows.
Config:
#!/bin/bash
# shellcheck disable=SC2034
###############################################################################
# DIRECTORIES
###############################################################################
media_dir="/opt/cloud"
# Cloud directories
cloud_encrypt_dir="${media_dir}/.cloud-encrypt"
cloud_decrypt_dir="${media_dir}/.cloud-decrypt"
# Local directory
local_decrypt_dir="/local-decrypt"
plexdrive_temp_dir="/chunks"
# Media directory (FINAL)
local_media_dir="/local-media"
# Encrypt media (0 means no encryption)
encrypt_media=1
###############################################################################
# UNION-FS
###############################################################################
ufs_bin="/usr/bin/unionfs"
ufs_options="-o cow,allow_other,direct_io,nonempty,auto_cache,sync_read"
###############################################################################
# PLEXDRIVE
###############################################################################
plexdrive_dir="${media_dir}/plexdrive"
plexdrive_bin="${plexdrive_dir}/plexdrive-linux-amd64"
mongo_database="plexdrive"
mongo_host="localhost"
mongo_user=""
mongo_password=""
plexdrive_options="--temp=${plexdrive_temp_dir} -o allow_other,read_only --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M"
###############################################################################
# RCLONE
###############################################################################
rclone_dir="${media_dir}/rclone"
rclone_bin="${rclone_dir}/rclone"
rclone_config="${rclone_dir}/rclone.conf"
rclone_options="--buffer-size 500M --checkers 16"
rclone_mount_options="${rclone_options} --read-only --allow-non-empty --allow-other --max-read-ahead 30G"
# Rclone endpoints
rclone_cloud_endpoint="gd-crypt:"
rclone_local_endpoint="local-crypt:"
google_drive_media_directory="Media" # Empty means media files are in root
###############################################################################
# MISC. CONFIG
###############################################################################
date_format="+%F@%T"
# Select to remove media files based on 'time', 'space' or 'instant'
remove_files_based_on="instant"
#remove_files_based_on="time"
#remove_files_based_on="space"
# Remove media files based on 'time'
remove_files_older_than=60 # In days
# Remove media files based on 'space'
remove_files_when_space_exceeds=500 # In Gigabytes
freeup_atleast=300 # In Gigabytes
When running mount.remote, after using setup.sh, the first mount (.cloud-encrypt) and second mount (.cloud-decrypt) succeed, then .cloud-decrypt cannot be found.
Fresh Ubuntu 16.04 install, running from root for testing.
[ 2017-10-19@03:46:45 ] ###### Start Mount ######
[ 2017-10-19@03:46:50 ] Mounting Google Drive with Plexdrive in mountpoint: /home/services/cloud-media-scripts/.cloud-encrypt
[ 2017-10-19@03:46:50 ] Mounted successfully with Plexdrive
[ 2017-10-19@03:46:55 ] Mounting decrypted Google Drive with Rclone in mountpoint: /home/services/cloud-media-scripts/.cloud-decrypt
[ 2017-10-19@03:46:55 ] Mounted successfully with Rclone
I am just running into problem after problem and I'm sure a lot is due to me. However, now I am seeing this when attempting to mount?
[OPT/CLOUD/PLEXDRIVE/PLEXDRIVE-LINUX-AMD64] [2017-11-03 09:06] ERROR : Could not open mongo db connection
Yet if I type mongo i get the cli interface and if I do show dbs I see plexdrive?
redacted@mediaserver:/var/lib/mongodb$ mongo
MongoDB shell version v3.4.10
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.10
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten]
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten]
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten]
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2017-11-03T09:02:38.550-0500 I CONTROL [initandlisten]
> show dbs
local 0.078GB
plexdrive 0.078GB
>
So I know last year plexidrive added the ability to delete files. I have edited my config etc... to remove ready only and change RO to RW. So far it's working great,
Am I correct that UFS should write to local-decrypt 1st as UFS lists the local decrypt dir as 1st level? Seems to be working so far. This would allow us to delete in plex and have it remove the files from GD.
My changes currently:
Per testing it is properly loading files to decrypt when downloading.
ufs_options="-o cow,allow_other,direct_io,nonempty,auto_cache,sync_read,uid=$uid,gid=$gid"
plexdrive_options="--temp=${plexdrive_temp_dir} -o allow_other --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M"
ufs_mounts="${local_decrypt_dir}=RW:${cloud_decrypt_dir}=RW"
If you think it's a decent idea to add, I can help code it in the config/mount.remote.
So I am now seeing a new issue with this and I'm pretty sure it's a rclone issue but I don't know why it keeps happening? Every single day I start seeing oauth errors. Any reasoning I migth be running into this? My main rclone is no problem, so I could remove the --config and just use that, but I'd like to drill down to why it keeps happening.
2018/01/12 13:39:27 Failed to create file system for "gd-crypt:/Media/Shows/Black Mirror/Season 4": failed to make remote "gd:Media/k27ti189mhh81aujfcjfe7ct90/34dic7kr6v2e6sg2dh2pq9g8ak/su4dqm6ldtrsi9c7pk8tp1houo/r19nfjmb1pmclsidsakqeibn2k" to wrap: couldn't read info about Drive: Get https://www.googleapis.com/drive/v2/about?alt=json: oauth2: cannot fetch token: 401 Unauthorized
Response: {
"error" : "invalid_client",
"error_description" : "The OAuth client was not found."
}
I have only 20g space on vps. so I would like to stream on /root/cloud-media-scripts/.cloud-decrypt dir. My idea is download file to /root/cloud-media-scripts/.local-decrypt and upload it right after download completed and remove the file. and stream the file on /root/cloud-media-scripts/.cloud-decrypt dir.
is this the correct way to use this script?
every time I execute /root/cloud-media-scripts/scripts/cloudupload the script will upload file from /root/cloud-media-scripts/.local-decrypt. However after upload completed. I still can not find the newly uploaded file under /root/cloud-media-scripts/.cloud-decrypt. Only I can found it when I reboot the server.
How can I make the file appear in /root/cloud-media-scripts/.cloud-decrypt immediately right after I upload the file?
Thanks.
Hi
Can I use this with plexdrive 5 ?
I already have version 5 setup and I have no root access.
So at some point I noticed in plex I only had like 2 shows and I wasnt sure why. I ran the mount check and it came back as everything successfully mounted. Obviously something dropped but wasn't catching what dropped. I am assuming it had to be rclone since it was all my gdrive stuff that wasn't showing.
This also caused radarr/sonarr to go hey missing files, lets go find them again. Any other logs I could pull or things I could look into to see why it didnt identify it as dropped? My plex has been scanning for hours trying to rebuild the metadata lol
I'm not sure what else to check for the root cause? I tried to umount.remote and then mount.remote but it just hang on .cloud-decrypt mounting. But when I reboot the server it mounts no problem.
Status.log:
[ 2018-01-07@20:15:01 ] /local-media is mounted properly [ 2018-01-07@20:15:01 ] /opt/cloud/.cloud-encrypt is mounted properly [ 2018-01-07@20:15:01 ] /opt/cloud/.cloud-decrypt is mounted properly [ 2018-01-07@20:15:01 ] /chunks is mounted properly [ 2018-01-07@20:20:01 ] /local-media is mounted properly [ 2018-01-07@20:20:01 ] /opt/cloud/.cloud-encrypt is mounted properly [ 2018-01-07@20:20:01 ] /opt/cloud/.cloud-decrypt is mounted properly [ 2018-01-07@20:20:01 ] /chunks is mounted properly [ 2018-01-07@20:30:01 ] /local-media is mounted properly [ 2018-01-07@20:30:01 ] /opt/cloud/.cloud-encrypt is mounted properly [ 2018-01-07@20:30:01 ] /opt/cloud/.cloud-decrypt is mounted properly [ 2018-01-07@20:30:01 ] /chunks is mounted properly [ 2018-01-07@20:35:01 ] /local-media is mounted properly [ 2018-01-07@20:35:01 ] /opt/cloud/.cloud-encrypt is mounted properly [ 2018-01-07@20:35:01 ] /opt/cloud/.cloud-decrypt is mounted properly [ 2018-01-07@20:35:01 ] /chunks is mounted properly [ 2018-01-07@20:40:01 ] /local-media is mounted properly [ 2018-01-07@20:40:01 ] /opt/cloud/.cloud-encrypt is mounted properly [ 2018-01-07@20:40:01 ] /opt/cloud/.cloud-decrypt is mounted properly [ 2018-01-07@20:40:01 ] /chunks is mounted properly
Mount.log - You can see the last time it mounted 2 days ago and it dropped something today, but not sure when.
[ 2018-01-05@17:26:26 ] ###### Mounted successfully ###### [ 2018-01-05@18:38:11 ] ###### Start Mount ###### [ 2018-01-05@18:38:16 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud/.cloud-encrypt [ 2018-01-05@18:38:16 ] Mounted successfully with Plexdrive [ 2018-01-05@18:38:21 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud/.cloud-decrypt [ 2018-01-05@18:38:21 ] Mounted successfully with Rclone [ 2018-01-05@18:38:26 ] Mounting final library with UnionFS in mountpoint: /local-media [ 2018-01-05@18:38:26 ] Mounted successfully with UnionFS [ 2018-01-05@18:38:26 ] ###### Mounted successfully ###### [ 2018-01-07@20:28:59 ] ###### Start Mount ###### [ 2018-01-07@20:29:04 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud/.cloud-encrypt [ 2018-01-07@20:29:04 ] Mounted successfully with Plexdrive [ 2018-01-07@20:29:09 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud/.cloud-decrypt [ 2018-01-07@20:29:09 ] Mounted successfully with Rclone [ 2018-01-07@20:29:14 ] Mounting final library with UnionFS in mountpoint: /local-media [ 2018-01-07@20:29:14 ] Mounted successfully with UnionFS [ 2018-01-07@20:29:14 ] ###### Mounted successfully ######
Disregard, I'm an idiot.
Request to update docker container with latest version of plexdrive (5.0) and latest version of rclone
I am running into a permission issue /w local-media folder. For some reason radarr keeps getting permission denials with directory. Radarr is run in docker.
Couldn't import episode /downloads/complete/tv/redacted: Access to the path "/tv/redacted is denied.
All source files in /downloads are owned by user 1000. /tv is a docker folder that is mapped to /local-media/Media/Shows. It seems like there is a write issue to local-media for some reason? That should write it to my local-decrypt?
So I've been trying to configure everything, but no matter what it seems that the files are never encrypted and uploaded to Google Drive. Seems to never encrypt anything either. Here's the log from cloudupload.
[ 2017-10-14@12:53:07 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/season-specials.tbn to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:07 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/season01-banner.jpg to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:08 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/season-specials-banner.jpg to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:09 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/.actors/Tom_Green.tbn to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/.actors.
[ 2017-10-14@12:53:09 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/.actors/Anna_Popplewell.tbn to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/.actors.
[ 2017-10-14@12:53:10 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/tvshow.nfo to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:11 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/folder.jpg to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:12 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/poster.jpg to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
[ 2017-10-14@12:53:12 ] Transfering file -> /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/PleXXX/TV Shows/Halo 4 - Forward Unto Dawn/season01.tbn to Google Drive in /PleXXX/TV Shows/Halo 4 - Forward Unto Dawn.
root@PleXXX:/mnt/Library/Plex-Servers/PleXXX#
Hey madslundt. So when I was playing around with my fork I noticed that the rclone.conf was not properly setting the local-crypt and was missing the GD folder after the local-crypt even though it shows it in the template. I tried to look through to see how you set that, but didn't figure it out off hand. Want to take a look at that?
[local-crypt]
type = crypt
remote = /opt/cloud-media-scripts/.cloud-encrypt
filename_encryption = standard
remote = <ENCRYPTED_FOLDER>/**<GOOGLE_DRIVE_MEDIA_DIRECTORY>**
FYI, my googledrive in config is Media.
Hey mads, can you take a look at this, I'm just realizing that cloudupload isn't working... I converted the find to a echo and ir errors outs.
dulanic@mediaserver:/opt/cloud-media-scripts/scripts$ ./cloudupload
[ 2018-03-16@14:35:12 ] ###### Start cloud upload ######
stat: missing operand
Try 'stat --help' for more information.
[ 2018-03-16@14:35:12 ] ###### Cloud upload ended successfully ######
I copied it to test and edited to echo it....
dulanic@mediaserver:/opt/cloud-media-scripts/scripts$ ./test
[ 2018-03-16@14:36:24 ] ###### Start cloud upload ######
find /local-decrypt/Media -type f -print0 | xargs -0 stat --format '%Y :%y %n' | sort -nr | cut -d: -f2- | awk '{===; print ./test}'
dulanic@mediaserver:/opt/cloud-media-scripts/scripts$ find /local-decrypt/Media -type f -print0 | xargs -0 stat --format '%Y :%y %n' | sort -nr | cut -d: -f2- | awk '{===; print ./test}'
awk: line 1: syntax error at or near ==
awk: 1: unexpected character '.'
stat: missing operand
Try 'stat --help' for more information.
I would recommend changing a few things in your main Config file and then in your setup.sh. I would do this myself, but honestly don't use github enough to know how to properly do pulls and commits.
I would recommend adding a "install_folder" variable that will contain only cloud-media-scripts contents so it is separate from any "Media."
I modified it and have it working on my end, but I run my VM on a different device than my storage. I would also recommend putting in a mount option for your media as well and have that built into the "mount.remote" script. This mount would allow you to mount your media directories directly into your local-decrypt folder.
#!/bin/bash
# shellcheck disable=SC2034
###############################################################################
# DIRECTORIES
###############################################################################
media_dir="/mnt/Library"
install_dir="/opt/cloud-media-scripts"
# Cloud directories
cloud_encrypt_dir="${media_dir}/.cloud-encrypt"
cloud_decrypt_dir="${media_dir}/.cloud-decrypt"
# Local directory
local_decrypt_dir="${media_dir}/.local-decrypt/Media"
plexdrive_temp_dir="${media_dir}/.local-decrypt/plexdrive"
# Media directory (FINAL)
local_media_dir="${media_dir}/media"
# Encrypt media (0 means no encryption)
encrypt_media=0
I then replaced all the ${media_dir} with ${install_dir} so that rclone/plexdrive install into the scripts folder rather than having it install into the "Media" folder as well. To me its cleaner, and you can move things around without breaking stuff. The only issue you run into with this is using fstab to mount your directories. You will need to have your rclone/plexdrive mounted before you can mount your Media into local-decrypt.
EDIT - Not all instances of ${media_dir} were changed, only the ones that were relevant.
Example of setup.sh attached.
setup.zip
My current setup is like this:
/mnt/Library/.local-decrypt/
/mnt/Library/.cloud-decrypt/
/mnt/Library/media/
Then after mount.remote is run, I have my media mounted into /mnt/Library/.local-decrypt/
This way I have my already setup media being moved into the cloud without killing my currently running plex. All my media is still in the same location on my NAS, and when it finishes uploading I can just switch my library locations very easily.
This is how I set this up for me, so you may or may not see any of this as helpful.
Hello,
I am attempting to use this version to see if it works around the docker issue I opened. However,m when I run setup.sh it fails /w this:
No Rclone config exists
setup.sh: 69: setup.sh: Bad substitution
I am running ubuntu 16.04.
So my mount stopped mounting. So I reran the curl command and now I'm getting this...
redacted@mediaserver:/opt$ sudo /bin/bash <( curl -Ls https://github.com/madslundt/cloud-media-scripts/raw/master/INSTALL ) /opt/cloud
/bin/bash: /dev/fd/63: No such file or directory
redacted@mediaserver:/opt$ (23) Failed writing body
So I got all my stuff uploaded to google drive now, but when the rmlocal script runs I receive the following error:
du: cannot access ‘' /mnt/Library/Plex-Servers/PleXXX/local-decrypt/PleXXX/[P]/Misc/Season 02/S0002E0006.mp4'’: No such file or directory
/mnt/Library/Plex-Servers/PleXXX/scripts/rmlocal: line 126: 0 + : syntax error: operand expected (error token is "+ ")
Looking at the scripts, I think it has to do with the escape characters not existing when using $n as a parameter for fileSize=$(du -sb "$n" | awk '{print $1}') on line 124.
find "${local_decrypt_dir}" -type f -exec stat -c "%x %n" {} \; | sort | awk '{$1=$2=$3=""; print $0}' |
while IFS= read -r n; do
if [ "$addedSpace" -gt "$freeup" ]; then
spaceInGb=$(($addedSpace / 1000 / 1000 / 1000))
spaceLeft=$(($(du -sb "$local_decrypt_dir" | awk '{print $1}') / 1000 / 1000 / 1000))
echo "[ $(date ${date_format}) ] Removed ${spaceInGb} GB. Media in total ${spaceLeft} GB."
break
fi
This is not an issue. Apologizes if this isn't the place for this.
Seeing how you're creating a cloud of content from ex: homeserver. Could there be a script you can run lets say on a Windows desktop at an other location and mount GDrive locally?
I have already used the current script to mount on linux. This is aimed towards Windows.
I'm sure this may be an easy fix. Also their may be a possiblity I did something wrong. However I've spent an entire day attempting to figure out the issue.
[ 2018-02-10@11:15:21 ] ###### Start Mount ###### [ 2018-02-10@11:15:26 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud-media-scripts/.cloud-encrypt [ 2018-02-10@11:15:26 ] Mounted successfully with Plexdrive [ 2018-02-10@11:15:31 ] Mounting decrypted Google Drive with Rclone in mountpoint: /opt/cloud-media-scripts/.cloud-decrypt [ 2018-02-10@11:15:31 ] Mounted successfully with Rclone [ 2018-02-10@11:15:36 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:15:46 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:15:56 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:06 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:16 ] Could not find decrypted mountpoint: /opt/cloud-media-scripts/.cloud-decrypt. Retrying in 10 seconds. [ 2018-02-10@11:16:26 ] Could not read mountpoint: /opt/cloud-media-scripts/.cloud-decrypt [ 2018-02-10@11:16:26 ] Mount failed
What else can I provide to get this up and running?
Submitted commit to fix. logs folder was not created in the script used by curl.
So this was running for me as mentioned previously. I purged all of my current install for this and reran the scripts. I am finding again that it won't mount now. Below is what I am seeing and not sure what the issue is....
CLI:
dulanic@mediaserver:/opt/cloud/scripts$ ./mount.remote
[ 2018-01-08@10:05:50 ] ###### Start Mount ######
[ 2018-01-08@10:05:55 ] Mounting Google Drive with Plexdrive in mountpoint: /opt/cloud/.cloud-encrypt
[ 2018-01-08@10:05:55 ] Mounted successfully with Plexdrive
[ 2018-01-08@10:06:00 ] Could not find encrypted mountpoint: /opt/cloud/.cloud-encrypt. Retrying in 10 seconds.
[ 2018-01-08@10:06:10 ] Could not find encrypted mountpoint: /opt/cloud/.cloud-encrypt. Retrying in 10 seconds.
[ 2018-01-08@10:06:20 ] Could not find encrypted mountpoint: /opt/cloud/.cloud-encrypt. Retrying in 10 seconds.
[ 2018-01-08@10:06:30 ] Could not find encrypted mountpoint: /opt/cloud/.cloud-encrypt. Retrying in 10 seconds.
[ 2018-01-08@10:06:40 ] Could not find encrypted mountpoint: /opt/cloud/.cloud-encrypt. Retrying in 10 seconds.
[ 2018-01-08@10:06:50 ] Could not read mountpoint: /opt/cloud/.cloud-encrypt
[ 2018-01-08@10:06:50 ] Mount failed
My rclone is good....
dulanic@mediaserver:/opt/cloud/rclone$ rclone --config rclone.conf ls gd-crypt:
1024 .fstab.swp
4 Media/test
306 Media/Books/.driveinfo.calibre
2 Media/Books/.metadata.calibre
My config:
#!/bin/bash
# shellcheck disable=SC2034
###############################################################################
# DIRECTORIES
###############################################################################
media_dir="/opt/cloud"
# Cloud directories
cloud_encrypt_dir="${media_dir}/.cloud-encrypt"
cloud_decrypt_dir="${media_dir}/.cloud-decrypt"
# Local directory
local_decrypt_dir="${media_dir}/.local-decrypt/Media"
plexdrive_temp_dir="/chunks"
# Media directory (FINAL)
local_media_dir="/local-media"
# Encrypt media (0 means no encryption)
encrypt_media=1
###############################################################################
# UNION-FS
###############################################################################
ufs_bin="/usr/bin/unionfs"
# Mount user Id's. Replace $(id with a different ID if you need to override.
uid=1000
gid=1000
ufs_options="-o cow,allow_other,direct_io,nonempty,auto_cache,sync_read,uid=$uid,gid=$gid"
###############################################################################
# PLEXDRIVE
###############################################################################
plexdrive_dir="${media_dir}/plexdrive"
plexdrive_bin="${plexdrive_dir}/plexdrive-linux-amd64"
mongo_database="plexdrive"
mongo_host="192.168.2.155"
mongo_user=""
mongo_password=""
plexdrive_options="--temp=${plexdrive_temp_dir} -o allow_other,read_only --clear-chunk-max-size=300G --clear-chunk-age=24h --chunk-size=10M"
###############################################################################
# RCLONE
###############################################################################
rclone_dir="${media_dir}/rclone"
rclone_bin="${rclone_dir}/rclone"
rclone_config="${rclone_dir}/rclone.conf"
rclone_options="--buffer-size 500M --checkers 16"
rclone_mount_options="${rclone_options} --read-only --allow-non-empty --allow-other --max-read-ahead 30G"
# Rclone endpoints
rclone_cloud_endpoint="gd-crypt:"
rclone_local_endpoint="local-crypt:"
google_drive_media_directory="Media" # Empty means media files are in root
###############################################################################
# Plex (empty trash)
###############################################################################
plex_url="http://192.168.2.155:32400"
plex_token="redacted"
###############################################################################
# MISC. CONFIG
###############################################################################
date_format="+%F@%T"
# Select to remove media files based on 'time', 'space' or 'instant'
#remove_files_based_on="instant"
#remove_files_based_on="time"
remove_files_based_on="space"
# Remove media files based on 'time'
remove_files_older_than=60 # In days
# Remove media files based on 'space'
remove_files_when_space_exceeds=500 # In Gigabytes
freeup_atleast=300 # In Gigabytes
Is this possible to add a requirements section to the readme
I run into a loop trouble with the mongoDB, and we didn't get log to get the behavior quickly, maybe a good point to add log on plexdrive and rclone
Good work @madslundt 👍
I've experienced several buffering issues when using google drive as storage. Plexdrive has an option named 'chunk-load-ahead'. Could you expose it ?
By the way, I've seen that your merged a PR allowing deletion on the cloud mount. Would that work with google drive ? At the moment, when radarr/sonarr replaces content, I get a duplicate file on google drive as it can't delete the old. Would that PR fix this ?
Hey,
Still somewhat new with git stuff, and don't really know how to pull and make changes and commit em (haven't spent much time looking it either). But anyways, there seems to be a weird character in the config file. I noticed it created a folder named "plexdrive?" and it's not usable. Here is what it looks like:
# Local directory
local_decrypt_dir="${media_dir}/.local-decrypt/Media"
plexdrive_temp_dir="${media_dir}/.local-decrypt/plexdrive"​
Changing to this seems to be fine, not sure where it comes from tho.
# Local directory
local_decrypt_dir="${media_dir}/.local-decrypt/Media"
plexdrive_temp_dir="${media_dir}/.local-decrypt/plexdrive"
It isn't visible in nano when opening the script on debian, but there is definitely some kind of character there as i can delete the "blank" characters within nano. Either way, just wanted to help out.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.