Git Product home page Git Product logo

plex-cluster's Introduction

Plex Cluster

Synchronizes the watched and timeline status between any number of Plex servers all using standard Plex APIs.

This project is in BETA TESTING.

Plex Cluster contains two applications:

  • Plex Cluster Proxy, which is installed alongside every Plex server and acts as a proxy (using nginx) between Plex Media Server and the internet. This application's job is to pass any requests that come to it along to the Plex server while catching any requests that mark watched status and also forwarding them on the Plex Cluster Manager. There is one instance of Plex Cluster Proxy for each Plex server.
  • Plex Cluster Manager is then used to synchronize the status between Plex servers. There is only one instance of Plex Cluster Manager which is used by all instances of Plex Cluster Proxy.

An example use case:

  1. Plex client requests to watch a show with GIUD (unique identifier) ABC from the TV Shows library on Plex Server 1
  2. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1
  3. The user stops the show midway.
  4. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1, but is also passed along to Plex Cluster Manager
  5. Plex Cluster Manager takes the request and checks for any other Plex server with a library named TV Shows and GUID ABC
  6. For any other servers it finds, such as Plex Server 2, it forwards the request on to them and the show is instantly marked as watched up to that midway point.
  7. User switches over to Plex Server 2 and sees that they are midway through show ABC

Features

  • Syncs watched status on-demand, right away
  • Uses standard Plex APIs without accessing the database
  • Can work across multiple Plex servers
  • Scheduled "full update" can completely sync user's watched status for any user that has been active on each Plex server since running Plex Cluster
  • Syncs media that is contained on both (or multiple) servers without erroring when media does not exist
  • Works for Plex.tv or Managed users

To Do

  • Support "full sync" for users on some Roku models and other devices that encapsulate a temporary token in the request header (this does not impact on-demand sync)

Requirements

  • Each Plex server must have a dedicated DNS or Dynamic DNS hostname

Installation

The best way to use Plex Cluster is with Docker. You can follow the Dockerfiles to set up the proxy manually, however if you're not running Docker for your services, you're missing out.

This installation follows this example:

  • 2 Plex servers, one at home called Home and one remote called Remote
  • Home has...
    • a public DNS record of plex-home.mydomain.com
    • a Plex server running on 192.168.0.10:32400
    • a Plex Cluster Proxy instance running on https://plex-home.mydomain.com:32401
    • a wildcard SSL certificate for *.mydomain.com in the /certs folder
  • Remote has...
    • a public DNS record of plex-remote.mydomain.com
    • a Plex server running on 192.168.1.10:32400
    • a Plex Cluster Proxy instance running on https://plex-remote.mydomain.com:32401
    • a Plex Cluster Manager instance running on https://plex-remote.mydomain.com:32402
    • a wildcard SSL certificate for *.mydomain.com in the /certs folder

Setting up the Home instance

First, we need a Plex server, and a copy of Plex Cluster Proxy running at home. These can be spun up with the following docker-compose.yml:

version: '3.7'

services:

  # A proxy based on nginx that sits between the Plex server and
  # the internet. Every time a request is made to Plex, if that
  # request is to mark status, an API call is made
  # to plex-cluster-manager.
  #
  # There is one of these for every Plex server.
  plex-cluster-proxy:
    image: nowsci/plex-cluster-proxy
    container_name: plex-cluster-proxy
    environment:
      - PLEX_IP=192.168.0.10 # The IP address of the Plex server's NIC
      - PLEX_PORT=32400      # The port Plex is listening on
      - HTTPS_HOST=plex-home.mydomain.com # The host that Plex Cluster Proxy will listen on
      - HTTPS_PORT=32401                  # The port that Plex Cluster Proxy will listen on
      - RESOLVERS=8.8.4.4 8.8.8.8 # DNS servers that Plex Cluster Proxy should use
      - SSL_CERTIFICATE=/certs/fullchain.pem   # Mapped location of the below SSL cert
      - SSL_CERTIFICATE_KEY=/certs/privkey.pem # Mapped location of the below SSL cert
      - CLUSTER_MANAGER=https://plex-remote.mydomain.com:32402 # URL of Plex Cluster Manager
      - CLUSTER_MANAGER_TOKEN=JgUNaJ6DcB7FhhYGcUpaa9DwPy       # CHANGE THIS: A random token that Plex Cluster Proxy will use to authenticate with Manager
    volumes:
      - /etc/localtime:/etc/localtime:ro # Keeps time in sync
      - ./certs/fullchain.pem:/certs/fullchain.pem:ro # Mapped path to the SSL certificate
      - ./certs/privkey.pem:/certs/privkey.pem:ro     # Mapped path to the SSL certificate
      - plex-cluster-proxy:/data # Docker volume where log data is stored for processing
    ports:
      - 32401:32401 # Port mapping (same as HTTPS_PORT), forward this port on your firewall
    restart: always # Restart on failure

  # The Home Plex server
  # This assumes you have Plex Pass
  plex:
    image: plexinc/pms-docker:plexpass
    container_name: plex
    environment:
      - TZ=America/New_York
    network_mode: bridge
    ports:
      - "192.168.0.10:32400:32400/tcp" # The port and IP to listen on, do not forward this port on your firewall
    volumes:
      - ./plex/plex/config:/config # Where you want your Plex Library/Config folder to be
      - /storage/transcode/plex:/transcode # A path for transcoding
      - /storage/transcode/plex/Sync+:/config/Library/Application Support/Plex Media Server/Cache/Transcode/Sync+ # To keep sync transcoding in the transcode folder
      - /storage/transcode/plex/Sync:/config/Library/Application Support/Plex Media Server/Cache/Transcode/Sync   # To keep sync transcoding in the transcode folder
      - /storage/media:/media # Media folder
    restart: always

volumes:
  plex-cluster-proxy:    

After you spin this up with docker-compose up -d, navigate in your browser to http://192.168.0.10:32400/web and login to your new server. A few key things to set up are:

  • Settings -> Remote Access -> Disable Remote Access (Don't worry, you'll still be able to get here through your custom URL)
  • Settings -> Network -> Click Show Advanced
  • Settings -> Network -> Secure connections -> Choose Required
  • Settings -> Network -> Custom server access URLs -> Enter https://plex-home.mydomain.com:32401

The home server setup is complete.

Setting up the Remote instance

Next we need a Plex server, a copy of Plex Cluster Proxy and a copy of Plex Cluster Manager running remotely. These can be spun up with the following docker-compose.yml:

version: '3.7'

services:

  # A proxy based on nginx that sits between the Plex server and
  # the internet. Every time a request is made to Plex, if that
  # request is to mark status, an API call is made
  # to plex-cluster-manager.
  #
  # There is one of these for every Plex server.
  plex-cluster-proxy:
    image: nowsci/plex-cluster-proxy
    container_name: plex-cluster-proxy
    environment:
      - PLEX_IP=192.168.1.10 # The IP address of the Plex server's NIC
      - PLEX_PORT=32400      # The port Plex is listening on
      - HTTPS_HOST=plex-remote.mydomain.com # The host that Plex Cluster Proxy will listen on
      - HTTPS_PORT=32401                    # The port that Plex Cluster Proxy will listen on
      - RESOLVERS=8.8.4.4 8.8.8.8 # DNS servers that Plex Cluster Proxy should use
      - SSL_CERTIFICATE=/certs/fullchain.pem   # Mapped location of the below SSL cert
      - SSL_CERTIFICATE_KEY=/certs/privkey.pem # Mapped location of the below SSL cert
      - CLUSTER_MANAGER=http://plex-cluster-manager:32402 # URL of Plex Cluster Manager (local Docker URL in this case)
      - CLUSTER_MANAGER_TOKEN=JgUNaJ6DcB7FhhYGcUpaa9DwPy  # CHANGE THIS: A random token that Plex Cluster Proxy will use to authenticate with Manager
    volumes:
      - /etc/localtime:/etc/localtime:ro # Keeps time in sync
      - ./certs/fullchain.pem:/certs/fullchain.pem:ro # Mapped path to the SSL certificate
      - ./certs/privkey.pem:/certs/privkey.pem:ro     # Mapped path to the SSL certificate
      - plex-cluster-proxy:/data # Docker volume where log data is stored for processing
    ports:
      - 32401:32401 # Port mapping (same as HTTPS_PORT), forward this port on your firewall
    restart: always # Restart on failure

  # The service that sychronizes the Plex servers. It takes the API calls
  # from plex-cluster-proxy, and reaches out to any other configured Plex
  # servers to set their status just as if the client connecting to the
  # original Plex server was connecting to that one instead.
  #
  # You will want to secure this behind an SSL proxy in the real world.
  #
  # There is only ever one of these.
  plex-cluster-manager:
    image: nowsci/plex-cluster-manager
    container_name: plex-cluster-manager
    environment:
      - CLUSTER_MANAGER_TOKEN=JgUNaJ6DcB7FhhYGcUpaa9DwPy # CHANGE THIS: A random token that Plex Cluster Proxy will use to authenticate with Manager
      - UPDATE_ON_START=true # Do a full sync on start
      - DEBUG=false # Output debug logs
      - FULL_SYNC_SCHEDULE=0 2 * * * # A cron-styled schedule for when to run the full sync
    volumes:
      - /etc/localtime:/etc/localtime:ro # Keeps time in sync
      - ./config/plex-cluster-manager.yml:/config/plex-cluster-manager.yml:ro # A config file, see below
      - plex-cluster-manager:/data # Where Plex Cluster Manager stores it's SQLite DB
    restart: always

  # The Remote Plex server
  # This assumes you have Plex Pass
  # You could also clone the Home server and remove Preferences.xml before starting
  plex:
    image: plexinc/pms-docker:plexpass
    container_name: plex
    environment:
      - TZ=America/New_York
    network_mode: bridge
    ports:
      - "192.168.1.10:32400:32400/tcp" # The port and IP to listen on, do not forward this port on your firewall
    volumes:
      - ./plex/plex/config:/config # Where you want your Plex Library/Config folder to be
      - /storage/transcode/plex:/transcode # A path for transcoding
      - /storage/transcode/plex/Sync+:/config/Library/Application Support/Plex Media Server/Cache/Transcode/Sync+ # To keep sync transcoding in the transcode folder
      - /storage/transcode/plex/Sync:/config/Library/Application Support/Plex Media Server/Cache/Transcode/Sync   # To keep sync transcoding in the transcode folder
      - /storage/media:/media # Media folder
    restart: always

volumes:
  plex-cluster-manager:
  plex-cluster-proxy:    

Before starting it up, you will need to create the plex-cluster-manager.yml configuration file (sample). This file looks like:

hosts:
  - host: plex.mydomain.com
    port: 11111
    token: xxxxxxxxxxxxxxxxxxxx
    #log: ./plex.log    # This is not commonly used. It is only used to tail a local log.

The token should be a Plex token for the administrative user on the Plex server. You can get a token from Plex using this script.

After you spin this up with docker-compose up -d, navigate in your browser to http://192.168.1.10:32400/web and login to your new server. A few key things to set up are:

  • Settings -> Remote Access -> Disable Remote Access (Don't worry, you'll still be able to get here through your custom URL)
  • Settings -> Network -> Click Show Advanced
  • Settings -> Network -> Secure connections -> Choose Required
  • Settings -> Network -> Custom server access URLs -> Enter https://plex-remote.mydomain.com:32401

The remote server setup is complete.

Final steps

Once everything is complete, login via https://plex.tv and if you watch the logs via docker-compose logs -ft and mark shows watched you should start seeing plex-cluster-proxy and plex-cluster-manager synchronize the status.

plex-cluster's People

Contributors

dependabot[bot] avatar fmstrat avatar simssj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

plex-cluster's Issues

Error response from daemon: pull access denied for nowsci/plex-cluster-proxy,

Error response from daemon: pull access denied for nowsci/plex-cluster-proxy, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

I have the feeling that this Docker image no longer exists. I was really looking forward to testing it, it's too bad. :(

JavaScript heap out of memory

im trying to fix that , but im not sure how

  Getting 1000 records starting at 63000
  Getting 1000 records starting at 64000
  Getting 1000 records starting at 65000
  Getting 1000 records starting at 66000
  Getting 1000 records starting at 67000
  Getting 1000 records starting at 68000
  Getting 1000 records starting at 69000
  Getting 1000 records starting at 70000
  Getting 1000 records starting at 71000
  Getting 1000 records starting at 72000
  Getting 1000 records starting at 73000

<--- Last few GCs --->

[18:0x55ee99d69c60]   158559 ms: Mark-sweep 2046.0 (2050.8) -> 2045.4 (2051.1) MB, 3764.2 / 0.0 ms  (average mu = 0.047, current mu = 0.002) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x55ee95460439]
    1: StubFrame [pc: 0x55ee954a0899]
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Security context: 0x1815f9f80921 <JSObject>
    2: match [0x1815f9f8cc81](this=0x04351f357ce9 <String[2]\: \n\n>,0x04351f357f11 <JSRegExp <String[#5]: ^\s*$>>)
    3: /* anonymous */ [0x3df618b526d9] [/app/node_modules/xml2js/lib/parser.js:~173] [pc=0x92a5ecf7264](this=0x1253185f8f81 <SAXParser map = 0x1c0062bfae11>)
    4: closeTag [0x3df0ccac02a1] [/app/node_mod...

some ideas with docker-compose

Hi,
are you ok if :

  • plex-cluster-manager and plex-cluster-proxy are created by docker-compose ?
  • we change to 3 docker-compose file and one environment file (.env) ?
    - docker-compose-remote.yml
    - docker-compose-local.yml
    - docker-compose-plex.yml

Build image docker plex-cluster-manager error

I can't build image plex-cluster-manager.

Step 1/7 : FROM node:alpine
---> 87c43f8d8077
Step 2/7 : COPY --chown=node:node src /app
---> 9e12f64c2bbb
Step 3/7 : WORKDIR /app
---> Running in 80c56c30d2fd
Removing intermediate container 80c56c30d2fd
---> 9708e9975e3e
Step 4/7 : RUN mkdir /data && chown node:node /data
---> Running in ecedad4362ad
Removing intermediate container ecedad4362ad
---> e76985d32fb8
Step 5/7 : USER node
---> Running in 5e8c9f916c2e
Removing intermediate container 5e8c9f916c2e
---> 8d071f934f61
Step 6/7 : RUN npm install
---> Running in 9fb4d1bc14ae

[email protected] install /app/node_modules/sqlite3
node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.1.1/node-v83-linux-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v83 ABI, musl) (falling back to source compile with node-gyp)
gyp ERR! find Python
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON
gyp ERR! find Python checking if "python" can be used
gyp ERR! find Python - "python" is not in PATH or produced an error
gyp ERR! find Python checking if "python2" can be used
gyp ERR! find Python - "python2" is not in PATH or produced an error
gyp ERR! find Python checking if "python3" can be used
gyp ERR! find Python - "python3" is not in PATH or produced an error
gyp ERR! find Python
gyp ERR! find Python **********************************************************
gyp ERR! find Python You need to install the latest version of Python.
gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
gyp ERR! find Python you can try one of the following options:
gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
gyp ERR! find Python (accepted by both node-gyp and npm)
gyp ERR! find Python - Set the environment variable PYTHON
gyp ERR! find Python - Set the npm configuration variable python:
gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
gyp ERR! find Python For more information consult the documentation at:
gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
gyp ERR! find Python **********************************************************
gyp ERR! find Python
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Python installation to use
gyp ERR! stack at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
gyp ERR! stack at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
gyp ERR! stack at PythonFinder. (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
gyp ERR! stack at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
gyp ERR! stack at exithandler (child_process.js:310:5)
gyp ERR! stack at ChildProcess.errorhandler (child_process.js:322:5)
gyp ERR! stack at ChildProcess.emit (events.js:315:20)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:274:12)
gyp ERR! stack at onErrorNT (internal/child_process.js:468:16)
gyp ERR! stack at processTicksAndRejections (internal/process/task_queues.js:84:21)
gyp ERR! System Linux 4.15.0-99-generic
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
gyp ERR! cwd /app/node_modules/sqlite3
gyp ERR! node -v v14.2.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64 --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
node-pre-gyp ERR! stack at ChildProcess. (/app/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack at ChildProcess.emit (events.js:315:20)
node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:1051:16)
node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Linux 4.15.0-99-generic
node-pre-gyp ERR! command "/usr/local/bin/node" "/app/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /app/node_modules/sqlite3
node-pre-gyp ERR! node -v v14.2.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/app/node_modules/sqlite3/lib/binding/node-v83-linux-x64 --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/node/.npm/_logs/2020-05-17T00_24_15_720Z-debug.log
The command '/bin/sh -c npm install' returned a non-zero code: 1

Development discussion

@hergonoway (will add others once they join)

You've gotten the first round of invites because you have offered to contribute to the project as well as test it out.

Any PRs should come into the develop branch. Any issues should be opened as tickets on GitHub, and please indicate if you are working on a patch for the fix.

Let me know if there are any questions!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.