Git Product home page Git Product logo

crop's Introduction

made-with-golang License: GPL v3 last commit (master) Discord Contributing Donate

crop

CLI tool to run upload/sync jobs with rclone.

Example Configuration

rclone:
  config: /home/seed/.config/rclone/rclone.conf
  path: /usr/bin/rclone
  stats: 30s
  live_rotate: false
  service_account_remotes:
    '/opt/rclone/service_accounts/crop':
      - tv
      - movies
      - music
      - 4k_movies
      - source_4k_movies
      - staging
  global_params:
    default:
      move:
        - '--order-by=modtime,ascending'
        - '--transfers=8'
        - '--delete-empty-src-dirs'
      sync:
        - '--fast-list'
        - '--tpslimit-burst=50'
        - '--max-backlog=2000000'
        - '--track-renames'
        - '--use-mmap'
        - '--no-update-modtime'
        - '--drive-chunk-size=128M'
      dedupe:
        - '--dedupe-mode=newest'
        - '--tpslimit=5'
uploader:
  - name: cloudbox_unionfs
    enabled: true
    check:
      limit: 360
      type: age
    hidden:
      cleanup: true
      enabled: true
      folder: /mnt/local/.unionfs-fuse
      type: unionfs
    local_folder: /mnt/local/Media
    remotes:
      clean:
        - 'gdrive:'
        - 'staging:'
      move: 'staging:/Media'
      move_server_side:
        - from: 'staging:/Media'
          to: 'gdrive:/Media'
    rclone_params:
      global_move: default
      move_server_side:
        - '--delete-empty-src-dirs'
      global_dedupe: default
  - name: tv
    enabled: true
    check:
      limit: 1440
      type: age
    local_folder: /mnt/local/Media/TV
    remotes:
      move: 'tv:/Media/TV'
    rclone_params:
      global_move: default
  - name: movies
    enabled: true
    check:
      limit: 720
      type: age
    local_folder: /mnt/local/Media/Movies
    remotes:
      move: 'movies:/Media/Movies'
    rclone_params:
      global_move: default
syncer:
  - name: 4k_movies
    enabled: true
    source_remote: 'source_4k_movies:/'
    remotes:
      sync:
        - '4k_movies:/'
      dedupe:
        - '4k_movies:/'
    rclone_params:
      global_sync: default
      global_dedupe: default

Example Commands

  • Clean - Perform clean for associated uploader job(s).

crop clean --dry-run

crop clean -u google

crop clean

  • Upload - Perform uploader job(s)

crop upload --dry-run

crop upload -u google

crop upload -u google --no-check

crop upload

  • Sync - Perform syncer job(s)

crop sync --dry-run

crop sync -s google

crop sync

crop sync -p 2

  • Manual - Perform manual sync/copy job(s)

crop manual --copy --src remote1:/Backups --dst remote2:/Backups --sa /opt/service_accounts -- --dry-run

crop manual --sync --src remote1:/Backups --dst remote2:/Backups --sa /opt/service_accounts --dedupe --


Notes

  • Make use of --dry-run and -vv to ensure your configuration is correct and yielding expected results.

  • live_rotate will enable on-demand live-rotation of service accounts for a customized build of rclone / gclone.

Credits

  • rclone - Without this awesome tool, this project would not exist!
  • sasync - Sync ideas and service account technique originated from here.

Donate

If you find this project helpful, feel free to make a small donation to the developer:

crop's People

Contributors

l3uddz avatar renovate-bot avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.