Git Product home page Git Product logo

rt-retention's Introduction

rt-retention

rt-retention is a simple JFrog CLI plugin to facilitate enforcing retention policies in Artifactory.

How it works

rt-retention deletes artifacts matching FileSpecs in a given directory. It also has templating capabilities to help maintain similar retention policies.

To set up your retention policies, define them as FileSpecs (or templates thereof). To enforce them, set up a humble cron job running the plugin.

Installing rt-retention

docker

  • docker pull verifa/rt-retention

manual

  • Download the latest version from the Releases page
  • Place its contents in your ~/.jfrog/plugins directory (If there’s no plugins directory under .jfrog, create it)

Running rt-retention

commands

run

  • Usage: jf rt-retention run [command options] <filespecs-path>

  • Arguments:

    • filespecs-path (Path to the FileSpecs file/dir)
  • Options:

    • --dry-run do not delete artifacts [Default: true]
    • --recursive recursively find FileSpecs files in the given dir [Default: false]
    • --threads amount of worker threads [Default: 3]

expand

  • Usage: jf rt-retention expand [command options] <config-path> <templates-path> <output-path>

  • Arguments:

    • config-path (Path to the JSON config file)
    • output-path (Path to output the generated FileSpecs)
  • Options:

    • --threads amount of worker threads [Default: 3]

running with verbose output

For verbose output to aid in debugging, set JFROG_CLI_LOG_LEVEL=DEBUG.

Templating

The expand command can generate FileSpecs from Go templates, populated with values from a JSON config file.

The JSON config file may contain one or more policy definitions. Each specifies the template file it uses, as well as a list of entries. The template file path should be relative to the config file. Each entry will result in a FileSpecs file being generated.

Example config.json:

{
    "my-junk-repositories": {
        "template": "templates/entire-repo.json",
        "entries": [
            { "Repo": "scratch-local" }
        ]
    },
    "my-dev-repositories": {
        "template": "templates/older-than.json",
        "entries": [
            { "Repo": "generic-dev-local", "Time": "3w" },
            { "Repo": "libs-snapshot-local", "Time": "1y" }
        ]
    }
}

The templates themselves are Go text templates. Properties from the JSON config entry will be used to populate the template.

Example templates/delete-older-than.json:

{
  "files": [
    {
      "aql": {
        "items.find": {
          "repo": "{{.Repo}}",
          "created": {
            "$before": "{{.Time}}"
          }
        }
      }
    }
  ]
}

Pass the config file, the templates directory and the output directory to the expand command to generate the retention policies.

$ jf rt-retention expand config.json templates/ policies/
[🔵Info] Reading config file
[🔵Info] Parsing config JSON
[🔵Info] Expanding policies
[🔵Info]   [ my-junk-repositories ]
[🔵Info]   [ my-dev-repositories ]
[🔵Info] Done

Extra templating properties

deleteParent

Policies can set deleteParent to delete the parent paths of what the FileSpecs would match, rather than the matches themselves.

This is useful for deleting entire directories if they contain an artifact matching certain conditions, or deleting Docker images based on conditions on their manifest file.

{
    "template-one": {
        "template": "templates/template.json",
        "deleteParent": true,
        "entries": [
            { "Repo": "scratch-local" }
        ]
    }
}

nameProperty

Policies can optionally set a nameProperty, which can be used to change the generated FileSpecs' filename to the value of the given property key. Without it, FileSpecs are generated using the name of the template, and the index of their entry.

The below example uses the Repo property value to use as the FileSpecs' filename.

{
    "template-one": {
        "template": "templates/template.json",
        "nameProperty": "Repo",
        "entries": [
            { "Repo": "scratch-local" },
            { "Repo": "dev-local" }
        ]
    }
}

Expanding the templates will result in the following generated files:

output/
 `- template-one/
     |- scratch-local-0.json
     `- dev-local-0.json

rt-retention's People

Contributors

jlarfors avatar praqma-thi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

naveen225

rt-retention's Issues

not a valid name or character

Hey Verifa.

Got a problem with users creating paths and artifacts with 'special characters' , like '?'.

I have not dig:ed into your code and looked into if you can intercept and "url-encode" these before executing deletes.

But if you do, would be nice since, as always, we the operators need to address and fix these issues.

Example:
{ "errors": [ { "status": 500, "message": "Invalid path. '?' is not a valid name character: vce-art-level0-local/art/NGAH/feature/?NGAH-6806-seperate-pedal-behaviour/M1ECUx86/0.6.1-37-j0nix/j0nixrulez.zip" } ] }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.