Git Product home page Git Product logo

docker-curator's Introduction

Curator in docker

This is dockerized version of elasticsearch curator,tool to manage time-based indices.

the docker image for curator baseon the python:3.6-alpine(this image size only 30mb)

docker hub https://hub.docker.com/r/anjia0532/docker-curator/

Automated build Docker Pulls

Elastic Curator Official Image : untergeek/curator

My Curator Image : anjia0532/docker-curator

Curator is breaking into version dependent releases. Curator 6.x will work with Elasticsearch 6.x, Curator 7.x will work with Elasticsearch 7.x, Curator 8.x will work with Elasticsearch 8.x

Usage

docker-compose.yml

version: '2'

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:x.y.z
    ports:
    - 9200:9200/tcp
    #environment:
    #    - http.host=0.0.0.0
    #    - transport.host=127.0.0.1
    # Uncomment this section to have elasticsearch data loaded from a volume
    #volumes:
    #    - /data/path/to/host/:/usr/share/elasticsearch/data

  # default version :latest 
  # more information https://hub.docker.com/repository/docker/anjia0532/docker-curator/tags?page=1&ordering=last_updated
  curator:
    image: anjia0532/docker-curator
    environment:
        UNIT_COUNT: 10
        UNIT: days
        ES_HOST: elasticsearch
    depends_on:
      - elasticsearch
    volumes:
        - /config/path/to/host/:/config

action_file.yml(default)

env default value

  • UNIT_COUNT: 1
  • UNIT: months
  • ES_HOST: 127.0.0.1
actions:
  1:
    action: delete_indices
    description: >-
       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for logstash-
       prefixed indices. Ignore the error if the filter does not result in an
       actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: ${UNIT:months}
      unit_count: ${UNIT_COUNT:1}
      exclude:
#  2:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for filebeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: filebeat-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:
#  3:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for heartbeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: heartbeat-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:
#  4:
#    action: delete_indices
#    description: >-
#       Delete indices older than ${UNIT_COUNT:1} ${UNIT:months} (based on index name), for heartbeat-
#       prefixed indices. Ignore the error if the filter does not result in an
#       actionable list of indices (ignore_empty_list) and exit cleanly.
#    options:
#      ignore_empty_list: True
#      timeout_override:
#      continue_if_exception: True
#      disable_action: False
#    filters:
#    - filtertype: pattern
#      kind: prefix
#      value: zipkin-
#      exclude:
#    - filtertype: age
#      source: name
#      direction: older
#      timestring: '%Y.%m.%d'
#      unit: ${UNIT:months}
#      unit_count: ${UNIT_COUNT:1}
#      exclude:

config_file.yml

env default value

  • USE_SSL: False
  • TIMEOUT: 120
  • MASTER_ONLY: True
---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
# old version 5.x-7.x
client:
  hosts:
    - ${ES_HOST:127.0.0.1}
  port: 9200
  url_prefix:
  use_ssl: ${USE_SSL:False}
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  timeout: ${TIMEOUT:120}
  master_only: ${MASTER_ONLY:True}

# curator 8.0+ version
elasticsearch:
  client:
    hosts: ${ES_HOST}
    cloud_id:
    bearer_auth:
    opaque_id:
    request_timeout: ${TIMEOUT:120}
    http_compress:
    verify_certs: False
    ca_certs:
    client_cert:
    client_key:
    ssl_assert_hostname:
    ssl_assert_fingerprint:
    ssl_version:
  other_settings:
    master_only: ${MASTER_ONLY:True}
    skip_version_test:
    username:
    password:
    api_key:
      id:
      api_key:

logging:
  loglevel: INFO
  logfile:
  logformat: default
  #blacklist: ['elasticsearch', 'urllib3']

cron job

every day at 0:00 am exec

0 0 * * * curator --config /config/config_file.yml /config/action_file.yml

Curator's doc

Full reference is available at: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html

docker-curator's People

Contributors

anjia0532 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

docker-curator's Issues

Update to recent version with support for Elasticsearch 7.0

First of all, thank you for this useful docker image!

Please update to the most recent curator version (currently still on master, release to pypi should come soon) with support for ES 7.

Maybe also push a version for 5.6.0.

Also may I suggest that you push your image into docker hub with version tags so that one can recognize which curator is baked into which published image.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.