An Ansible role to provide an automated ZREPL Sink Client templated deployment.

• This does not install the Zrepl client, existing installation is required
• This does not create/generate the TLS certificates
  • Does push TLS certificates needed by the client to connect to the sink server
• Uses a templated configuration but not designed to support unique per instance configurations
  • Intended to be used for pushing out a common baseline sink configuration
  • Usually only a single Zrepl sink job is required, examples here show two sink jobs:
    1. One for ZFS datasets
    2. Another for ZFS ZVOLs (requires different attribute settings)
• Some per host settings can be defined via Ansible inventory or other ansible methods to override defaults
  • ZFS pool name, encrypted send, bandwidth limits
• Clients can be easily configured to enable Prometheus metrics
  • You then define your own scrape job on Prometheus to collect metrics.
• A Threshold script is provided which can be used to:
  • Prevent Zrepl from creating zero-byte snapshots
  • Or set a minimum size of bytes change to a dataset before a new snapshot is taken
  • Some people consider hundreds of small or zero-byte snapshots to be wasteful
  • NOTE: When a snapshot creation is denied, Zrepl logs this as a fatal error, it does not allow a way to report a non-fatal reason for refusing a snapshot. The error is harmless, but spams system logs.

• Target computers with Zrepl client already installed
• Generate TLS certificates for each client and sink server
• Pre-existing Zrepl sink server to connect with

• none

You can design your inventory file as you see fit, I like to use yaml format such as:

zrepl_sink_tls_client_group:
  hosts:
    k3s[01:03].mydomain.com:
      zrepl_zfs_pool_name: "rpool"
      zrepl_filesystems: |
        "{{ zrepl_zfs_pool_name }}/ROOT<": true,
        "{{ zrepl_zfs_pool_name }}/USERDATA<": true,

    k3s[04:06].mydomain.com:
      zrepl_zfs_pool_name: "{{ ansible_hostname }}"
      zrepl_filesystems: |
        "{{ zrepl_zfs_pool_name }}/ROOT<": true,

    k3s[01:05].mydomain.com:
      zrepl_send_bandwidth_limit: "15 MiB"

  vars:
    zrepl_sink_server: "truenas.mydomain.com:9448"
    enable_prometheus: true

• Hosts k3s01 through k3s03 use ZFS pool name rpool, whereas k3s04 through k3s06 use pool names matching the hostname.
• The filesystems to replicate use the standard zrepl conventions
• Hosts k3s01 - k3s05 have a 15 MiB replication limit set, whereas k3s06 has no restriction and can replicate at full link speed
• The zrepl_sink_server defines the hostname and port of the Zrepl sink server make a TLS connection with. I have it running on my TrueNAS Scale server.
• The enable_prometheus configures the client to expose Prometheus metrics

The defaults/main.yml can be used to configure the default values including multiple sink jobs for the client. These defaults can be overwritten by any of the typical Ansible variable methods such as using inventory variables.

# Define location to store zrepl.yml and TLS certificates
zrepl_config_path: "/etc/zrepl"

# Should Prometheus stanza in zrepl.yml be included
enable_prometheus: true

# Should Threshold Script be enabled
enable_threshold_script: false

# Default Name and port of remote Zrepl Sink Server
zrepl_sink_server: "zrepl.local:9448"

# Default Name of ZFS zpool to work with
zrepl_zfs_pool_name: "rpool"

# Define bandwidth replication limit when sending
# Examples: "10 MiB", "23.5 MiB", or disable: "-1 MiB"
zrepl_send_bandwidth_limit: -1 MiB

# Define Client TLS Certificate Names
zrepl_tls_ca_crt_name: "ca.crt"
zrepl_tls_cert_name: "{{ ansible_hostname }}.crt"
zrepl_tls_private_key_name: "{{ ansible_hostname }}.key"
zrepl_tls_server_cn: "sink-srv"

zrepl_snapjobs contains a named yaml list of one of more jobs to create on each client. Ideally you just need a single job for datasets and an optional second one for ZVOLs as they have different properties.

zrepl_snapjobs:
  - name: rancher           # snapjob_rancher
    filesystems: |
      "{{ zrepl_zfs_pool_name }}/rancher<": true,

• Follows normal patterns used in zrepl filesystem filters

    send_options: |
      encrypted: true
      send_properties: false
      bandwidth_limit:
        max: {{ zrepl_send_bandwidth_limit }}

• Follows normal zrepl send options
• See Zrepl Project on bandwidth limits notes
• IMPORTANT: Keep send_properties: false. Setting this to true can result in sink server issues / unbootable. This prevents sink server and its host mounts from getting clobbered.

    replication_options: |
      protection:
        initial: guarantee_resumability
        incremental: guarantee_incremental

• Follows normal zrepl replication options

    snapshotting_options: |
      type: periodic
      interval: 10m
      prefix: zrepl_
      timestamp_format: dense

• Follows normal zrepl periodic snapshoting options

This sections allows different policies for local (keep_sender) vs. remote sink server (keep_receiver) snapshot retention. The sink server typically has vast amount of storage and can retain snapshots much longer.

You can define multiple zrepl pruning policies for each keep_sender and keep_receiver.

    pruning_options: |
      keep_sender:
        # fade-out scheme for snapshots
        # - keep all created in the last hour
        # - then destroy snapshots such that we keep 6 each 1 hour apart
        # - then destroy snapshots such that we keep 1 each 1 day apart
        # - then destroy all older snapshots
        - type: grid
          grid: 1x1h(keep=all) | 6x1h | 1x1d
          regex: "^zrepl_.*"
        # keep all snapshots that don't have the `zrepl_` prefix.
        - type: regex
          negate: true
          regex: "^(zrepl)_.*"

    pruning_options: |
      keep_receiver:
        # Keep snapshots longer on sink server as it has more storage
        # keep all created in the last hour
        # keep 24 each 1 hour apart / keep 30 each 1 day apart / keep 4 each 30 days apart
        # then destroy all older snapshots
        - type: grid
          grid: 1x1h(keep=all) | 24x1h | 30x1d | 4x30d
          regex: "^zrepl_.*"
        # retain all non-zrepl snapshots on the sink server
        - type: regex
          negate: true
          regex: "^zrepl_.*"

This is an example playbook named zrepl_sink_tls_client.yml:

---
- name: Update zrepl TLS Client for Sink Server Replication
  hosts: zrepl_sink_tls_client_group
  become: true
  gather_facts: true

  roles:
    - role: zrepl_sink_tls_client

# Apply playbook to all hosts defined in group
ansible-playbook -i inventory zrepl_sink_tls_client.yml

# Use Ansible's limit parameter to specify individual hostname to run on:
ansible-playbook -i inventory zrepl_sink_tls_client.yml -l testlinux.mydomain.com