Pause Backup after pre definied time about k8up HOT 12 CLOSED

@SchoolGuy Sure if you're willing to write up something just open a PR. It should probably go into the "How-to guides" section.

from k8up.

Hi @SchoolGuy

Thanks for your suggestion. But unfortunately, this looks like a very specific feature and I think it's not something that would fit into K8up.

But I can give you a suggestion how you could achieve the same functionality. You could create a schedule that starts the backup at the desired time. Then you could create a cronjob that deletes the backup spawned by the schedule at your desired time. That effectively kills the job. The great thing about restic (the underlying backup tool) is that it can resume aborted backups where it left of.

from k8up.

@Kidswiss That is good enough for me. Do you think it is worth to document this in your documentation somewhere. If yes I will happily contribute this once I got it working. If no, I would close the issue.

from k8up.

Update: Next week I will get started creating the long running backups and thus will experiment with this.

from k8up.

@Kidswiss It turns out you maybe have a cheap way to integrate this after all.

If I understand the k8s docs correctly, there is the feature called activeDeadlineSeconds which would be pretty useful. You correctly pointed out that restic can continue backups where it was aborted. This would mean that we could have a guaranteed window where the backup runs. The current solution I am beginning to research is not very "clean" in the way that I either need to pass credentials via a secret and then run kubectl commands (via a CronJob) or I need to create a controller in Golang (which would essentially be the solution you could then integrate).

Especially since you have #581 & #399 I believe this is a duplicate.

from k8up.

@SchoolGuy: yeah I was thinking about the activeDeadlineSeconds, too. But when the job exceeds the deadline it will terminate with a failed state.

That is, the Job termination mechanisms activated with .spec.activeDeadlineSeconds and .spec.backoffLimit result in a permanent Job failure that requires manual intervention to resolve.

We currently rely on the failed state for the monitoring: https://github.com/k8up-io/k8up/blob/master/charts/k8up/templates/prometheus/prometheusrule.yaml#L41.

Which unfortunately makes supporting the activeDeadlineSeconds a bit more complicated.

from k8up.

@Kidswiss Why is that a problem? A backup that exceeds the timeout window will either signal that the storage has grown in an unforseen way or that there are other infrastructure problems (like slow network speed). Thus a failed state is appropriate imho. If you can't complete the backups you need to either fine-tune your alerts to your specific use-case or you need to invest into better infrastructure. In cloud native environments you anyway have basically unlimited bandwith due to the cloud providers infrastructure.

from k8up.

@SchoolGuy but in your use case it would be completely fine if the backup isn't finished in the time you specified, right? So then for your use case you would get false positives...

from k8up.

@Kidswiss Well yes it is fine but as said in my comment, then I need to cover my corner case with a silence for the alert. That of course is non-trivial but a single user should not be able define the feature for the majority of users. I would see the alert desired since I then have the signal for a big data transfer (which should not happen very often since restic is incremental).

from k8up.

@SchoolGuy Sounds fine to me.

Is it okay for you if I close this as a duplicate of #399? We should have a bit of capacity soon, so that an engineer can have a look at this feature.

from k8up.

@Kidswiss Yes that is a lovely outcome. If you need help for testing just ping me, I am willing to install a preview into my environment for such purposes.

from k8up.

Great! I'm closing here. For more questions or feedback please follow #399.

from k8up.