unexpected status code 200 while pushing to pushgateway about k8up HOT 10 CLOSED

Hi @gi8lino

Can you reproduce this issue with wrestic 0.1.9? It reports that the code is 200, which indicates that it worked. Maybe there's a bug in the status check.

Are the metrics there after the backup ran?

Best
Simon

from k8up.

Hi @Kidswiss

With wrestic 0.1.9 there is no log entry if something was send to the pushgateway.
Yes, it is a 200, but wrestic reports it as an error. As in the log written, the error occurs in line 145 in backup.go.

I found an issue. Maybe the client_golang is outdated.

I also found another issue to this problem which refers to actual prometheus go code here. Maybe this can help you...

Yes, the metrics are in the pushgateway.

from k8up.

Hi @gi8lino

Thanks for digging into this, I'll have a look at that.

Best
Simon

from k8up.

Hi @gi8lino

Finally had some time to get to this.

A simple update of the prometheus libraries in wrestic solves the issues.

0902 15:26:06.819188   22487 handler.go:44] wrestic/statsHandler/promStats "level"=0 "msg"="sending prometheus stats"  "url"="http://localhost:9091/"
I0902 15:26:06.838591   22487 backup.go:86] wrestic/backup "level"=0 "msg"="backup finished, sending snapshot list"  
I0902 15:26:06.838612   22487 snapshots.go:37] wrestic/snapshots "level"=0 "msg"="getting list of snapshots"  
I0902 15:26:07.423279   22487 handler.go:68] wrestic/statsHandler/webhookStats "level"=0 "msg"="sending webhooks"  "url"="http://localhost:8091"

Opened k8up-io/wrestic#23 for this.

Thanks for reporting the issues!

Best
Simon

from k8up.

Hi @Kidswiss

As soon as a new wrestic release is created, I will test it.

Best
gi8

from k8up.

Hi @Kidswiss

Any news about the pr?

Best
gi8

from k8up.

@gi8lino Sorry for the long radio silence.

I can build a temp image for further testing. If some other unforseen issues would arise.

from k8up.

@Kidswiss

A temp image would help and I will test it!

from k8up.

@gi8lino

You can use vshn/wrestic:prom_fix for testing. Looking forward to your results.

from k8up.

@Kidswiss

At the beginning I had some issues with the backups. It didn't find the buckets. I delete all the buckets and then the backups worked properly.
Now also the status code from prometheus seems to be processed correctly:

I0927 23:02:10.242398       1 handler.go:44] wrestic/statsHandler/promStats "level"=0 "msg"="sending prometheus stats"  "url"="http://prometheus-pushgateway.monitoring.svc.cluster.local:9091"
I0927 23:02:10.259513       1 backup.go:86] wrestic/backup "level"=0 "msg"="backup finished, sending snapshot list"  
I0927 23:02:10.259549       1 snapshots.go:37] wrestic/snapshots "level"=0 "msg"="getting list of snapshots"  

from k8up.