What did you do? I was running prom/prometheus:v2.37.6 <h3 dir

Let's start by making <a href="https://prometheus.io/docs/prometheus/latest/storage/#o

Let's start by making <a href="https://prometheus.io/docs/prometheus/late

Default --storage.tsdb.retention.time about prometheus HOT 6 OPEN

jonathan-dev commented on June 4, 2024

Default --storage.tsdb.retention.time

from prometheus.

Comments (6)

machine424 commented on June 4, 2024

Thanks for reporting this. /flags returns the flags the user has set and the defaults as known to kingpin (the flags parser we use)

prometheus/cmd/prometheus/main.go

Lines 806 to 813 in 76b0318

 boilerplateFlags := kingpin.New("", "").Version("") 

 for _, f := range a.Model().Flags { 

 if boilerplateFlags.GetFlag(f.Name) != nil { 

 continue 

 } 

 cfg.web.Flags[f.Name] = f.Value.String() 

 }

For --storage.tsdb.retention.time, the default value we inform kingpin about is indeed 0s:

prometheus/cmd/prometheus/main.go

Lines 345 to 346 in 76b0318

 serverOnlyFlag(a, "storage.tsdb.retention.time", "How long to retain samples in storage. When this flag is set it overrides \"storage.tsdb.retention\". If neither this flag nor \"storage.tsdb.retention\" nor \"storage.tsdb.retention.size\" is set, the retention time defaults to "+defaultRetentionString+". Units Supported: y, w, d, h, m, s, ms."). 

 SetValue(&newFlagRetentionDuration)

as the zero value of

prometheus/cmd/prometheus/main.go

Line 251 in 76b0318

newFlagRetentionDuration model.Duration

15d is only used when neither --storage.tsdb.retention.time nor --storage.tsdb.retention.size is set. It's done here:

prometheus/cmd/prometheus/main.go

Lines 571 to 574 in 76b0318

 if cfg.tsdb.RetentionDuration == 0 && cfg.tsdb.MaxBytes == 0 { 

 cfg.tsdb.RetentionDuration = defaultRetentionDuration 

 level.Info(logger).Log("msg", "No time or size retention was set so using the default time retention", "duration", defaultRetentionDuration) 

 }

outside of kingpin (as it doesn’t seem to support such advanced dependencies/conditions)

What we can do, is make https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects as explicit as the description in

prometheus/cmd/prometheus/main.go

Line 251 in 76b0318

newFlagRetentionDuration model.Duration

and maybe add a sentence on the /flags page about how we consider the defaults.

We could set newFlagRetentionDuration as well to defaultRetentionDuration in

prometheus/cmd/prometheus/main.go

Lines 571 to 574 in 76b0318

 if cfg.tsdb.RetentionDuration == 0 && cfg.tsdb.MaxBytes == 0 { 

 cfg.tsdb.RetentionDuration = defaultRetentionDuration 

 level.Info(logger).Log("msg", "No time or size retention was set so using the default time retention", "duration", defaultRetentionDuration) 

 }

to make kingpin know about the "real/final" default, but I don't think it's a good idea.

from prometheus.

jonathan-dev commented on June 4, 2024

Thanks a lot for the quick and detailed answer. The reason I noticed this behavior was because the cloudflare pint tool was complaining about my retention time being 0s and thus being shorter than some of the ranges that I was querying (like 2m). After that I was a bit confused whats going on because when I was running a query it seemed like that there was a history kept but I was not able to validate that it was the 15days claimed or just the cleanup running infrequently. After checking I just noticed that the /status endpoint shows me the 15d retention time when I do not use the flags which is kind of what I was looking for. Noting it under the operational aspects probably would also helpful I think.

from prometheus.

machine424 commented on June 4, 2024

Let's start by making https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects as clera as

prometheus/cmd/prometheus/main.go

Lines 345 to 346 in 76b0318

 serverOnlyFlag(a, "storage.tsdb.retention.time", "How long to retain samples in storage. When this flag is set it overrides \"storage.tsdb.retention\". If neither this flag nor \"storage.tsdb.retention\" nor \"storage.tsdb.retention.size\" is set, the retention time defaults to "+defaultRetentionString+". Units Supported: y, w, d, h, m, s, ms."). 

 SetValue(&newFlagRetentionDuration)

then.

from prometheus.

tesla59 commented on June 4, 2024

Let's start by making https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects as clera as

prometheus/cmd/prometheus/main.go

Lines 345 to 346 in 76b0318

serverOnlyFlag(a, "storage.tsdb.retention.time", "How long to retain samples in storage. When this flag is set it overrides \"storage.tsdb.retention\". If neither this flag nor \"storage.tsdb.retention\" nor \"storage.tsdb.retention.size\" is set, the retention time defaults to "+defaultRetentionString+". Units Supported: y, w, d, h, m, s, ms.").

SetValue(&newFlagRetentionDuration)

then.

Hi @machine424,
I have updated the doc as mentioned by u in the PR #13982
I'd like to further work on this task. Will work on this and let u know

from prometheus.

kushalShukla-web commented on June 4, 2024

/assign

from prometheus.

Default --storage.tsdb.retention.time about prometheus HOT 6 OPEN

Comments (6)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	boilerplateFlags := kingpin.New("", "").Version("")
	for _, f := range a.Model().Flags {
	if boilerplateFlags.GetFlag(f.Name) != nil {
	continue
	}

	cfg.web.Flags[f.Name] = f.Value.String()
	}

	serverOnlyFlag(a, "storage.tsdb.retention.time", "How long to retain samples in storage. When this flag is set it overrides \"storage.tsdb.retention\". If neither this flag nor \"storage.tsdb.retention\" nor \"storage.tsdb.retention.size\" is set, the retention time defaults to "+defaultRetentionString+". Units Supported: y, w, d, h, m, s, ms.").
	SetValue(&newFlagRetentionDuration)

	if cfg.tsdb.RetentionDuration == 0 && cfg.tsdb.MaxBytes == 0 {
	cfg.tsdb.RetentionDuration = defaultRetentionDuration
	level.Info(logger).Log("msg", "No time or size retention was set so using the default time retention", "duration", defaultRetentionDuration)
	}