Skip to content

charts: make honorTimestamps configurable for linkerd-proxy PodMonitor#15080

Merged
alpeb merged 1 commit into
linkerd:mainfrom
bezarsnba:add-honortimestamps
May 15, 2026
Merged

charts: make honorTimestamps configurable for linkerd-proxy PodMonitor#15080
alpeb merged 1 commit into
linkerd:mainfrom
bezarsnba:add-honortimestamps

Conversation

@bezarsnba
Copy link
Copy Markdown
Contributor

When scraping linkerd-proxy metrics with Prometheus, samples are sometimes dropped due to duplicate timestamps with different values, producing warnings like:

Error on ingesting samples with different value but same timestamp scrape_pool=podMonitor/linkerd/linkerd-proxy

This is especially common in large clusters with frequent pod restarts and short scrape intervals, causing operational noise and alert fatigue. Add an optional honorTimestamps field to the proxy and serviceMirror PodMonitor endpoints in the linkerd-control-plane Helm chart. When set to false, Prometheus uses the scrape time instead of exporter-provided timestamps, preventing duplicate timestamp drops while keeping backward compatibility for users who do not set the field.

Usage:

  podMonitor:
    proxy:
      honorTimestamps: false
    serviceMirror:
      honorTimestamps: false

Existing behavior is preserved when the field is not configured.

Fixes #14905

Signed-off-by: bezarsnba@gmail.com

@bezarsnba bezarsnba requested a review from a team as a code owner March 21, 2026 13:41
@bezarsnba bezarsnba force-pushed the add-honortimestamps branch from 9f3e232 to e44acbb Compare March 21, 2026 13:44
@alpeb alpeb self-assigned this May 8, 2026
Copy link
Copy Markdown
Member

@alpeb alpeb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bezarsnba , any reason why this wasn't applied to the control plane PodMonitor?
Also, you'll need to run go test ./... -update in order to update the test fixtures.

@bezarsnba bezarsnba force-pushed the add-honortimestamps branch from 00e6ba2 to 0f18845 Compare May 15, 2026 01:04
**Why is this PR necessary?**
In clusters with frequent pod restarts, Prometheus may drop scrapes
due to duplicate timestamps reported by the exporter. Allowing
honorTimestamps to be disabled lets Prometheus use scrape time
instead, preventing those drops.

**What does it do?**
Adds an optional honorTimestamps field to the controller, proxy and
serviceMirror PodMonitors in the linkerd-control-plane chart. The
field is rendered only when set, preserving backward compatibility.

**References:**
N/A

**Notes:**
N/A

Signed-off-by: bezarsnba <bezarsnba@gmail.com>
@bezarsnba bezarsnba force-pushed the add-honortimestamps branch from 0f18845 to 0f7c910 Compare May 15, 2026 01:08
@bezarsnba
Copy link
Copy Markdown
Contributor Author

Thanks for the review @alpeb!

Good catch — there was no particular reason, I just missed the control-plane PodMonitor. I've now extended honorTimestamps to all three PodMonitors (controller, serviceMirror, proxy) for consistency.

Also ran go test ./... -update and the regenerated golden fixtures are included in the latest push.

@bezarsnba bezarsnba requested a review from alpeb May 15, 2026 13:46
@alpeb alpeb merged commit 3e3e787 into linkerd:main May 15, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make honorTimestamps configurable for linkerd-proxy PodMonitor

2 participants