Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-62275

Prometheus scraping: default to legacy validation/escaping scheme until utf-8 is fully supported by prom-operator

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • 4.20.0
    • 4.19.z, 4.20.0
    • Monitoring
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • MON Sprint 277
    • 1
    • Done
    • Release Note Not Required
    • Hide
      Before this fix, Prometheus (both for the platform and UWM) would negotiate and accept UTF-8 metrics, even though the monitoring stack does not yet fully support UTF-8.
      With this fix, Prometheus will no longer accept UTF-8 metrics.
      Show
      Before this fix, Prometheus (both for the platform and UWM) would negotiate and accept UTF-8 metrics, even though the monitoring stack does not yet fully support UTF-8. With this fix, Prometheus will no longer accept UTF-8 metrics.
    • None
    • None
    • None
    • None

      This is a clone of issue OCPBUGS-62166. The following is the description of the original issue:
      —
      Description of problem:

      
Prometheus v3 (introduced in 4.19) defaults to UTF-8 for metrics validation and escaping.
However, the monitoring stack is not yet fully ready for this change: the Prometheus Operator does not fully support UTF-8.

As a result, the current behavior is inconsistent. For example, a UTF-8 target can be scraped successfully, but adding recording rules or alerts for it may not be possible. See https://issues.redhat.com/browse/MON-4280.

Even once UTF-8 support is complete, we may want it disabled by default, requiring users to explicitly enable it per target. Otherwise, a target silently switching from `foo_bar` to `foo.bar` could be confusing.

4.x updates should avoid introducing such breaking changes.

We should default to the legacy validation scheme for scraping.

Note that promtool e.g. may still use UTF-8 for validation.

      Version-Release number of selected component (if applicable):

      How reproducible:

      Steps to Reproduce:

          1.
    2.
    3.

      Actual results:

      Expected results:

      Additional info:

              rh-ee-amrini Ayoub Mrini
              rh-ee-amrini Ayoub Mrini
              None
              None
              Junqi Zhao Junqi Zhao
              Eliska Romanova Eliska Romanova
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: