Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-38622

Prometheus no longer accepts samples of the same series with different timestamps

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.16
    • Monitoring
    • None
    • -
    • Important
    • Yes
    • MON Sprint 258
    • 1
    • False
    • Hide

      None

      Show
      None

      Description of problem:

          See https://github.com/prometheus/prometheus/issues/14503 for more details
          

      Version-Release number of selected component (if applicable):

          4.16
          

      How reproducible:

      
          

      Steps to Reproduce:
      1. Make Prometheus scrape a target that exposes multiple samples of the same series with different explicit timestamps, for example:

      # TYPE requests_per_second_requests gauge
      # UNIT requests_per_second_requests requests
      # HELP requests_per_second_requests test-description
      requests_per_second_requests 16 1722466225604
      requests_per_second_requests 14 1722466226604
      requests_per_second_requests 40 1722466227604
      requests_per_second_requests 15 1722466228604
      # EOF
      

      2. Not all the samples will be ingested
      3. If Prometheus continues scraping that target for a moment, the PrometheusDuplicateTimestamps will fire.
      Actual results:

      
          

      Expected results: all the samples should be considered (or course if the timestamps are too old or are too in the future, Prometheus may refuses them.)

      
          

      Additional info:

           Regression introduced in Prometheus 2.52.
          Proposed upstream fixes: https://github.com/prometheus/prometheus/pull/14683 https://github.com/prometheus/prometheus/pull/14685 
          

            rh-ee-amrini Ayoub Mrini
            rh-ee-amrini Ayoub Mrini
            Junqi Zhao Junqi Zhao
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: