Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-7093

PTP metrics - Unexpected metrics for old phc2sys appears in metrics after modify ptpconfigs

XMLWordPrintable

    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Unexpected/outdated phc2sys related metrics are showing up in ptp metrics after modifying ptpconfigs. phc2sys can be associated to a different ptp4l config after ptpconfig change.
      
      Both old and new phc2sys metrics show up in ptp metrics, with the old metrics showing up with process status down, etc. 
      
      The metrics for old phc2sys should not show up in ptp metrics.  

      Version-Release number of selected component (if applicable):

      4.12

      How reproducible:

      100% (after phc2sys moves from one ptp4l config to another)

      Steps to Reproduce:

      1. Configure at least 2 ptp profiles
      2. Modify ptpconfigs to change ptp thresholds 
      3. Observe that phc2sys process got restarted and moved from one ptpconfig (ptp4l.4.config) to another one (ptp4l.1.config). 
      4. Check ptp metrics

      Actual results:

      old phc2sys metics associated with ptp4l.4.config remains in ptp metrics.
      
      # HELP openshift_ptp_offset_nsĀ 
      # TYPE openshift_ptp_offset_ns gauge
      openshift_ptp_offset_ns{from="master",iface="enp49s0fx",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_offset_ns{from="master",iface="ens4fx",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} -11
      openshift_ptp_offset_ns{from="master",iface="ens5fx",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} -5
      openshift_ptp_offset_ns{from="master",iface="ens7fx",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_offset_ns{from="phc",iface="CLOCK_REALTIME",node="helix49.ptp.lab.eng.bos.redhat.com",process="phc2sys"} -10
      # HELP openshift_ptp_process_restart_countĀ 
      # TYPE openshift_ptp_process_restart_count counter
      openshift_ptp_process_restart_count{config="ptp4l.0.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_process_restart_count{config="ptp4l.1.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="phc2sys"} 1
      openshift_ptp_process_restart_count{config="ptp4l.1.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_process_restart_count{config="ptp4l.2.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_process_restart_count{config="ptp4l.3.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_process_restart_count{config="ptp4l.4.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="phc2sys"} 2
      openshift_ptp_process_restart_count{config="ptp4l.4.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      openshift_ptp_process_restart_count{config="ptp4l.5.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 3
      # HELP openshift_ptp_process_status 0 = DOWN, 1 = UP
      # TYPE openshift_ptp_process_status gauge
      openshift_ptp_process_status{config="ptp4l.0.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_process_status{config="ptp4l.1.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="phc2sys"} 1
      openshift_ptp_process_status{config="ptp4l.1.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_process_status{config="ptp4l.2.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_process_status{config="ptp4l.3.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_process_status{config="ptp4l.4.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="phc2sys"} 0
      openshift_ptp_process_status{config="ptp4l.4.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      openshift_ptp_process_status{config="ptp4l.5.config",node="helix49.ptp.lab.eng.bos.redhat.com",process="ptp4l"} 1
      

      Expected results:

      old phc2sys metics associated with ptp4l.4.config should not appear in ptp metrics.

      Additional info:

      Workaround: restart ptp linux daemon pod will put the metrics back to a clean state.

              aputtur@redhat.com Aneesh Puttur
              rhn-support-yliu1 Yang Liu
              Yang Liu Yang Liu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: