Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-57036

Downstream slave role LISTENING when upstream master is down

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • None
    • None
    • None
    • In Progress
    • Release Note Not Required
    • N/A
    • None
    • None
    • None
    • None

      This is a clone of issue OCPBUGS-54984. The following is the description of the original issue:

      Description of problem:
      BC/OC test environment has 2 HA master interfaces.  When we take down both HA interfaces, the downstream slave remains locked, the clock class remains locked and the downstream slave interface role changes from slave to listening. 

      Interface Setup:
      ens3f2 (bc1) is active HA interface, slave to source clock
      ens2f0 (bc2) is inactive HA interface, slave to source clock
      ens3f1 is master to downstream slave ens1f0 (slave-bc1)
      ens2f1 is master to downstream slave ens1f1 (slave-bc2)

      When HA masters are taken down, the downstream slave clock state and ptp clock class gauges show locked status:

      # HELP openshift_ptp_clock_class 6 = Locked, 7 = PRC unlocked in-spec, 52/187 = PRC unlocked out-of-spec, 248 = Default, 255 = Slave Only Clock
      # TYPE openshift_ptp_clock_class gauge
      openshift_ptp_clock_class{node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 6
      # HELP openshift_ptp_clock_state 0 = FREERUN, 1 = LOCKED, 2 = HOLDOVER
      # TYPE openshift_ptp_clock_state gauge
      openshift_ptp_clock_state{iface="CLOCK_REALTIME",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="phc2sys"} 0
      openshift_ptp_clock_state{iface="ens1fx",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 1
      openshift_ptp_clock_state{iface="ens2fx",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 0
      openshift_ptp_clock_state{iface="ens3fx",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 0
      

      At the same time, the interface role for the downstream slave is shown as listening, and the master interfaces are master:

      # HELP openshift_ptp_interface_role 0 = PASSIVE, 1 = SLAVE, 2 = MASTER, 3 = FAULTY, 4 = UNKNOWN, 5 = LISTENING
      # TYPE openshift_ptp_interface_role gauge
      openshift_ptp_interface_role{iface="ens1f0",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 5
      openshift_ptp_interface_role{iface="ens1f1",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 5
      openshift_ptp_interface_role{iface="ens2f0",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 3
      openshift_ptp_interface_role{iface="ens2f1",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 2
      openshift_ptp_interface_role{iface="ens3f1",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 2
      openshift_ptp_interface_role{iface="ens3f2",node="helix73.telcoqe.eng.rdu2.dc.redhat.com",process="ptp4l"} 3

      Version-Release number of selected component (if applicable):

      4.19

      How reproducible:

      100%    

      Steps to Reproduce:

          1. Deploy SNO with ptp HA master interfaces and downstream slaves on two ports.
          2. Take down both HA master interfaces
          3. Watch metrics for clock class, clock state and role.
          

      Actual results:

      Downstream slaves remain as LOCKED state and LISTENING role

      Expected results:

      When master is down, state and role should change to reflect down status of master

      Additional info:

          

              aputtur@redhat.com Aneesh Puttur
              openshift-crt-jira-prow OpenShift Prow Bot
              None
              None
              Bonnie Block Bonnie Block
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: