Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-22816

ptp4l process restarted unexpected with dual NIC boundary clocks configured

XMLWordPrintable

    • No
    • False
    • Hide

      None

      Show
      None
    • Hide
      11/17: Waiting on QE to complete the backport process with cherry-pick label and verify
      11/06: Waiting for 4.15 to be verified
      10/30: Waiting for 4.15 to be verified
      10/19: Created a fix ,customer is running dev version of the fix on their lab. Waiting on results .
      10/12: Deployed dev version on labs with debug ; so far unable to recreate; still working on this
      10/9 : There was no carrier loss as well so a very limited impact. So far fresh applied pods doesn't have this behavior. Waiting to recreate this by delete running pods .
      Show
      11/17: Waiting on QE to complete the backport process with cherry-pick label and verify 11/06: Waiting for 4.15 to be verified 10/30: Waiting for 4.15 to be verified 10/19: Created a fix ,customer is running dev version of the fix on their lab. Waiting on results . 10/12: Deployed dev version on labs with debug ; so far unable to recreate; still working on this 10/9 : There was no carrier loss as well so a very limited impact. So far fresh applied pods doesn't have this behavior. Waiting to recreate this by delete running pods .

      This is a clone of issue OCPBUGS-22400. The following is the description of the original issue:

      This is a clone of issue OCPBUGS-22325. The following is the description of the original issue:

      Description of problem:

      ptp4l process restarted unexpected with dual NIC boundary clocks configured

      Version-Release number of selected component (if applicable):

      4.12.z, may exist in other version as well like 4.13.z and 4.14

      How reproducible:

      Always, but may need to wait for hours 

      Steps to Reproduce:

      1. Enable dual NIC boundary clocks on a SNO cluster
      2. Check linuxptp-daemon pod logs(linuxptp-daemon-container) 

      Actual results:

      Seeing the ptp4l process restarted unexpectedly

      Expected results:

      ptp4l process should not be restarted if there is no ptpconfig changes

      Additional info:

      ptpconfig: 
      boundary-clock1: https://drive.google.com/file/d/1VpGAqs5foML_iNnjAARTew8qLIkVZlvw/view?usp=drive_link
      boundary-clock2: https://drive.google.com/file/d/17DeV-vXV3i0PWBEg8PKVAI2cYnSAYJn0/view?usp=drive_link
      ptp pod logs: https://drive.google.com/file/d/1S6c5l7nk8TR-Di28EnHvmT5VNRNUST50/view?usp=drive_link 
      
      ptpconfigs were applied at around:2023-10-05T15:57/2023-10-05T15:58
      
      ptp4l restarted at 2023-10-05T15:58, 2023-10-05T15:59 and 2023-10-05T18:24
      
      3.log.20231005-221656:2023-10-05T15:58:08.921143892+00:00 stderr F I1005 15:58:08.921094   12963 daemon.go:165] stopping process.... &{name:phc2sys ifaces:[ens1f1 ens1f1] ptp4lSocketPath: ptp4lConfigPath: configName:ptp4l.0.config exitCh:0xc0001f6060 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc00040a000 parentClockClass:0 pmcCheck:false}
      3.log.20231005-221656:2023-10-05T15:58:08.921154740+00:00 stderr F I1005 15:58:08.921140   12963 daemon.go:540] Stopping phc2sys...
      3.log.20231005-221656:2023-10-05T15:58:09.921778436+00:00 stderr F I1005 15:58:09.921709   12963 daemon.go:165] stopping process.... &{name:ptp4l ifaces:[ens1f1 ens1f1] ptp4lSocketPath:/var/run/ptp4l.0.socket ptp4lConfigPath:/var/run/ptp4l.0.config configName:ptp4l.0.config exitCh:0xc0001f6120 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc00040a840 parentClockClass:0 pmcCheck:true}
      3.log.20231005-221656:2023-10-05T15:58:09.921778436+00:00 stderr F I1005 15:58:09.921731   12963 daemon.go:540] Stopping ptp4l...
      3.log.20231005-221656:2023-10-05T15:58:11.949251030+00:00 stderr F I1005 15:58:11.949211   12963 daemon.go:436] Starting phc2sys...
      3.log.20231005-221656:2023-10-05T15:58:12.949947690+00:00 stderr F I1005 15:58:12.949905   12963 daemon.go:436] Starting ptp4l...
      3.log.20231005-221656:2023-10-05T15:59:38.920785353+00:00 stderr F I1005 15:59:38.920753   12963 daemon.go:165] stopping process.... &{name:phc2sys ifaces:[ens1f0 ens1f1] ptp4lSocketPath: ptp4lConfigPath: configName:ptp4l.0.config exitCh:0xc0001e4c00 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc0002fc000 parentClockClass:0 pmcCheck:false}
      3.log.20231005-221656:2023-10-05T15:59:38.920833763+00:00 stderr F I1005 15:59:38.920782   12963 daemon.go:540] Stopping phc2sys...
      3.log.20231005-221656:2023-10-05T15:59:39.922458635+00:00 stderr F I1005 15:59:39.922407   12963 daemon.go:165] stopping process.... &{name:ptp4l ifaces:[ens1f0 ens1f1] ptp4lSocketPath:/var/run/ptp4l.0.socket ptp4lConfigPath:/var/run/ptp4l.0.config configName:ptp4l.0.config exitCh:0xc0001e4cc0 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc0002fc2c0 parentClockClass:0 pmcCheck:true}
      3.log.20231005-221656:2023-10-05T15:59:39.922458635+00:00 stderr F I1005 15:59:39.922448   12963 daemon.go:540] Stopping ptp4l...
      3.log.20231005-221656:2023-10-05T15:59:40.958442517+00:00 stderr F I1005 15:59:40.958364   12963 daemon.go:249] applyNodePtpProfile: not starting phc2sys, phc2sysOpts is empty
      3.log.20231005-221656:2023-10-05T15:59:41.959080731+00:00 stderr F I1005 15:59:41.959031   12963 daemon.go:436] Starting phc2sys...
      3.log.20231005-221656:2023-10-05T15:59:42.959164607+00:00 stderr F I1005 15:59:42.959116   12963 daemon.go:436] Starting ptp4l...
      3.log.20231005-221656:2023-10-05T15:59:43.959767582+00:00 stderr F I1005 15:59:43.959717   12963 daemon.go:436] Starting ptp4l...
      3.log.20231005-221656:2023-10-05T18:24:38.924673796+00:00 stderr F I1005 18:24:38.924598   12963 daemon.go:165] stopping process.... &{name:phc2sys ifaces:[ens1f0 ens1f1] ptp4lSocketPath: ptp4lConfigPath: configName:ptp4l.0.config exitCh:0xc00049ec60 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc0001c6420 parentClockClass:0 pmcCheck:false}
      3.log.20231005-221656:2023-10-05T18:24:38.924732230+00:00 stderr F I1005 18:24:38.924699   12963 daemon.go:540] Stopping phc2sys...
      3.log.20231005-221656:2023-10-05T18:24:39.925391537+00:00 stderr F I1005 18:24:39.925344   12963 daemon.go:165] stopping process.... &{name:ptp4l ifaces:[ens1f0 ens1f1] ptp4lSocketPath:/var/run/ptp4l.0.socket ptp4lConfigPath:/var/run/ptp4l.0.config configName:ptp4l.0.config exitCh:0xc00049ed20 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc0001c6580 parentClockClass:0 pmcCheck:true}
      3.log.20231005-221656:2023-10-05T18:24:39.925391537+00:00 stderr F I1005 18:24:39.925376   12963 daemon.go:540] Stopping ptp4l...
      3.log.20231005-221656:2023-10-05T18:24:40.966311778+00:00 stderr F I1005 18:24:40.966248   12963 daemon.go:165] stopping process.... &{name:ptp4l ifaces:[ens2f0 ens2f1] ptp4lSocketPath:/var/run/ptp4l.1.socket ptp4lConfigPath:/var/run/ptp4l.1.config configName:ptp4l.1.config exitCh:0xc00049ede0 execMutex:{state:0 sema:0} stopped:false logFilterRegex:^$ cmd:0xc0001c66e0 parentClockClass:0 pmcCheck:true}
      3.log.20231005-221656:2023-10-05T18:24:40.966311778+00:00 stderr F I1005 18:24:40.966269   12963 daemon.go:540] Stopping ptp4l...
      3.log.20231005-221656:2023-10-05T18:24:42.001003751+00:00 stderr F I1005 18:24:42.000962   12963 daemon.go:249] applyNodePtpProfile: not starting phc2sys, phc2sysOpts is empty
      3.log.20231005-221656:2023-10-05T18:24:43.001555831+00:00 stderr F I1005 18:24:43.001516   12963 daemon.go:436] Starting ptp4l...
      3.log.20231005-221656:2023-10-05T18:24:44.002265775+00:00 stderr F I1005 18:24:44.002232   12963 daemon.go:436] Starting phc2sys...
      3.log.20231005-221656:2023-10-05T18:24:45.003463725+00:00 stderr F I1005 18:24:45.003316   12963 daemon.go:436] Starting ptp4l...
      
      
      

       

              aputtur@redhat.com Aneesh Puttur
              openshift-crt-jira-prow OpenShift Prow Bot
              Bonnie Block Bonnie Block
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: