Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-17681

clusteroperator/network is degraded because DaemonSet /openshift-multus/dhcp-daemon rollout is not making progress - pod dhcp-daemon-* is in CrashLoopBackOff State

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done-Errata
    • Undefined
    • 4.14.0
    • 4.13, 4.14
    • Networking / multus
    • CNF Network Sprint 240, CNF Network Sprint 241
    • 2
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      clusteroperator/network is degraded after running
      
          FEATURES_ENVIRONMENT="ci" make feature-deploy-on-ci
      
      from openshift-kni/cnf-features-deploy against IPI clusters with OCP 4.13 and 4.14 in CI jobs from Telco 5G DevOps/CI.
      
      Details for a 4.13 job:
      
      https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/42141/rehearse-42141-periodic-ci-openshift-release-master-nightly-4.13-e2e-telco5g/1689935408508440576
      
      Details for a 4.14 job:
      
      https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/42141/rehearse-42141-periodic-ci-openshift-release-master-nightly-4.14-e2e-telco5g/1689935408541995008
      
      For example, got to artifacts/e2e-telco5g/telco5g-gather-pao/build-log.txt and it will report:
      
      Error from server (BadRequest): container "container-00" in pod "cnfdu5-worker-0-debug" is waiting to start: ContainerCreating
      Running gather-pao for T5CI_VERSION=4.13
      Running for CNF_BRANCH=master
      Running PAO must-gather with tag pao_mg_tag=4.12
      [must-gather      ] OUT Using must-gather plug-in image: quay.io/openshift-kni/performance-addon-operator-must-gather:4.12-snapshot
      When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information:
      ClusterID: 60503edf-ecc6-48f7-b6a6-f4dc34842803
      ClusterVersion: Stable at "4.13.0-0.nightly-2023-08-10-021434"
      ClusterOperators:
      	clusteroperator/network is degraded because DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - pod dhcp-daemon-7lmlq is in CrashLoopBackOff State
      DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - pod dhcp-daemon-95tzb is in CrashLoopBackOff State
      DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - pod dhcp-daemon-hfxkd is in CrashLoopBackOff State
      DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - pod dhcp-daemon-mhwtp is in CrashLoopBackOff State
      DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - pod dhcp-daemon-q7gfb is in CrashLoopBackOff State
      DaemonSet "/openshift-multus/dhcp-daemon" rollout is not making progress - last change 2023-08-11T10:54:10Z
      
      

      Version-Release number of selected component (if applicable):

      branch release-4.13 from https://github.com/openshift-kni/cnf-features-deploy.git for OCP 4.13
      branch master from https://github.com/openshift-kni/cnf-features-deploy.git for OCP 4.14
      

      How reproducible:

      Always.
      
      

      Steps to Reproduce:

      1. Install OCP 4.13 or OCP 4.14 with IPI on 3x masters, 2x workers.
      2. Clone https://github.com/openshift-kni/cnf-features-deploy.git
      3. FEATURES_ENVIRONMENT="ci" make feature-deploy-on-ci
      4. oc wait nodes --all --for=condition=Ready=true --timeout=10m
      5. oc wait clusteroperators --all --for=condition=Progressing=false --timeout=10m
      

      Actual results:

      See above.
      

      Expected results:

      All clusteroperators have finished progressing.
      

      Additional info:

      Without 'FEATURES_ENVIRONMENT="ci" make feature-deploy-on-ci' the steps to reproduce above work as expected.
      

      Attachments

        Issue Links

          Activity

            People

              carlosgoncalves Carlos Goncalves
              jmeng@redhat.com Jakob Meng
              Weibin Liang Weibin Liang
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: