Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-45205

[sig-api-machinery][Feature:APIServer][Late] [It] kubelet terminates kube-apiserver gracefully extended [Suite:openshift/conformance/parallel] is perma failing

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.19.0
    • kube-apiserver
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

       [sig-api-machinery][Feature:APIServer][Late] kubelet terminates kube-apiserver gracefully extended [Suite:openshift/conformance/parallel]
        github.com/openshift/origin/test/extended/apiserver/graceful_termination.go:70
          STEP: Creating a kubernetes client @ 11/28/24 21:29:47.506
        I1128 21:29:47.507893  112195 discovery.go:214] Invalidating discovery information
        I1128 21:29:47.654828 112195 client.go:275] configPath is now "/tmp/configfile1322929535"
        I1128 21:29:47.654870 112195 client.go:324] The user is now "e2e-test-monitoring-collection-profiles-kl2f7-user"
        I1128 21:29:47.654880 112195 client.go:326] Creating project "e2e-test-monitoring-collection-profiles-kl2f7"
        I1128 21:29:47.809525 112195 client.go:334] Waiting on permissions in project "e2e-test-monitoring-collection-profiles-kl2f7" ...
        I1128 21:29:47.877389 112195 client.go:363] DeploymentConfig capability is enabled, adding 'deployer' SA to the list of default SAs
        I1128 21:29:47.944955 112195 client.go:378] Waiting for ServiceAccount "default" to be provisioned...
        I1128 21:29:48.071367 112195 client.go:378] Waiting for ServiceAccount "builder" to be provisioned...
        I1128 21:29:48.273655 112195 client.go:378] Waiting for ServiceAccount "deployer" to be provisioned...
        I1128 21:29:48.421700 112195 client.go:388] Waiting for RoleBinding "system:image-pullers" to be provisioned...
        I1128 21:29:48.421789  112195 reflector.go:305] Starting reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.421804  112195 reflector.go:341] Listing and watching *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.423382  112195 reflector.go:368] Caches populated for *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.917651  112195 reflector.go:311] Stopping reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.917673 112195 client.go:388] Waiting for RoleBinding "system:image-builders" to be provisioned...
        I1128 21:29:48.917724  112195 reflector.go:305] Starting reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.917737  112195 reflector.go:341] Listing and watching *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.919555  112195 reflector.go:368] Caches populated for *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.921172  112195 reflector.go:311] Stopping reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.921188 112195 client.go:388] Waiting for RoleBinding "system:deployers" to be provisioned...
        I1128 21:29:48.921221  112195 reflector.go:305] Starting reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.921231  112195 reflector.go:341] Listing and watching *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.922856  112195 reflector.go:368] Caches populated for *v1.RoleBinding from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:48.924269  112195 reflector.go:311] Stopping reflector *v1.RoleBinding (0s) from k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243
        I1128 21:29:49.178341 112195 client.go:421] Project "e2e-test-monitoring-collection-profiles-kl2f7" has been fully provisioned.
          STEP: Creating a kubernetes client @ 11/28/24 21:29:49.178
        I1128 21:29:49.179214  112195 discovery.go:214] Invalidating discovery information
          STEP: Getting log files for kube-apiserver on master: ip-10-0-54-155.ec2.internal @ 11/28/24 21:29:49.189
        I1128 21:29:49.189260 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-54-155.ec2.internal --path=kube-apiserver/'
          STEP: Getting and processing termination.log file for kube-apiserver on master: ip-10-0-54-155.ec2.internal @ 11/28/24 21:29:49.282
        I1128 21:29:49.283096 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-54-155.ec2.internal --path=kube-apiserver/termination.log'
          STEP: Getting log files for kube-apiserver on master: ip-10-0-58-220.ec2.internal @ 11/28/24 21:29:49.38
        I1128 21:29:49.380339 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-58-220.ec2.internal --path=kube-apiserver/'
          STEP: Getting and processing termination.log file for kube-apiserver on master: ip-10-0-58-220.ec2.internal @ 11/28/24 21:29:49.475
        I1128 21:29:49.475213 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-58-220.ec2.internal --path=kube-apiserver/termination.log'
          STEP: Getting log files for kube-apiserver on master: ip-10-0-90-35.ec2.internal @ 11/28/24 21:29:50.205
        I1128 21:29:50.205378 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-90-35.ec2.internal --path=kube-apiserver/'
          STEP: Getting and processing termination.log file for kube-apiserver on master: ip-10-0-90-35.ec2.internal @ 11/28/24 21:29:50.303
        I1128 21:29:50.303380 112195 client.go:934] Running 'oc --kubeconfig=/tmp/kubeconfig-2702259231 adm node-logs ip-10-0-90-35.ec2.internal --path=kube-apiserver/termination.log'
          [FAILED] in [It] - github.com/openshift/origin/test/extended/apiserver/graceful_termination.go:99 @ 11/28/24 21:29:50.413
          STEP: Collecting events from namespace "e2e-test-monitoring-collection-profiles-kl2f7". @ 11/28/24 21:29:50.414
          STEP: Found 0 events. @ 11/28/24 21:29:50.426
        I1128 21:29:50.442577 112195 resource.go:168] POD  NODE  PHASE  GRACE  CONDITIONS
        I1128 21:29:50.442619 112195 resource.go:178] 
        I1128 21:29:50.460559 112195 dump.go:81] skipping dumping cluster info - cluster too large
        ...
          STEP: Destroying namespace "e2e-test-monitoring-collection-profiles-kl2f7" for this suite. @ 11/28/24 21:29:50.492
        • [FAILED] [3.009 seconds]
        [sig-api-machinery][Feature:APIServer][Late] [It] kubelet terminates kube-apiserver gracefully extended [Suite:openshift/conformance/parallel]
        github.com/openshift/origin/test/extended/apiserver/graceful_termination.go:70
      
          [FAILED] The following API Servers weren't gracefully terminated: 
           kube-apiserver on node ip-10-0-58-220.ec2.internal wasn't gracefully terminated, reason: Previous pod kube-apiserver-ip-10-0-58-220.ec2.internal started at 2024-11-28 20:07:59.664221609 +0000 UTC did not terminate gracefully
           kube-apiserver on node ip-10-0-90-35.ec2.internal wasn't gracefully terminated, reason: Previous pod kube-apiserver-ip-10-0-90-35.ec2.internal started at 2024-11-28 20:24:50.339782962 +0000 UTC did not terminate gracefully
          In [It] at: github.com/openshift/origin/test/extended/apiserver/graceful_termination.go:99 @ 11/28/24 21:29:50.413    

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          

      Steps to Reproduce:

          1.
          2.
          3.
          

      Actual results:

          

      Expected results:

          

      Additional info:

      Job history: https://prow.ci.openshift.org/job-history/gs/test-platform-results/pr-logs/directory/pull-ci-openshift-cluster-network-operator-master-e2e-ovn-ipsec-step-registry
      
      e2e log: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-network-operator/2527/pull-ci-openshift-cluster-network-operator-master-e2e-ovn-ipsec-step-registry/1862197242052481024/artifacts/e2e-ovn-ipsec-step-registry/openshift-e2e-test/artifacts/e2e.log
      
      

       

              Unassigned Unassigned
              pepalani@redhat.com Periyasamy Palanisamy
              None
              None
              Wen Wang Wen Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated: