Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4731

leak after running conformance/serial

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Undefined
    • None
    • 4.12
    • Test Framework
    • None
    • Moderate
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      On Openshift on Openstack CI, we are running conformance serial tests, and the test [sig-cli] oc status can show correct status after switching between projects [apigroup:project.open
      shift.io][apigroup:image.openshift.io][Serial] [Suite:openshift/conformance/serial] is leaking some resources in error status on the cluster that are provoking failures on our automation:

      (shiftstack) [stack@undercloud-0 ~]$ KUBECONFIG=~/.kube/config ./openshift-tests run-test "[sig-cli] oc status can show correct status after switching between projects [apigroup:project.open
      shift.io][apigroup:image.openshift.io][Serial] [Suite:openshift/conformance/serial]"                                                                                                         
      Dec 12 09:42:09.333: INFO: Enabling in-tree volume drivers                                                                                                                                   
      [BeforeEach] TOP-LEVEL                                                                                                                                                                        
        github.com/openshift/origin/test/extended/util/framework.go:1486                                                                                                                           
      [BeforeEach] TOP-LEVEL                                                                                                                                                                        
        github.com/openshift/origin/test/extended/util/framework.go:1486                                                                                                                           
      [BeforeEach] TOP-LEVEL                                                                                                                                                                       
        github.com/openshift/origin/test/extended/util/test.go:58                                                                                                                                   
      [BeforeEach] [sig-cli] oc status                                                                                                                                                             
        github.com/openshift/origin/test/extended/util/client.go:160                                                                                                                                
      STEP: Creating a kubernetes client 12/12/22 09:42:10.19                                                                                                                                      
      [BeforeEach] [sig-cli] oc status                                                                                                                                                              
        github.com/openshift/origin/test/extended/util/client.go:134                                                                                                                               
      Dec 12 09:42:10.609: INFO: configPath is now "/tmp/configfile1329447316"                                                                                                                      
      Dec 12 09:42:10.609: INFO: The user is now "e2e-test-oc-status-qkkkd-user"                                                                                                                    
      Dec 12 09:42:10.609: INFO: Creating project "e2e-test-oc-status-qkkkd"                                                                                                                       
      Dec 12 09:42:10.780: INFO: Waiting on permissions in project "e2e-test-oc-status-qkkkd" ...                                                                                                  
      Dec 12 09:42:10.854: INFO: Waiting for ServiceAccount "default" to be provisioned...                                                                                                         
      Dec 12 09:42:10.960: INFO: Waiting for ServiceAccount "deployer" to be provisioned...                                                                                                        
      Dec 12 09:42:11.069: INFO: Waiting for ServiceAccount "builder" to be provisioned...                                                                                                         
      Dec 12 09:42:11.177: INFO: Waiting for RoleBinding "system:image-pullers" to be provisioned...                                                                                               
      Dec 12 09:42:11.185: INFO: Waiting for RoleBinding "system:image-builders" to be provisioned...                                                                                              
      Dec 12 09:42:11.196: INFO: Waiting for RoleBinding "system:deployers" to be provisioned...                                                                                                   
      Dec 12 09:42:11.465: INFO: Project "e2e-test-oc-status-qkkkd" has been fully provisioned.                                                                                                    
      [It] can show correct status after switching between projects [apigroup:project.openshift.io][apigroup:image.openshift.io][Serial] [Suite:openshift/conformance/serial]                      
        github.com/openshift/origin/test/extended/cli/status.go:31                                                                                                                                 
      Dec 12 09:42:11.466: INFO: Running 'oc --namespace=e2e-test-oc-status-qkkkd --kubeconfig=/tmp/configfile1329447316 status --all-namespaces'                                                  
      Dec 12 09:42:11.615: INFO: Running 'oc --namespace=e2e-test-oc-status-qkkkd --kubeconfig=/tmp/configfile1329447316 status -A'                                                                
      STEP: create a new project 12/12/22 09:42:11.811                                                                                                                                             
      Dec 12 09:42:11.812: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 new-project e2e-test-oc-status-qkkkd-project-bar --display-name=my project --description=test project'         
      Now using project "e2e-test-oc-status-qkkkd-project-bar" on server "https://api.ostest.shiftstack.com:6443".                                                                                 
                                                                                                                                                                                                   
          kubectl create deployment hello-node --image=k8s.gcr.io/e2e-test-images/agnhost:2.33 -- /agnhost serve-hostname                                                                          
      Dec 12 09:42:12.197: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 project'                                                                                                       
      STEP: make sure `oc status` does not use "no projects" message if there is a project created 12/12/22 09:42:12.291
      Dec 12 09:42:12.291: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 status'
      STEP: create a second project 12/12/22 09:42:12.459
      Dec 12 09:42:12.459: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 new-project e2e-test-oc-status-qkkkd-project-bar-2 --display-name=my project 2 --description=test project 2'
      Now using project "e2e-test-oc-status-qkkkd-project-bar-2" on server "https://api.ostest.shiftstack.com:6443".
      
      You can add applications to this project with the 'new-app' command. For example, try:
      
          oc new-app rails-postgresql-example
      
      to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:
      
          kubectl create deployment hello-node --image=k8s.gcr.io/e2e-test-images/agnhost:2.33 -- /agnhost serve-hostname
      Dec 12 09:42:12.702: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 project'
      STEP: delete the current project `e2e-test-oc-status-qkkkd-project-bar-2` and make sure `oc status` does not return the "no projects" message since `e2e-test-oc-status-qkkkd-project-bar` sti
      ll exists 12/12/22 09:42:12.805
      Dec 12 09:42:12.806: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 delete project e2e-test-oc-status-qkkkd-project-bar-2'
      Dec 12 09:42:13.060: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 project e2e-test-oc-status-qkkkd-project-bar'
      Now using project "e2e-test-oc-status-qkkkd-project-bar" on server "https://api.ostest.shiftstack.com:6443".
      Dec 12 09:42:13.180: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 delete project e2e-test-oc-status-qkkkd-project-bar'
      project.project.openshift.io "e2e-test-oc-status-qkkkd-project-bar" deleted
      Dec 12 09:42:19.659: INFO: Running 'oc --namespace=e2e-test-oc-status-qkkkd --kubeconfig=/tmp/configfile1329447316 get projects'
      Dec 12 09:42:19.797: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 new-project e2e-test-oc-status-qkkkd-project-status --display-name=my project --description=test project'
      Now using project "e2e-test-oc-status-qkkkd-project-status" on server "https://api.ostest.shiftstack.com:6443".
      
      You can add applications to this project with the 'new-app' command. For example, try:
      
          oc new-app rails-postgresql-example
      
      to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:
      
      
                                                                                                                                                                                                   
      You can add applications to this project with the 'new-app' command. For example, try:                                                                                                        
                                                                                                                                                                                                   
          oc new-app rails-postgresql-example                                                                                                                                                       
                                                                                                                                                                                                   
      to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application:                                                                                         
      
          kubectl create deployment hello-node --image=k8s.gcr.io/e2e-test-images/agnhost:2.33 -- /agnhost serve-hostname
      STEP: verify jobs are showing in status 12/12/22 09:42:20.152
      Dec 12 09:42:20.152: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 create job pi --image=image-registry.openshift-image-registry.svc:5000/openshift/tools:latest -- perl -Mbignum=b
      pi -wle 'print bpi(2000)''
      Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "pi" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (
      container "pi" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "pi" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container
      "pi" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
      job.batch/pi created
      Dec 12 09:42:20.285: INFO: Running 'oc --kubeconfig=/tmp/configfile1329447316 status'
      [AfterEach] [sig-cli] oc status
        github.com/openshift/origin/test/extended/util/client.go:158
      Dec 12 09:42:20.483: INFO: Deleted {user.openshift.io/v1, Resource=users  e2e-test-oc-status-qkkkd-user}, err: <nil>
      Dec 12 09:42:20.497: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthclients  e2e-client-e2e-test-oc-status-qkkkd}, err: <nil>
      Dec 12 09:42:20.511: INFO: Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens  sha256~0QodLJemrdDv5q4xiYIvaUD4-E0JK4s6JN3BXnEkLto}, err: <nil>
      [AfterEach] [sig-cli] oc status
        github.com/openshift/origin/test/extended/util/client.go:159
      STEP: Destroying namespace "e2e-test-oc-status-qkkkd" for this suite. 12/12/22 09:42:20.511
      

      A namespace with Error pods remains on the cluster:

      $ oc get pods -n e2e-test-oc-status-qkkkd-project-status
      NAME       READY   STATUS   RESTARTS   AGE
      pi-85cpc   0/1     Error    0          8s
      pi-t9qlf   0/1     Error    0          13s
      pi-xd5dd   0/1     Error    0          3s
      
      $ oc logs -n e2e-test-oc-status-qkkkd-project-status pi-85cpc
      Can't locate bignum.pm in @INC (you may need to install the bignum module) (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5).
      BEGIN failed--compilation aborted.
                                                                                                                                                    
      

      Version-Release number of selected component (if applicable):

      $ git branch
      * release-4.12
      $ git log | head
      commit 7c39a7d52e43c54a0cef2cf83900e48c9ab73009
      Author: OpenShift Merge Robot <openshift-merge-robot@users.noreply.github.com>
      Date:   Mon Dec 5 13:28:17 2022 -0500
      
          Merge pull request #27588 from vrutkovs/4.12-bump-k8s
          
          OCPBUGS-2927: [release-4.12] Bump kubernetes to latest release-4.12
      
      

      How reproducible:

      Always
      

      Steps to Reproduce:

      1. Run conformance tests case:
      
      [sig-cli] oc status can show correct status after switching between projects [apigroup:project.open
      shift.io][apigroup:image.openshift.io][Serial] [Suite:openshift/conformance/serial]
      
      2. Check the namespace generated and the remaining pods in error.
      3.
      

      Actual results:

      Test passing with leaks that are failing post-checks on our automation.
      

      Expected results:

      Test passing without leaks
      

      Additional info:

      N/A
      

      Attachments

        Activity

          People

            aguclu@redhat.com Arda Guclu
            rlobillo Ramón Lobillo
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: