Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-22999

podman image prune -f "podman-auto-update.service" break some pods

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • 4.12.z
    • Containers
    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Periodically sudo privileges are lost for one of our pod(openstackclient). Something is removing the lowerdir soft link for this container. 
      
      Customer follow this KB(https://access.redhat.com/solutions/5843611) to take the backup of etcd daily at midnight. But periodically they have lost the sudo privileges inside the pod. 

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Customer installed 3 OCP cluster and they reported this issue on all of the clusters at different time. As per them, this issue trigger in around 3 weeks. Customer deleted the pod and it solves their issue, but it trigger again in around 3 weeks.
      

      Steps to Reproduce:

      We believe the procedure described in this KB(https://access.redhat.com/solutions/5843611) for etcd-backup will trigger this issue.
      
      

      Actual results:

      $ oc rsh openstackclient
      Defaulted container "openstackclient" out of: openstackclient, init-0 (init)
      sh-4.4$ whoami
      cloud-admin
      sh-4.4$ id
      uid=42401(cloud-admin) gid=42401(cloud-admin) groups=42401(cloud-admin)
      sh-4.4$ sudo -l
      
      We trust you have received the usual lecture from the local System
      Administrator. It usually boils down to these three things:
      
          #1) Respect the privacy of others.
          #2) Think before you type.
          #3) With great power comes great responsibility.

      Expected results:

      $ oc rsh openstackclient
      Defaulted container "openstackclient" out of: openstackclient, init-0 (init)
      sh-4.4$ whoami
      cloud-admin
      sh-4.4$ id
      uid=42401(cloud-admin) gid=42401(cloud-admin) groups=42401(cloud-admin)
      sh-4.4$ sudo -l
      Runas and Command-specific defaults for cloud-admin:
          Defaults!/usr/bin/run-validation !requiretty
      
      User cloud-admin may run the following commands on openstackclient:
          (ALL) NOPASSWD: ALL

      Additional info:

      We have some previous discussions on this topic on this BZ (https://bugzilla.redhat.com/show_bug.cgi?id=2239768#c5) where we think it may be due to any periodic execution of command `podman system prune -a -f` which was not aware of some of the users of images that `cri-o` used, so it removed images that were actually still needed. However, the customer confirmed they are not aware of the execution of this command `podman system prune -a -f`
      
      You can find the summary of our analysis on this case(03617802). Refer comment#59 (Nov 7)

              pehunt@redhat.com Peter Hunt
              rh-ee-anaim Ahmed Naim
              David Darrah David Darrah (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: