Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-65770

Router pods not able to be started due to permission issue

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.16
    • Networking / router
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

          Router pods not able to be started due to permission issue:
       sed: couldn't open temporary file /var/lib/haproxy/conf/sedXtmZTV: Permission denied
      Unable to allow router pods with scc of hostnetwork and correct SELinux to write until applying temporary workaround. 

      Version-Release number of selected component (if applicable):

          4.16.51

      How reproducible:

      Unsure why it happened, but it was having correct SCC of hostnetwork and correct SELinux configuration, and after redeployment of pods it was not working, after it was modified the namespace UID-range and supplemental-groups momentarily

      Issue:

      LAST SEEN   TYPE      REASON    OBJECT                                MESSAGE
      1h7m        Warning   BackOff   pod/router-default-xxx-xxx   Back-off restarting failed container router in pod router-default-xxx-xxx-ingress(39160e3e-86c8-4020-ab6a-738454b170d1)
      
      lastState:
            terminated:
              containerID: cri-o://50bab4de7f324017803497982c9a24e9ec805901264c652aa2ea9b6ee8edbc20
              exitCode: 4
              finishedAt: "2025-11-18T10:13:51Z"
              message: |
                sed: couldn't open temporary file /var/lib/haproxy/conf/sedXtmZTV: Permission denied
              reason: Error
              startedAt: "2025-11-18T10:13:51Z"
          name: router
      
      > oc debug pod/router-default-xxx-xx -n openshift-ingress
      Defaulting container name to router.
      Use 'oc describe pod/router-default-xx-xx-debug-rgxts -n openshift-ingress' to see all of the containers in this pod.
      
      Starting pod/router-default-xx-xx-debug-rgxts, command was: /bin/sh -c sed -i 's~httponly~{{or (index $cfg.Annotations "router.camba.at/cookie_options") "httponly"}}~' $TEMPLATE_FILE && exec /usr/bin/openshift-router --v=2
      Pod IP: 192.168.20.155
      If you don't see a command prompt, try pressing enter.
      sh-5.1$ id
      uid=1000630000(1000630000) gid=0(root) groups=0(root),1000630000
      sh-5.1$ ls -Zld /var/lib/haproxy/conf
      drwxr-xr-x. 1 root root system_u:object_r:container_file_t:s0:c15,c25 26 Nov 19 08:47 /var/lib/haproxy/conf
      sh-5.1$ touch /var/lib/haproxy/conf/test_write
      touch: cannot touch '/var/lib/haproxy/conf/test_write': Permission denied
      
      

      Actual results:

      To fix it, we did change the ns openshift-ingress as per following:
      openshift.io/sa.scc.supplemental-groups: 0/10000  
      openshift.io/sa.scc.uid-range: 0/10000
      After that, rolling out the router pods were able to be started.         

      Expected results:

          Not having to use a workaround, as this has never been reported as an issue having correct SCC and correct SELinux.

      Additional info:

      Must-gather in 04221290    

              alebedev@redhat.com Andrey Lebedev
              rhn-support-dahernan David Hernandez Fernandez
              None
              None
              Shudi Li Shudi Li
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: