Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1788

oc adm inspect --rotated-pod-logs not working properly for static pods

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • None
    • 4.10.0
    • oc
    • None
    • Moderate
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      This is a clone of issue OCPBUGS-613. The following is the description of the original issue:

      Description of problem:

      The path used by --rotated-pod-logs to gather the rotated pod logs from /var/log/pods node folder via /api/v1/nodes/${NODE}/proxy/logs/${LOG_PATH} is only valid for regular pods but not for static pods.

      The main problem is that, while normal pods have their rotated logs at this /var/log/pods/${POD_NAME}_${POD_UID_IN_API}/${CONTAINER_NAME}, static pods have them at /var/log/pods/${POD_NAME}_${CONFIG_HASH}/${CONTAINER_NAME} because the UID cannot be known at the time that the static pod is born (because static pods are created by kubelet before registering them in the kube-apiserver, and UID is assigned by the kube-apiserver).

      The visible results of that are:

      • Spurious errors of not found resources related to the pods.
      • Rotated pod logs are not gathered even if present.

      Version-Release number of selected component (if applicable):

      4.10

      How reproducible:

      Always if there are static pods.

      Steps to Reproduce:

      1. oc adm inspect --rotated-pod-logs ns/openshift-etcd (or any other project with static pods).

      Actual results:

      • Rotated pods not gathered.
      • Errors like these
        error: errors occurred while gathering data:
            one or more errors occurred while gathering pod-specific data for namespace: openshift-etcd
        
            [one or more errors occurred while gathering container data for pod etcd-master-0.example.net:
        
            the server could not find the requested resource, one or more errors occurred while gathering container data for pod etcd-master-1.example.net:
        
            the server could not find the requested resource, one or more errors occurred while gathering container data for pod etcd-master-2.example.net:
        
            the server could not find the requested resource]
        

      Expected results:

      No errors like the ones above and rotated pod logs to be gathered, if present.

      Additional info:

      Despite being marked as experimental, this --rotated-pod-logs is used in must-gather, so this issue can be easily reproduced by just running a default must-gather. I focused on bare oc adm inspect reproducers for simplicity.

      Attachments

        Issue Links

          Activity

            People

              jchaloup@redhat.com Jan Chaloupka
              openshift-crt-jira-prow OpenShift Prow Bot
              Rama Kasturi Narra Rama Kasturi Narra
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: