Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1497

[4.11.z] oc adm inspect --rotated-pod-logs not working properly for static pods

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 4.11.z
    • 4.10.0
    • oc
    • None
    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      The path used by --rotated-pod-logs to gather the rotated pod logs from /var/log/pods node folder via /api/v1/nodes/${NODE}/proxy/logs/${LOG_PATH} is only valid for regular pods but not for static pods.

      The main problem is that, while normal pods have their rotated logs at this /var/log/pods/${POD_NAME}_${POD_UID_IN_API}/${CONTAINER_NAME}, static pods have them at /var/log/pods/${POD_NAME}_${CONFIG_HASH}/${CONTAINER_NAME} because the UID cannot be known at the time that the static pod is born (because static pods are created by kubelet before registering them in the kube-apiserver, and UID is assigned by the kube-apiserver).

      The visible results of that are:

      • Spurious errors of not found resources related to the pods.
      • Rotated pod logs are not gathered even if present.

      Version-Release number of selected component (if applicable):

      4.10

      How reproducible:

      Always if there are static pods.

      Steps to Reproduce:

      1. oc adm inspect --rotated-pod-logs ns/openshift-etcd (or any other project with static pods).

      Actual results:

      • Rotated pods not gathered.
      • Errors like these
        error: errors occurred while gathering data:
            one or more errors occurred while gathering pod-specific data for namespace: openshift-etcd
        
            [one or more errors occurred while gathering container data for pod etcd-master-0.example.net:
        
            the server could not find the requested resource, one or more errors occurred while gathering container data for pod etcd-master-1.example.net:
        
            the server could not find the requested resource, one or more errors occurred while gathering container data for pod etcd-master-2.example.net:
        
            the server could not find the requested resource]
        

      Expected results:

      No errors like the ones above and rotated pod logs to be gathered, if present.

      Additional info:

      Despite being marked as experimental, this --rotated-pod-logs is used in must-gather, so this issue can be easily reproduced by just running a default must-gather. I focused on bare oc adm inspect reproducers for simplicity.

            jchaloup@redhat.com Jan Chaloupka
            rhn-support-palonsor Pablo Alonso Rodriguez
            ying zhou ying zhou
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: